Clustering Deviations for Black Box Regression Testing of Database Applications

Erik Rogstad, Lionel C. Briand

Research output: Contribution to journalArticlepeer-review

Abstract

Regression tests often result in many deviations (differences between two system versions), either due to changes or regression faults. For the tester to analyze such deviations efficiently, it would be helpful to accurately group them, such that each group contains deviations representing one unique change or regression fault. Because it is unlikely that a general solution to the above problem can be found, we focus our work on a common type of software system: database applications. We investigate the use of clustering, based on database manipulations and test specifications (from test models), to group regression test deviations according to the faults or changes causing them. We also propose assessment criteria based on the concept of entropy to compare alternative clustering strategies. To validate our approach, we ran a large scale industrial case study, and our results show that our clustering approach can indeed serve as an accurate strategy for grouping regression test deviations. Among the four test campaigns assessed, deviations were clustered perfectly for two of them, while for the other two, the clusters were all homogenous. Our analysis suggests that this approach can significantly reduce the effort spent by testers in analyzing regression test deviations, increase their level of confidence, and therefore make regression testing more scalable.

Original languageEnglish
Article number7123685
Pages (from-to)4-18
Number of pages15
JournalIEEE Transactions on Reliability
Volume65
Issue number1
DOIs
Publication statusPublished - 1 Mar 2016
Externally publishedYes

Keywords

  • clustering
  • regression test deviations
  • Software regression testing
  • test analysis

Fingerprint

Dive into the research topics of 'Clustering Deviations for Black Box Regression Testing of Database Applications'. Together they form a unique fingerprint.

Cite this