TY - GEN
T1 - Empirical investigation of the effects of test suite properties on similarity-based test case selection
AU - Hemmati, Hadi
AU - Arcuri, Andrea
AU - Briand, Lionel
PY - 2011
Y1 - 2011
N2 - Our experience with applying model-based testing on industrial systems showed that the generated test suites are often too large and costly to execute given project deadlines and the limited resources for system testing on real platforms. In such industrial contexts, it is often the case that only a small subset of test cases can be run. In previous work, we proposed novel test case selection techniques that minimize the similarities among selected test cases and outperforms other selection alternatives. In this paper, our goal is to gain insights into why and under which conditions similarity-based selection techniques, and in particular our approach, can be expected to work. We investigate the properties of test suites with respect to similarities among fault revealing test cases. We thus identify the ideal situation in which a similarity-based selection works best, which is useful for devising more effective similarity functions. We also address the specific situation in which a test suite contains outliers, that is a small group of very different test cases, and show that it decreases the effectiveness of similarity-based selection. We then propose, and successfully evaluate based on two industrial systems, a solution based on rank scaling to alleviate this problem.
AB - Our experience with applying model-based testing on industrial systems showed that the generated test suites are often too large and costly to execute given project deadlines and the limited resources for system testing on real platforms. In such industrial contexts, it is often the case that only a small subset of test cases can be run. In previous work, we proposed novel test case selection techniques that minimize the similarities among selected test cases and outperforms other selection alternatives. In this paper, our goal is to gain insights into why and under which conditions similarity-based selection techniques, and in particular our approach, can be expected to work. We investigate the properties of test suites with respect to similarities among fault revealing test cases. We thus identify the ideal situation in which a similarity-based selection works best, which is useful for devising more effective similarity functions. We also address the specific situation in which a test suite contains outliers, that is a small group of very different test cases, and show that it decreases the effectiveness of similarity-based selection. We then propose, and successfully evaluate based on two industrial systems, a solution based on rank scaling to alleviate this problem.
KW - Adaptive Random Testing
KW - Distance Function
KW - Genetic Algorithms
KW - Model Based Testing
KW - Similarity Measure
KW - Test Case Selection
UR - http://www.scopus.com/inward/record.url?scp=79958718295&partnerID=8YFLogxK
U2 - 10.1109/ICST.2011.12
DO - 10.1109/ICST.2011.12
M3 - Conference contribution
AN - SCOPUS:79958718295
SN - 9780769543420
T3 - Proceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011
SP - 327
EP - 336
BT - Proceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011
T2 - 4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011
Y2 - 21 March 2011 through 25 March 2011
ER -