Empirical investigation of the effects of test suite properties on similarity-based test case selection

Hadi Hemmati, Andrea Arcuri, Lionel Briand

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Our experience with applying model-based testing on industrial systems showed that the generated test suites are often too large and costly to execute given project deadlines and the limited resources for system testing on real platforms. In such industrial contexts, it is often the case that only a small subset of test cases can be run. In previous work, we proposed novel test case selection techniques that minimize the similarities among selected test cases and outperforms other selection alternatives. In this paper, our goal is to gain insights into why and under which conditions similarity-based selection techniques, and in particular our approach, can be expected to work. We investigate the properties of test suites with respect to similarities among fault revealing test cases. We thus identify the ideal situation in which a similarity-based selection works best, which is useful for devising more effective similarity functions. We also address the specific situation in which a test suite contains outliers, that is a small group of very different test cases, and show that it decreases the effectiveness of similarity-based selection. We then propose, and successfully evaluate based on two industrial systems, a solution based on rank scaling to alleviate this problem.

Original languageEnglish
Title of host publicationProceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011
Pages327-336
Number of pages10
DOIs
Publication statusPublished - 2011
Externally publishedYes
Event4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011 - Berlin, Germany
Duration: 21 Mar 201125 Mar 2011

Publication series

NameProceedings - 4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011

Conference

Conference4th IEEE International Conference on Software Testing, Verification, and Validation, ICST 2011
Country/TerritoryGermany
CityBerlin
Period21/03/1125/03/11

Keywords

  • Adaptive Random Testing
  • Distance Function
  • Genetic Algorithms
  • Model Based Testing
  • Similarity Measure
  • Test Case Selection

Fingerprint

Dive into the research topics of 'Empirical investigation of the effects of test suite properties on similarity-based test case selection'. Together they form a unique fingerprint.

Cite this