Using simulation to empirically investigate test coverage criteria based on statechart

A number of testing strategies have been proposed using state machines and statecharts as test models in order to derive test sequences and validate classes or class clusters. Though such criteria have the advantage of being systematic, little is known on how cost effective they are and how they compare to each other. This article presents a precise simulation and analysis procedure to analyze the cost-effectiveness of statechart-based testing techniques. We then investigate, using this procedure, the cost and fault detection effectiveness of adequate test sets for the most referenced coverage criteria for statecharts on three different representative case studies. Through the analysis of common results and differences across studies, we attempt to draw more general conclusions regarding the costs and benefits of using the criteria under investigation.