TY - GEN
T1 - Model based test validation and oracles for data acquisition systems
AU - Di Nardo, Daniel
AU - Alshahwan, Nadia
AU - Briand, Lionel C.
AU - Fourneret, Elizabeta
AU - Nakic-Alfirevic, Tomislav
AU - Masquelier, Vincent
PY - 2013
Y1 - 2013
N2 - This paper presents an automated, model based test validation and oracle approach for systems with complex input and output structures, such as Data Acquisition (DAQ) systems, which are common in many sectors including the satellite communications industry. We present a customised modelling methodology for such systems and a tool that automatically validates test inputs and, after test execution, applies an oracle that is based on mappings between the input and output. We also apply our proposed approach and tool to a complex industrial DAQ system and investigate the scalability and effectiveness of the approach in validating test cases, the DAQ system, or its specifications (captured as models). The results of the case study show that the approach is indeed scalable with respect to two dimensions: (1) model size and (2) test validation and oracle execution time. The size of the model for the DAQ system under study remains within practical bounds, and far below that of typical system models, as it includes a class diagram with 68 classes and 49 constraints. The developed test validation and oracles tool can handle satellite transmission files up to two GB within practical time constraints, taking, on a standard PC, less than three minutes for test validation and less than 50 minutes for applying the oracle. The approach was also effective in automatically applying the oracle successfully for the actual test suite of the DAQ system, accurately identifying all issues and violations that were expected, thus showing that an approach based on models can be sufficiently accurate.
AB - This paper presents an automated, model based test validation and oracle approach for systems with complex input and output structures, such as Data Acquisition (DAQ) systems, which are common in many sectors including the satellite communications industry. We present a customised modelling methodology for such systems and a tool that automatically validates test inputs and, after test execution, applies an oracle that is based on mappings between the input and output. We also apply our proposed approach and tool to a complex industrial DAQ system and investigate the scalability and effectiveness of the approach in validating test cases, the DAQ system, or its specifications (captured as models). The results of the case study show that the approach is indeed scalable with respect to two dimensions: (1) model size and (2) test validation and oracle execution time. The size of the model for the DAQ system under study remains within practical bounds, and far below that of typical system models, as it includes a class diagram with 68 classes and 49 constraints. The developed test validation and oracles tool can handle satellite transmission files up to two GB within practical time constraints, taking, on a standard PC, less than three minutes for test validation and less than 50 minutes for applying the oracle. The approach was also effective in automatically applying the oracle successfully for the actual test suite of the DAQ system, accurately identifying all issues and violations that were expected, thus showing that an approach based on models can be sufficiently accurate.
UR - http://www.scopus.com/inward/record.url?scp=84893528493&partnerID=8YFLogxK
U2 - 10.1109/ASE.2013.6693111
DO - 10.1109/ASE.2013.6693111
M3 - Conference contribution
AN - SCOPUS:84893528493
SN - 9781479902156
T3 - 2013 28th IEEE/ACM International Conference on Automated Software Engineering, ASE 2013 - Proceedings
SP - 540
EP - 550
BT - 2013 28th IEEE/ACM International Conference on Automated Software Engineering, ASE 2013 - Proceedings
T2 - 2013 28th IEEE/ACM International Conference on Automated Software Engineering, ASE 2013
Y2 - 11 November 2013 through 15 November 2013
ER -