TY - GEN
T1 - Testing the untestable
T2 - 2016 IEEE/ACM 38th IEEE International Conference on Software Engineering, ICSE 2016
AU - Briand, Lionel
AU - Nejati, Shiva
AU - Sabetzadeh, Mehrdad
AU - Bianculli, Domenico
N1 - Publisher Copyright:
© 2016 ACM.
PY - 2016/5/14
Y1 - 2016/5/14
N2 - Increasingly, we are faced with systems that are untestable, meaning that traditional testing methods are expensive, time-consuming or infeasible to apply due to factors such as the systems' continuous interactions with the environment and the deep intertwining of software with hardware. In this paper we outline our vision to enable testing of untestable systems. Our key idea is to frame testing on models rather than operational systems. We refer to such testing as model testing. Our goal is to raise the level of abstraction of testing from operational systems to models of their behaviors and properties. The models that underlie model testing are executable representations of the relevant aspects of a system and its environment, alongside the risks of system failures. Such models necessarily have uncertainties due to complex, dynamic environment behaviors and the unknowns about the system. This makes it crucial for model testing to be uncertainty-aware. We propose to synergistically combine metaheuristic search, increasingly used in traditional software testing, with system and risk models to drive the search for faults that entail the most risk. We expect model testing to bring early and cost-effective automation to the testing of many critical systems that defy existing automation techniques, thus significantly improving the dependability of such systems.
AB - Increasingly, we are faced with systems that are untestable, meaning that traditional testing methods are expensive, time-consuming or infeasible to apply due to factors such as the systems' continuous interactions with the environment and the deep intertwining of software with hardware. In this paper we outline our vision to enable testing of untestable systems. Our key idea is to frame testing on models rather than operational systems. We refer to such testing as model testing. Our goal is to raise the level of abstraction of testing from operational systems to models of their behaviors and properties. The models that underlie model testing are executable representations of the relevant aspects of a system and its environment, alongside the risks of system failures. Such models necessarily have uncertainties due to complex, dynamic environment behaviors and the unknowns about the system. This makes it crucial for model testing to be uncertainty-aware. We propose to synergistically combine metaheuristic search, increasingly used in traditional software testing, with system and risk models to drive the search for faults that entail the most risk. We expect model testing to bring early and cost-effective automation to the testing of many critical systems that defy existing automation techniques, thus significantly improving the dependability of such systems.
UR - http://www.scopus.com/inward/record.url?scp=84989170087&partnerID=8YFLogxK
U2 - 10.1145/2889160.2889212
DO - 10.1145/2889160.2889212
M3 - Conference contribution
AN - SCOPUS:84989170087
T3 - Proceedings - International Conference on Software Engineering
SP - 789
EP - 792
BT - Proceedings - 5th International Workshop on Green and Sustainable Software, GREENS 2016
PB - IEEE Computer Society
Y2 - 14 May 2016 through 22 May 2016
ER -