Testing the untestable: Model testing of complex software-intensive systems

Lionel Briand, Shiva Nejati, Mehrdad Sabetzadeh, Domenico Bianculli

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Increasingly, we are faced with systems that are untestable, meaning that traditional testing methods are expensive, time-consuming or infeasible to apply due to factors such as the systems' continuous interactions with the environment and the deep intertwining of software with hardware. In this paper we outline our vision to enable testing of untestable systems. Our key idea is to frame testing on models rather than operational systems. We refer to such testing as model testing. Our goal is to raise the level of abstraction of testing from operational systems to models of their behaviors and properties. The models that underlie model testing are executable representations of the relevant aspects of a system and its environment, alongside the risks of system failures. Such models necessarily have uncertainties due to complex, dynamic environment behaviors and the unknowns about the system. This makes it crucial for model testing to be uncertainty-aware. We propose to synergistically combine metaheuristic search, increasingly used in traditional software testing, with system and risk models to drive the search for faults that entail the most risk. We expect model testing to bring early and cost-effective automation to the testing of many critical systems that defy existing automation techniques, thus significantly improving the dependability of such systems.

Original languageEnglish
Title of host publicationProceedings - 5th International Workshop on Green and Sustainable Software, GREENS 2016
PublisherIEEE Computer Society
Pages789-792
Number of pages4
ISBN (Electronic)9781450341615, 9781450342056
DOIs
Publication statusPublished - 14 May 2016
Externally publishedYes
Event2016 IEEE/ACM 38th IEEE International Conference on Software Engineering, ICSE 2016 - Austin, United States
Duration: 14 May 201622 May 2016

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257

Conference

Conference2016 IEEE/ACM 38th IEEE International Conference on Software Engineering, ICSE 2016
Country/TerritoryUnited States
CityAustin
Period14/05/1622/05/16

Fingerprint

Dive into the research topics of 'Testing the untestable: Model testing of complex software-intensive systems'. Together they form a unique fingerprint.

Cite this