Search-based automated testing of continuous controllers: Framework, tool support, and case studies

Reza Matinnejad, Shiva Nejati, Lionel Briand, Thomas Bruckmann, Claude Poull

Research output: Contribution to journalArticlepeer-review

Abstract

Context: Testing and verification of automotive embedded software is a major challenge. Software production in automotive domain comprises three stages: Developing automotive functions as Simulink models, generating code from the models, and deploying the resulting code on hardware devices. Automotive software artifacts are subject to three rounds of testing corresponding to the three production stages: Model-in-The-Loop (MiL), Software-in-The-Loop (SiL) and Hardware-in-The-Loop (HiL) testing. Objective: We study testing of continuous controllers at the Model-in-Loop (MiL) level where both the controller and the environment are represented by models and connected in a closed loop system. These controllers make up a large part of automotive functions, and monitor and control the operating conditions of physical devices. Method: We identify a set of requirements characterizing the behavior of continuous controllers, and develop a search-based technique based on random search, adaptive random search, hill climbing and simulated annealing algorithms to automatically identify worst-case test scenarios which are utilized to generate test cases for these requirements. Results: We evaluated our approach by applying it to an industrial automotive controller (with 443 Simulink blocks) and to a publicly available controller (with 21 Simulink blocks). Our experience shows that automatically generated test cases lead to MiL level simulations indicating potential violations of the system requirements. Further, not only does our approach generate significantly better test cases faster than random test case generation, but it also achieves better results than test scenarios devised by domain experts. Finally, our generated test cases uncover discrepancies between environment models and the real world when they are applied at the Hardware-in-The-Loop (HiL) level. Conclusion: We propose an automated approach to MiL testing of continuous controllers using search. The approach is implemented in a tool and has been successfully applied to a real case study from the automotive domain.

Original languageEnglish
Pages (from-to)705-722
Number of pages18
JournalInformation and Software Technology
Volume57
Issue number1
DOIs
Publication statusPublished - 2015
Externally publishedYes

Keywords

  • Automotive software systems
  • Continuous controllers
  • Model-in-The-loop testing
  • Search-based testing
  • Simulink models

Fingerprint

Dive into the research topics of 'Search-based automated testing of continuous controllers: Framework, tool support, and case studies'. Together they form a unique fingerprint.

Cite this