Integrating scenario-based and measurement-based software product assessment

The software industry needs means to evaluate software products and compare development and implementation technologies in the context of actual projects. Solutions need to be cost-effective but also technically sound. This paper presents a methodology to combine two software product evaluation techniques: measurement of structural design properties, and evaluation of change scenarios. The goal is to use these two approaches together so that they can address each other's limitations. In a case study in the context of the European aerospace industry, this combined methodology was used to assess the impact of choice of programming language and distribution technology on the maintainability of resulting systems. It encompasses the comparison of C++ and Java, as well as distribution/communication technologies such as IPC via sockets, and CORBA implementation. Lessons learned in terms of benefits and limitations are presented. The study shows the usefulness of the approach presented but it is also clear that it needs to be used in combination with other means of evaluation and with a critical mind, as for any engineering solution.