Rouages: Revealing the mechanisms of digital musical instruments to the audience

Digital musical instruments bring new possibilities for musical performance. They are also more complex for the audience to understand, due to the diversity of their components and the magical aspect of the musicians’ actions when compared to acoustic instruments. This complexity results in a loss of liveness and possibly a poor experience for the audience. Our approach, called Rouages, is based on a mixed-reality display system and a 3D visualization application. Rouages reveals the mechanisms of digital musical instruments by amplifying musicians’ gestures with virtual extensions of the sensors, by representing the sound components with 3D shapes and specific behaviors and by showing the impact of musicians’ gestures on these components. In its current implementation, it focuses on MIDI controllers as input devices. We believe that Rouages opens up new perspectives for instrument makers and musicians to improve audience experience with their digital musical instruments.