Virtual Control and Synthesis of Music Performances: Qualitative Evaluation of Synthesized Timpani Exercises

Alexandre Bouënard (1, 2) , Marcelo Mortensen Wanderley (3) , Sylvie Gibet (2) , Fabrice Marandola (4)

1 IDMIL - Input Devices and Music Interaction Laboratory
2 SEASIDE - SEarch, Analyze, Synthesize and Interact with Data Ecosystems
3 IDMIL - Input Devices and Music Interaction Laboratory
4 CIRMMT - Centre Interdisciplinaire de Recherche en Musique, Médias et Technologie [Montréal]

The increasing availability of software for creating real-time simulations of musical instrument sounds allows for the design of new visual and sounding media. These past decades have especially focused on the control of real and virtual instruments by natural gestures. In this paper, we present and extensively evaluate a framework (Figure 1) for the control of virtual percussion instruments, by modeling and simulating virtual percussionists gestures. By positioning the virtual performer at the center of the gesture-sound synthesis system, we aim at providing original tools to analyze and synthesize instrumental gesture performances. Our physics-based approach for gesture simulation brings some insight into the effect of biomechanical parameters of the gesture on the instrumental performance. Simulating both gesture and sound by physical models leads also to a coherent and human-centered interaction and provides new ways of exploring the mapping between gesture and sound. The use of motion capture data enables the realistic synthesis of both pre-recorded and novel percussion sequences from the specification of gesture scores. Such scores involve motion editing techniques applied to simple beat attacks. We therefore propose an original gesture language based on the instrumental playing techniques. This language is characterized by expressivity, interactivity with the user, and the possibility to take into account co-articulation between gesture units. Finally, providing 3D visual rendering synchronized with sound rendering allows us to observe virtual performances to the light of real ones, and to qualitatively evaluate both pedagogical and compositional capabilities of such a system.

Domaines: Modélisation et simulation; Multimédia [cs.MM]

Alexandre Bouënard, Marcelo Mortensen Wanderley, Sylvie Gibet, Fabrice Marandola. Virtual Control and Synthesis of Music Performances: Qualitative Evaluation of Synthesized Timpani Exercises. Computer Music Journal, 2011, 35 (3), pp.57-72. ⟨10.1162/COMJ_a_00069⟩. ⟨hal-00763271⟩