Music research at Sony CSL Paris is concerned with two main areas: the multiple facets of interaction in music, and the challenges of robust music description. This work leads to the creation of intelligent musical systems that propose new modes of access to music, interaction with sound, and human interaction.
Question: how to mix music when you are not a sound engineer ?
The original goal of the MusicSpace project was to answer this question by introducing a constraint solver to control the location and movements of sound sources. Movements initiated by the user trigger the solver which then moves automatically other sound sources to satisfy a set of “constraints”. We have shown that a limited set of constraints (available in MusicSpaces’s constraint palette) suffice to ensure, for instance, that every movement of user would always produce a “good” mix. Other constraints can be set to produce automatic trajectories of sound sources. The original MusicSpace system and technology are described in several papers, including Olivier Delerue’s Ph.D thesis.
MusicSpace is now integrated into Max/MSP (as an mxj object). As a consequence, MusicSpace’s original constraint solver can be used to manipulate not only spatialisation parameters, but also arbitrary Max/MSP data. This integration opens up exciting new possibilities which we are only starting to explore, notably for controlling sound synthesis engines (see the videos for examples).
Selected Papers:
Delerue, O. Pachet, F. and Roy, P. A new MusicSpace integrated in Max, to appear.
Spatialisation du son et programmation par contraintes: le système MusicSpace. Université Paris 6, Paris, January 2004. 
On-The-Fly Multi-Track Mixing. Proceedings of AES 109th Convention, Los Angeles, USA, 2000. AES.
MusicSpace: a Constraint-based Control System for Music Spatialization. Proceedings of ICMC 1999, pages 272-275, Beijing, China, 1999. ICMA.
Constraint-Based Spatialization. First COST-G6 Workshop on Digital Audio Effects (DAFX98), pages 71-75, Barcelona, Spain, November 1998.
MidiSpace: a Constraint-based Temporal Music Spatializer. ACM Multimedia Conference, pages 351-359, Bristol, UK, 1998.
Question: “Can we build systems that interact meaningfully with animals ?”
As a followup of our work on audio feature generation and classification, we have become interested in animal audio communication. The initial idea was to test whether our techniques developped for music applications would also fit with animal vocalizations. We have shown (together with the team of Csaba Molnar) that feature generation could improve dog bark classification to reach performance that exceed the performance of human experts. This work led to the stuy of other animals, such as parrots and canaries. Now we are interested in establishing complete close-loop audio interactions between an animal and an artificial system. The goal of such a study is twofold. First, we want to trigger interactions and behaviors that are otherwise difficult if not impossible to observe. These behavior can help ethologists to understand animal behavior and cognition.. The second goal is to study the mechanisms of reflexive interaction in non human species to study the relationships between perception and action.
Selected papers:
Mise en place d’interactions acoustiques chez les canaris. Juin 2009. Rapport de stage effectué au Laboratoire d’Ethologie et Cognition Comparées, Université Paris Ouest - Nanterre La Défense / Sony
Evolving Acoustic Features for Parrot Vocalizations. submitted, 2009.
Classification of dog barks: a machine learning approach. Animal Cognition, 11(3):389-400 2008.
Different Acoustic Analysis Methods Yield Different Results. Vocal communication in birds and Mammals, St Andrews, Scotland, July 2008.
Question: “Can we push ourselves to be more creative” ?
The Continuator project started in 2000, and pionnered musical reflexive interaction through interactive style modeling. The system was experiences with many musicians at that time, including Bernard Lubat (Uzeste festival, Ircam) and György Kurtag Jr and Sr. Branch of this research activity was devoted to the study of children musical interaction, with the collaboration of Anna-Rita Addessi from the University of Bologna. In particular, we have shown that Continuator could be considered as a particular Flow machine, in the sense of Csikszentmihalyi.
This activity has then developped in other currently active branches: modeling of Bebop improvization, and music interaction games.
Selected Papers:
Bebop explained. submitted, 2009.
Description-Based Design of Melodies. Computer Music Journal, 33(4), Winter 2009.
The future of content is in ourselves. ACM Journal of Computers in Entertainment, 6(3), 2008.
Teaching to Improvise with the Continuator. Proceedings of the 28th International Society for Music Education World Conference, Bologna, Italy, 2008.
Creativity Studies and Musical Interaction. In Deliège, I. and Wiggins, G., editor, Musical Creativity: Multidisciplinary Research in Theory And Practice, Psychology Press. 2006.
New technologies for new music education: The Continuator in a classroom setting. In Baroni et al., editor, Proceedings of ICMPC 06, Bologna, Italy, 2006.
Experiments with a Musical Machine: Musical Style Replication in 3/5 year old Children.. British Journal of Music Education, 22(1):21-46 March 2005.
On the Design of Flow Machines. In Tokoro, M. and Steels, L, editor, A Learning Zone of One’s Own, The Future of Learning , IOS Press. Amsterdam, 2004.
Beyond the Cybernetic Jam fantasy: The Continuator. IEEE Computers Graphics and Applications, 4(1):31-35 January/February 2004. special issue on Emerging Technologies
When Children Reflect on Their Playing Style: The Continuator. ACM Computers in Entertainment, 1(2), 2004.
The Continuator: Musical Interaction with Style. Journal of New Music Research, 32(3):333-341 2003.
Interacting with a musical learning system: the continuator. In C. Anagnostopoulou, M. Ferrand, A. Smaill, editor, Music and Artificial Intelligence, Lecture Notes in Artificial Intelligence (vol. 2445 ), pages 119-132, Springer Verlag. September 2002.
The Continuator: Musical Interaction with Style. In ICMA, editor, Proceedings of ICMC, pages 211-218, Göteborg, Sweden, September 2002. ICMA. best paper award
Playing with Virtual Musicians: the Continuator in practice. IEEE Multimedia, 9(3):77-82 2002.
Question: How to extract features from audio signals (such as music titles or sound samples) that are efficient for a given classification task ?
Our team has pionneered the development of so-called “feature generation” techniques, in the audio domain. Feature generation consists in letting a system evolve features automatically, for a given classification problem, rather than relying on existing feature sets. We have introduced the notion of analytical features, as audio features consisting in mathematical compositions of elementary operators. The EDS system was designed to generate billions of these features and test them, to evolve efficient features and feature sets. We have shown that analytical features perform better than standard features on a series of well-known audio classification problems. We now focus on the mathematical study of the analytical feature space (e.g. is fitness continuous ?) using tools borrowed from complex system theory.
Selected Papers:
Analytical Features: a Knowledge-Based Approach to Audio Feature Generation. EURASIP Journal on Audio, Speech, and Music Processing, 2009(1), February 2009.
Classification of dog barks: a machine learning approach. Animal Cognition, 11(3):389-400 2008.
Improving the Classification of Percussive Sounds with Analytical Features: a Case Study. Proceedings of Ismir 07, pages 229-232, Vienna, Austria, 2007.
Exploring billions of audio features. In Eurasip, editor, Proceedings of CBMI 07, pages 227-235, Bordeaux, France, 2007.
Automatic Recognition of Urban Sound Sources. Proceedings of the 120th AES Conference, Paris, France, 2006.
Descriptor-based spatialization. Proceedings of AES Conference 2005, Barcelona, Spain, 2005.
Automatic X traditional descriptor extraction: The case of chord recognition.. Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR’2005), pages 444-449, London, U.K., September 2005.
Automatic Extraction of Music Descriptors from Acoustic Signals using EDS. Proceedings of the 116th AES Convention, Berlin, Germany, May 2004.
Extracting Automatically the Perceived Intensity of Music Titles. Proceedings of the 6th COST-G6 Conference on Digital Audio Effects (DAFX03), pages 180-183, London, U.K., September 2003. Queen Mary University
Evolving Automatically High-Level Music Descriptors From Acoustic Signals. Springer Verlag LNCS, 2771:42-53 2003.
Evolving Automatically High-Level Music Descriptors From Acoustic Signals. Sony CSL, 2003.
