Question: “Can we push ourselves to be more creative” ?

The Continuator project started in 2000, and pionnered musical reflexive interaction through interactive style modeling. The system was experiences with many musicians at that time, including Bernard Lubat (Uzeste festival, Ircam) and György Kurtag Jr and Sr. Branch of this research activity was devoted to the study of children musical interaction, with the collaboration of Anna-Rita Addessi from the University of Bologna. In particular, we have shown that Continuator could be considered as a particular Flow machine, in the sense of Csikszentmihalyi.
This activity has then developped in other currently active branches: modeling of Bebop improvization, and music interaction games.

Selected Papers:

Pachet, F. Bebop explained. submitted, 2009.

Pachet, F. Description-Based Design of Melodies. Computer Music Journal, 33(4), Winter 2009.

Pachet, F The future of content is in ourselves. ACM Journal of Computers in Entertainment, 6(3), 2008.

Benghi, D., Addessi, A.-R. and Pachet, F. Teaching to Improvise with the Continuator. Proceedings of the 28th International Society for Music Education World Conference, Bologna, Italy, 2008.

Pachet, F. Creativity Studies and Musical Interaction. In Deliège, I. and Wiggins, G., editor, Musical Creativity: Multidisciplinary Research in Theory And Practice, Psychology Press. 2006.

Ferrari, L. Addessi, A.-R. and Pachet F. New technologies for new music education: The Continuator in a classroom setting. In Baroni et al., editor, Proceedings of ICMPC 06, Bologna, Italy, 2006.

Addessi, A.-R. and Pachet, F. Experiments with a Musical Machine: Musical Style Replication in 3/5 year old Children.. British Journal of Music Education, 22(1):21-46 March 2005.

Pachet, F. On the Design of Flow Machines. In Tokoro, M. and Steels, L, editor, A Learning Zone of One’s Own, The Future of Learning , IOS Press. Amsterdam, 2004.

Pachet, Francois Beyond the Cybernetic Jam fantasy: The Continuator. IEEE Computers Graphics and Applications, 4(1):31-35 January/February 2004. special issue on Emerging Technologies

Pachet, F., Addessi, Anna-Rita When Children Reflect on Their Playing Style: The Continuator. ACM Computers in Entertainment, 1(2), 2004.

Pachet, Francois The Continuator: Musical Interaction with Style. Journal of New Music Research, 32(3):333-341 2003.

Pachet, Francois Interacting with a musical learning system: the continuator. In C. Anagnostopoulou, M. Ferrand, A. Smaill, editor, Music and Artificial Intelligence, Lecture Notes in Artificial Intelligence (vol. 2445 ), pages 119-132, Springer Verlag. September 2002.

Pachet, Francois The Continuator: Musical Interaction with Style. In ICMA, editor, Proceedings of ICMC, pages 211-218, Göteborg, Sweden, September 2002. ICMA. best paper award

Pachet, F. Playing with Virtual Musicians: the Continuator in practice. IEEE Multimedia, 9(3):77-82 2002.

The Continuator, developed by Francois Pachet, is a music system which allows musicians to “play with themselves”: the system is able to pick up the musical style of the musician and respond with musical phrases of its own so that a dialogue emerges in which the music evolves and changes in interaction with the human player. Several top musicians have already played duets with the Continuator, both in the domains of classical contemporary music (Gyorgy Kurtag jr. and sr.) and Jazz (Bernard Lubat, Alain Silva and Albert van Veenendaal).

Team: François Pachet

Collaboration: Albert van Veenendaal, Georgy Kurtag Sr. and Jr., and Olivier Desagnat

Question: How to program content-based management systems efficiently and easily? How to capitalize and reuse programming and design know-how for this new class of systems?

More precisely we propose three working hypothesis for building an environment that integrates smoothly all the content-management techniques covered by the Music Group research in an integrated manner, so as to propose novel applications that can expand the possibilities of music access.

• Integration of activities in a single environment. The different applications envisaged will necessarily share many information, data, metadata and also software components; It is therefore crucial that they can communicate with each other smoothly.
• Need for managing efficiently large databases. Metadata is interesting, by definition, only for managing large databases, which in turn creates issues of efficiency. Compilers which create efficient Sql queries are mandatory to create systems useable by non professionals.
• Need for vertical languages to develop these new systems. The development of a content-based music application requires the handling of many different layers of software development, from the design of audio acoustic descriptors to the development of graphical interfaces (Matlab, C++, Sql, Php, Java, etc.).

Managing these different levels and their interconnections is time consuming and acrobatic. Vertical languages reduce the difficulty by packaging vertically services, thereby freeing the developer to handle manually all these levels.

To implement the hypothesis proposed above, we have developed an object-oriented framework (in the sense of (Fayad et al. 1999)) called MCM (standing for Multimedia Content Management). This framework contains all the important services needed to build content-based music applications, from the design of perceptive descriptors (using the EDS system) to the creation of specific ontologies such as genre and the creation of user interfaces.

Question: How to extract features from audio signals (such as music titles or sound samples) that are efficient for a given classification task ?

Our team has pionneered the development of so-called “feature generation” techniques, in the audio domain. Feature generation consists in letting a system evolve features automatically, for a given classification problem, rather than relying on existing feature sets. We have introduced the notion of analytical features, as audio features consisting in mathematical compositions of elementary operators. The EDS system was designed to generate billions of these features and test them, to evolve efficient features and feature sets. We have shown that analytical features perform better than standard features on a series of well-known audio classification problems. We now focus on the mathematical study of the analytical feature space (e.g. is fitness continuous ?) using tools borrowed from complex system theory.

Selected Papers:

Pachet, F. and Roy, P Analytical Features: a Knowledge-Based Approach to Audio Feature Generation. EURASIP Journal on Audio, Speech, and Music Processing, 2009(1), February 2009.

Molnár, Csaba, Kaplan, Frédéric, Roy, Pierre, Pachet, Francois, Pongrácz, Péter, Dóka, Antal and Miklósi, Ádám Classification of dog barks: a machine learning approach. Animal Cognition, 11(3):389-400 2008.

Roy, P., Pachet, F. and Krakowski, S. Improving the Classification of Percussive Sounds with Analytical Features: a Case Study. Proceedings of Ismir 07, pages 229-232, Vienna, Austria, 2007.

Pachet, F. and Roy, P. Exploring billions of audio features. In Eurasip, editor, Proceedings of CBMI 07, pages 227-235, Bordeaux, France, 2007.

Defréville, B. Roy, P., Rosin, C. and Pachet, F. Automatic Recognition of Urban Sound Sources. Proceedings of the 120th AES Conference, Paris, France, 2006.

Monceaux Jérôme; Pachet, François; Amadu, Frédéric; Roy, Pierre and Aymeric Zils Descriptor-based spatialization. Proceedings of AES Conference 2005, Barcelona, Spain, 2005.

Giordano Cabral, François Pachet, and Jean-Pierre Briot. Automatic X traditional descriptor extraction: The case of chord recognition.. Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR’2005), pages 444-449, London, U.K., September 2005.

Zils, A. and Pachet, F. Automatic Extraction of Music Descriptors from Acoustic Signals using EDS. Proceedings of the 116th AES Convention, Berlin, Germany, May 2004.

Zils, A. & Pachet, F. Extracting Automatically the Perceived Intensity of Music Titles. Proceedings of the 6th COST-G6 Conference on Digital Audio Effects (DAFX03), pages 180-183, London, U.K., September 2003. Queen Mary University

Pachet, F. and Zils, A. Evolving Automatically High-Level Music Descriptors From Acoustic Signals. Springer Verlag LNCS, 2771:42-53 2003.

Pachet F., Zils A. Evolving Automatically High-Level Music Descriptors From Acoustic Signals. Sony CSL, 2003.

Question: How to find “interesting” music in large and unknown music collections? How to structure and index automatically music collections?

The intentionally ambiguous expression “Popular Music Browser” reflects the two main goals of this project, which started in 1998, at Sony CSL laboratory. First, we are interested in human-centered issues related to browsing “Popular Music”. Popular here means that the music accessed to is widely distributed, and known to many listeners. Second, we consider “popular browsing” of music, i.e. making music accessible to non specialists (music lovers), and allowing sharing of musical tastes and information within communities, departing from the usual, single user view of digital libraries.

The MusicBrowser is the first music content management tool able to handle large music catalogues, and offer users many novel content-based access methods in an integrated environment. It integrates all aspects of the music-to-listener chain, from music description - descriptor extraction from the music signal, or data mining techniques -, similarity based access and novel music retrieval methods such as automatic sequence generation, and user interface issues.

We are currently conducting a series of user-studies, first a workshop at University of Bologna in May 2004, then a week-long user study at Cité des Sciences, Paris in June 2004, and finally a 2-week “atelier” at Cité des Sciences during the “Villette Numérique” biennale in Septembre 2004, where communities of 20+ people extensively use the Browser, to validate existing descriptors, and design their own descriptors, using the automatic learning from EDS. To this effect we have introduced the notion of “music game”, in which users have to localize a particular song they hear, using the various search methods at their disposal.

Media

• Music Browser demo: Music Browser demo movie
• Music Browser interface:
  
• Pictures of a 3-day seminar at the Facolta di Scienze della Formazione, University of Bologna, Italy:
  
• Pictures of a week-long public workshop at the Multimedia Library, Cité des Sciences, Paris.
  

Question: How to create “interesting” audio music sequences from large collections of sound samples? The team invented the notion of musacing, by analogy with mosaicing, and explore new forms of interaction with music catalogues, such as song sampling.

The idea of musaicing is a transposition of the notion of image mosaicing to the world of audio. Musaicing makes intensive use of large databases of audio samples, and allows user to create music without having to perform the tedious and difficult task of listening and selecting individual samples. Musaicing consists in creating automatically large databases of samples by segmenting existing songs. Then metadata is computed for each sample to describe it in terms of perceptive parameters (such as timbre, percussivity, energy, pitchness, etc.). Finally the user can express high-level constraints to specify the structure and nature of a target sequence of samples. Constraints can be of various types, such as continuity (produce a sequence of samples which are continuous, timbre-wise), distribution (select a percussive sample every beat with tempo = 120) or cardinality (include at least 40% of samples which come from a Beatles song), or any combinations of these.

Media

• Interface screenshot