Question: How to program content-based management systems efficiently and easily? How to capitalize and reuse programming and design know-how for this new class of systems?

More precisely we propose three working hypothesis for building an environment that integrates smoothly all the content-management techniques covered by the Music Group research in an integrated manner, so as to propose novel applications that can expand the possibilities of music access.

• Integration of activities in a single environment. The different applications envisaged will necessarily share many information, data, metadata and also software components; It is therefore crucial that they can communicate with each other smoothly.
• Need for managing efficiently large databases. Metadata is interesting, by definition, only for managing large databases, which in turn creates issues of efficiency. Compilers which create efficient Sql queries are mandatory to create systems useable by non professionals.
• Need for vertical languages to develop these new systems. The development of a content-based music application requires the handling of many different layers of software development, from the design of audio acoustic descriptors to the development of graphical interfaces (Matlab, C++, Sql, Php, Java, etc.).

Managing these different levels and their interconnections is time consuming and acrobatic. Vertical languages reduce the difficulty by packaging vertically services, thereby freeing the developer to handle manually all these levels.

To implement the hypothesis proposed above, we have developed an object-oriented framework (in the sense of (Fayad et al. 1999)) called MCM (standing for Multimedia Content Management). This framework contains all the important services needed to build content-based music applications, from the design of perceptive descriptors (using the EDS system) to the creation of specific ontologies such as genre and the creation of user interfaces.

Question: How to extract features from audio signals (such as music titles or sound samples) that are efficient for a given classification task ?

Our team has pionneered the development of so-called “feature generation” techniques, in the audio domain. Feature generation consists in letting a system evolve features automatically, for a given classification problem, rather than relying on existing feature sets. We have introduced the notion of analytical features, as audio features consisting in mathematical compositions of elementary operators. The EDS system was designed to generate billions of these features and test them, to evolve efficient features and feature sets. We have shown that analytical features perform better than standard features on a series of well-known audio classification problems. We now focus on the mathematical study of the analytical feature space (e.g. is fitness continuous ?) using tools borrowed from complex system theory.

Selected Papers:

Pachet, F. and Roy, P Analytical Features: a Knowledge-Based Approach to Audio Feature Generation. EURASIP Journal on Audio, Speech, and Music Processing, 2009(1), February 2009.

Molnár, Csaba, Kaplan, Frédéric, Roy, Pierre, Pachet, Francois, Pongrácz, Péter, Dóka, Antal and Miklósi, Ádám Classification of dog barks: a machine learning approach. Animal Cognition, 11(3):389-400 2008.

Roy, P., Pachet, F. and Krakowski, S. Improving the Classification of Percussive Sounds with Analytical Features: a Case Study. Proceedings of Ismir 07, pages 229-232, Vienna, Austria, 2007.

Pachet, F. and Roy, P. Exploring billions of audio features. In Eurasip, editor, Proceedings of CBMI 07, pages 227-235, Bordeaux, France, 2007.

Defréville, B. Roy, P., Rosin, C. and Pachet, F. Automatic Recognition of Urban Sound Sources. Proceedings of the 120th AES Conference, Paris, France, 2006.

Monceaux Jérôme; Pachet, François; Amadu, Frédéric; Roy, Pierre and Aymeric Zils Descriptor-based spatialization. Proceedings of AES Conference 2005, Barcelona, Spain, 2005.

Giordano Cabral, François Pachet, and Jean-Pierre Briot. Automatic X traditional descriptor extraction: The case of chord recognition.. Proceedings of the 6th International Conference on Music Information Retrieval (ISMIR’2005), pages 444-449, London, U.K., September 2005.

Zils, A. and Pachet, F. Automatic Extraction of Music Descriptors from Acoustic Signals using EDS. Proceedings of the 116th AES Convention, Berlin, Germany, May 2004.

Zils, A. & Pachet, F. Extracting Automatically the Perceived Intensity of Music Titles. Proceedings of the 6th COST-G6 Conference on Digital Audio Effects (DAFX03), pages 180-183, London, U.K., September 2003. Queen Mary University

Pachet, F. and Zils, A. Evolving Automatically High-Level Music Descriptors From Acoustic Signals. Springer Verlag LNCS, 2771:42-53 2003.

Pachet F., Zils A. Evolving Automatically High-Level Music Descriptors From Acoustic Signals. Sony CSL, 2003.

Question: How to find “interesting” music in large and unknown music collections? How to structure and index automatically music collections?

The intentionally ambiguous expression “Popular Music Browser” reflects the two main goals of this project, which started in 1998, at Sony CSL laboratory. First, we are interested in human-centered issues related to browsing “Popular Music”. Popular here means that the music accessed to is widely distributed, and known to many listeners. Second, we consider “popular browsing” of music, i.e. making music accessible to non specialists (music lovers), and allowing sharing of musical tastes and information within communities, departing from the usual, single user view of digital libraries.

The MusicBrowser is the first music content management tool able to handle large music catalogues, and offer users many novel content-based access methods in an integrated environment. It integrates all aspects of the music-to-listener chain, from music description - descriptor extraction from the music signal, or data mining techniques -, similarity based access and novel music retrieval methods such as automatic sequence generation, and user interface issues.

We are currently conducting a series of user-studies, first a workshop at University of Bologna in May 2004, then a week-long user study at Cité des Sciences, Paris in June 2004, and finally a 2-week “atelier” at Cité des Sciences during the “Villette Numérique” biennale in Septembre 2004, where communities of 20+ people extensively use the Browser, to validate existing descriptors, and design their own descriptors, using the automatic learning from EDS. To this effect we have introduced the notion of “music game”, in which users have to localize a particular song they hear, using the various search methods at their disposal.

Media

• Music Browser demo: Music Browser demo movie
• Music Browser interface:
  
• Pictures of a 3-day seminar at the Facolta di Scienze della Formazione, University of Bologna, Italy:
  
• Pictures of a week-long public workshop at the Multimedia Library, Cité des Sciences, Paris.