# publications

Sony CSL authors: Vittorio Loreto, François Pachet

### Abstract

We introduce a Maximum Entropy model able to capture the statistics of melodies in music. The model can be used to generate new melodies that emulate the style of a given musical corpus. Instead of using the n–body interactions of (n−1)–order Markov models, traditionally used in automatic music generation, we use a k-nearest neighbour model with pairwise interactions only. In that way, we keep the number of parameters low and avoid over-fitting problems typical of Markov models. We show that long-range musical phrases don’t need to be explicitly enforced using high-order Markov interactions, but can instead emerge from multiple, competing, pairwise interactions. We validate our Maximum Entropy model by contrasting how much the generated sequences capture the style of the original corpus without plagiarizing it. To this end we use a data-compression approach to discriminate the levels of borrowing and innovation featured by the artificial sequences. Our modelling scheme outperforms both fixed-order and variable-order Markov models. This shows that, despite being based only on pairwise interactions, our scheme opens the possibility to generate musically sensible alterations of the original phrases, providing a way to generate innovation.

Type: article

Keywords: Applied Mathematics, Applied physics, Computational science, Information theory and computation

### Downloads

### BibTeX entry

@ARTICLE { sakellariou:17a, AUTHOR="Sakellariou, J. and Tria, F. and Loreto, V. and Pachet, F.", JOURNAL="Scientific Reports", MONTH="August", NUMBER="9172", TITLE="Maximum entropy models capture melodic styles", VOLUME="7", YEAR="2017", }

Sony CSL authors: Verena Hafner, Frédéric Kaplan

Type: article

Keywords: AIBO, information distance, activity clustering

### BibTeX entry

@ARTICLE { kaplan:06b, AUTHOR="Kaplan, F. and Hafner, V.", JOURNAL="Advanced Robotics", NOTE="in press", TITLE="Information-theoretic framework for unsupervised activity classification", YEAR="2006", }

Sony CSL authors: Verena Hafner, Frédéric Kaplan

### Abstract

This article presents a mathematical framework based on information theory to compare temporally-extended embodied sensorimotor organizations. Central to this approach is the notion of configuration: a set of distances between information sources, statistically evaluated for a given time span. Because information distances capture simultaneously effects of physical closeness, intermodality, functional relationship and external couplings, a configuration characterizes an embodied interaction with a particular environment. In this approach, collections of skills can be mapped in a unified space as configurations of configurations. This article describes these different abstractions in a formal manner and presents results of preliminary experiments showing how this framework can be used to capture the behavioral organization of an autonomous robot.

Type: inproceedings

Keywords: information theory, analogy, skills

### Downloads

### BibTeX entry

@INPROCEEDINGS { kaplan:05d, AUTHOR="Kaplan, F. and Hafner, V.V.", BOOKTITLE="Proceedings of the 4th IEEE International Conference on Development and Learning (ICDL-05)", PAGES="129--134", PUBLISHER="IEEE", TITLE="Mapping the space of skills: An approach for comparing embodied sensorimotor organizations", YEAR="2005", }

Sony CSL authors: Michele Bezzi

### Abstract

A novel definition of the stimulus-speciéc information is presented,which is particularly useful when the stimuli constitute a continuous and metric set, as, for example, position in space. The approach allows one to build the spatial information distribution of a given neural response. The method is applied to the investigation of putative differences in the coding of position in hippocampus and lateral septum.

Type: article

Keywords: Hippocampus, information, neural, cells

### Downloads

### BibTeX entry

@ARTICLE { bezzi:02a, AUTHOR="Bezzi, M. and Samengo, I. and Leutbeg, S. and Mizumori, S.J.Y.", JOURNAL="Neural Computation", NUMBER="2", TITLE="Measuring information spatial densities", VOLUME="14", YEAR="2002", }

Sony CSL authors: Michele Bezzi

### Abstract

Multielectrode arrays allow recording of the activity of many single neurons, from which correlations can be calculated. The functional roles of correlations can be revealed by measures of the information conveyed by neuronal activity; a simple formula has been shown to discriminate the information transmitted by individual spikes from the positive or negative contributions due to correlations (Panzeri et al., 1999). Here, this analysis, previously applied to recordings from small ensembles, is developed further by considering a model of a large ensemble, in which correlations among the signal and noise components of neuronal firing are small in absolute value and entirely random in origin. Even such small random correlations are shown to lead to large possible synergy or redundancy, whenever the time window for extracting information from neuronal firing extends to the order of the mean interspike interval. In addition, a sample of recordings from rat barrel cortex illustrates the mean time window at which such corrections dominate when correlations are, as often in the real brain, neither random nor small. The presence of this kind of correlations for a large ensemble of cells restricts further the time of validity of the expansion.

Type: article

Keywords: mutual information, population coding, short-time expansion, temporal coding, somatosensory system, barrel cortex

### Downloads

### BibTeX entry

@ARTICLE { bezzi:02b, AUTHOR="Bezzi M. and Diamond, M. and Treves, A.", JOURNAL="Journal of Computational Neuroscience", NUMBER="165", TITLE="Redundancy and synergy arising from correlations in large ensembles.", VOLUME="12", YEAR="2002", }

Sony CSL authors: Olivier Coenen

### Abstract

Continuous and real-time learning is a difficult problem in robotics. To learn efficiently, it is important to recognize the current situation and learn appropriately for that context. To be effective, this requires the integration of a large number of sensorimotor and cognitive signals. So far, few principles on how to perform this integration have been proposed. Another limitation is the difficulty to include the complete contextual information to avoid destructive interference while learning different tasks. We suggest that a vertebrate brain structure important for sensorimotor coordination, the cerebellum, may provide answers to these difficult problems. We investigate how learning in the input layer of the cerebellum may successfully encode contextual knowledge in a representation useful for coordination and life-long learning. We propose that a sparsely-distributed and statistically-independent representation provides a valid criterion for the self-organizing classification and integration of context signals. A biologically motivated unsupervised learning algorithm that approximate such a representation is derived from maximum likelihood. This representation is beneficial for learning in the cerebellum by simplifying the credit assignment problem between what must be learned and the relevant signals in the current context for learning it. Due to its statistical independence, this representation is also beneficial for life-long learning by reducing the destructive interference across tasks, while retaining the ability to generalize. The benefits of the learning algorithm are investigated in a spiking model that learns to generate predictive smooth pursuit eye movements to follow target trajectories.

Type: article

Keywords: cerebellum, learning, plasticity, maximum likelihood, maximum likelihood, sparse, statistically independent, neuroscience

### Downloads

### BibTeX entry

@ARTICLE { coenen:01b, AUTHOR="Coenen, O. J.-M. D. and Arnold, M.P. and Sejnowski, T.J. and Jabri, M.A.", JOURNAL="Autonomous Robots", MONTH="November", NUMBER="3", PAGES="291--297", TITLE="Parallel fiber coding in the cerebellum for life-long learning", VOLUME="11", YEAR="2001", }