Our experiments require that the agents are fully grounded in the world through a sensori-motor apparatus and that they can conceptualise the world. So we are carrying out research into vision and motor control to build up and maintain world models usable by the language systems, and we are developing a framework called IRL for the automated planning of complex conceptualisations. IRL has been released through emergent-languages.org

Team: Michael Spranger

Collaboration: Joris Bleys (VUB), Martin Loetzsch

The Playground Experiment aims at showing how a robot equipped with an intrinsic motivation system, and in particular artificial curiosity, can explore its environment autonomously and develop skills which were not pre-specified, and with an increasing complexity for an extended period of time.

Links: the homepage of the playground experiment

Participants: Pierre-Yves Oudeyer, Frédéric Kaplan, Verena V. Hafner and A. White

This research is concerned with the elaboration of a unified theory about the following three fundamental questions concerning the origins of the sound systems of human languages:

1. Origins of combinatoriality anf phonemic coding: how do we get from holistic to digital speech, in which syllables are composed of re-usable parts ?
2. Origins of phonotactics: how do we explain the structural regularities of phoneme inventories ?
3. Origins of a culturally shared speech code: How can a society of agents develop a shared system of vocalizations ?

The hypothesis that is explored is that the origins of speech can be understood only as the result of complex dynamical interactions between speaking and listening individuals, each of them being a complex system in which the vocal tract, the ear, and the neural system that connects them are coupled.

The dynamics of complex systems is difficult to understand, and one of the best tools to study them is computer modeling. This is why societies of artificial agents were built. Agents were endowed with artificial vocal tracts, ears and brains, in order to explore the possible mechanisms that could explain the origins of shared speech systems.

Architecture of the coupling between sensorimotor modalities
and between agents

Through these computational experiments, we have shown that from a minimal neural kit for vocal replication, a shared combinatorial speech code with structural regularities and diversity can spontaneously self-organize in a population of agents. This allows to understand that the evolutionary step from vocal replication systems to modern human speech systems might have been rather small.

The aim of these experiments is primarily exploratory. It is not supposed to prove directly what mechanisms were used for humans, but rather develop our intuitions and help structuring the research debate. In particular, we think that building artificial systems allows us to shape the search space of possible answers, in particular by showing what is sufficient and what is not necessary.

Links: More details on computational modeling of the origins of speech sounds.

The Robot’s Playroom was designed to offer new learning opportunities and exploration spaces for the Sony AIBO, and to test curiosity-driven learning algorithms. Thanks to this experiment the robot can now draw, ride a bike, control switches, pick up everyday objects, watch itself in a mirror, and more …
Team: Frédéric Kaplan, Pierre-Yves Oudeyer,

Collaborations: Martino d’Esposito, and ECAL Design Students

Question: Can we train a robot like a dog?

Some techniques used for animal training might be helpful for solving human robot interaction problems in the context of entertainment robotics. We present a model for teaching complex actions to an animal-like autonomous robot based on “clicker training”, a method used efficiently by professional trainers for animals of different species. Our implementation of clicker training on an enhanced version of AIBO, Sony’s four-legged robot, suggests that this new method can be a promising technique for teaching unusual behavior and sequences of actions to a pet robot.

Participants: Frédéric Kaplan, Pierre-Yves Oudeyer, and Eniko Kubinyi and Adam Miklosi from the Ethology Department of the Eötvös University (Budapest, Hungary)

Question: What are the mechanisms needed to learn the meaning of new words in natural social contexts? How are the social regulation mechanisms involved in language learning? How can one draw the attention of a robot towards particular aspects of their environment? What are the interactions between acquisition mechanisms and language evolution?

We investigate the mechanisms that enable humans and robots to learn new words and to use them in appropriate situations. We have built a number of robotic and computational experiments studying the mechanisms of concept formation, joint attention, social coordination and language games, and articulating the roles of learning, physical and environmental biases in language acquisition. The unifying theme of all these experiments is development: we explore the hypothesis that language can only be acquired through the progressive structuring of the sensorimotor and social experience. These experiments are described in the papers below.

Interaction between an AIBO and its human trainer in the
AIBO’s First Words experiment.

Participants: Luc Steels, Masahiro Fujita [Sony DCL Tokyo], Frédéric Kaplan, Angus McIntyre, and Pierre-Yves Oudeyer