Developmental Robotics

The PlaygroundGenerating plausible models for the processes underlying children’s development in the first years of their life is a challenging scientific issue at the crossroads of neuroscience, learning theories and developmental psychology. Children seem to acquire new know-how in a continuous and open-ended manner. A large amount of work describes how new skills seem to build one upon another, suggesting a continuum between sensory-motor development and higher cognitive functions. But very few plausible low-level mechanisms exist to explain how such skills emerge or self-organize.

Studying development is intrinsically difficult because of the complex interplay between embodiment, learning mechanisms and environmental dynamics. A relevant integrative approach can be pursued by viewing development as a complex system the dynamics of which can be studied with embodied models. In order to capture part of the open-ended nature that characterizes children?s development, we design new biologically-inspired architectures to control autonomous robots. In particular, we conduct research on motivational principles that can drive a robot to continuously try to master new know-how. The aim is to construct engines implementing such general capacities as “curiosity”, thus producing generic attention mechanisms with a minimum of preprogrammed biases.

This approach might not only help us understand the mechanisms underlying human development, but it might also provide radically new techniques for building intelligent robots. Indeed, as opposed to the work in classical artificial intelligence in which engineers impose pre-defined anthropocentric tasks to robots, the techniques we develop endow the robots with the capacity of deciding by themselves which are the activities that are maximally fitted to their current capabilities. Our developmental robots autonomously and actively choose their learning situations, thus beginning by simple ones and progressively increasing their complexity. No tasks are pre-specified to the robots, which are only provided with an internal abstract reward function. For example, in the case of the architecture which we developped, this internal reward function pushes the robot to search for situations where its learning progress is maximal.

Keywords : developmental robotics, epigenetic robotics, intrinsic motivation, curiosity, values, development, intrinsically motivated reinforcement learning, autonomy, epigenetic robotics, behavior, developmental trajectory, complexity, active learning.

Members: Frédéric Kaplan, Pierre-Yves Oudeyer, Verena Hafner

The Playground Experiment

Playground Experiment cover image

The Playground Experiment aims at showing how a robot equipped with an intrinsic motivation system, and in particular artificial curiosity, can explore its environment autonomously and develop skills which were not pre-specified, and with an increasing complexity for an extended period of time.

Links: the homepage of the playground experiment

Participants: Pierre-Yves Oudeyer, Frédéric Kaplan, Verena V. Hafner and A. White

A Robot’s Playroom

A Robot’<p>s PlayroomThe Robot’s Playroom was designed to offer new learning opportunities and exploration spaces for the Sony AIBO, and to test curiosity-driven learning algorithms. Thanks to this experiment the robot can now draw, ride a bike, control switches, pick up everyday objects, watch itself in a mirror, and more …
Team: Frédéric Kaplan, Pierre-Yves Oudeyer,

Collaborations: Martino d’Esposito, and ECAL Design Students

Robot clicker training

Question: Can we train a robot like a dog?

Some techniques used for animal training might be helpful for solving human robot interaction problems in the context of entertainment robotics. We present a model for teaching complex actions to an animal-like autonomous robot based on “clicker training”, a method used efficiently by professional trainers for animals of different species. Our implementation of clicker training on an enhanced version of AIBO, Sony’s four-legged robot, suggests that this new method can be a promising technique for teaching unusual behavior and sequences of actions to a pet robot.

Robot clicker training

Participants: Frédéric Kaplan, Pierre-Yves Oudeyer, and Eniko Kubinyi and Adam Miklosi from the Ethology Department of the Eötvös University (Budapest, Hungary)

AIBO’s First Words

Question: What are the mechanisms needed to learn the meaning of new words in natural social contexts? How are the social regulation mechanisms involved in language learning? How can one draw the attention of a robot towards particular aspects of their environment? What are the interactions between acquisition mechanisms and language evolution?

We investigate the mechanisms that enable humans and robots to learn new words and to use them in appropriate situations. We have built a number of robotic and computational experiments studying the mechanisms of concept formation, joint attention, social coordination and language games, and articulating the roles of learning, physical and environmental biases in language acquisition. The unifying theme of all these experiments is development: we explore the hypothesis that language can only be acquired through the progressive structuring of the sensorimotor and social experience. These experiments are described in the papers below.

AIBO’s First Words

Interaction between an AIBO and its human trainer in the
AIBO’s First Words experiment.

Participants: Luc Steels, Masahiro Fujita [Sony DCL Tokyo], Frédéric Kaplan, Angus McIntyre, and Pierre-Yves Oudeyer

Can a dog tell the difference?

Question: Do dogs see AIBO, Sony’s four-legged robot, as a conspecific?

We are conducting a series of exploratory studies on animal robot interactions in collaboration with the ethology group of the University of Eötvös University (Hungary). The purpose of these experiments is to investigate, from an ethological point of view, how much dogs see AIBO as a conspecific. The questions adressed are: what is the influence on the dog’s reactions of movement, smell, presence or absence of eyes, sounds, etc.

Two kinds of situations are tested. In the first one, puppies and adult dogs interact freely with the robot. In the second one, we organise a situation of implicit competition in which the dog has to defend a piece of meat against the arrival of the robot. Comparative studies are done with a remote control car and a real puppy. The results are being analysed and will be published in the near future.

Examples

AIBO and Hungarian Vizsla

Hungarian Vizsla/robot competition (first pilot study, April 2000)

AIBO and Mops

Mops/robot interaction (first pilot study, April 2000)

AIBO attacked by dog movie screenshot

Dog attacks AIBO movie

The horrible screams that you hear at the end of the movie were made by the experimenters, who were startled to see the dog attack the AIBO.

This was the first time that the AIBO was attacked, but it was not the last. During the course of the experiment, the AIBO was sometimes knocked over, bitten and chewed. It is still in perfect working order, and shows no visible signs of damage.

Nevertheless, we strongly advise you not to try anything similar with your AIBO. AIBO is strongly built, but it contains many delicate components that could be easily damaged. Your warranty will not cover you if AIBO is damaged in this way.

No animals were hurt or mistreated in any way during the course of this experiment.

Participants: Frédéric Kaplan, and Eniko Kubinyi and A. Miklosi from the Ethology Department of the Eötvös University (Budapest, Hungary)