The case grammar experiment investigates how a population of autonomous artificial agents can self-organize a grammatical language that features similar properties as those found in case-languages such as Latin, German and Turkish. The case grammar experiment is the most advanced experiment in artificial language evolution to date: it is the first multi-agent experiment ever that features multifunctional, polysemous grammatical categories as opposed to other experiments in the field in which the agents always converge on one-to-one mappings.

In the experiment, the agents have to describe dynamic real-world scenes to each other (click here to see an example movie). Starting with a lexicon but without grammar, the agents gradually start to introduce case markers in order to reduce the cognitive load they need for interpreting sentences. For example, an utterance such as boy girl pushed is ambiguous as to whom did what to whom, so the hearer has to witness the scene to find the correct meaning. Adding more grammar could thus avoid this effort by directing the hearer to the correct interpretation. For example, in the utterance boy-bo girl-ka push, the grammatical markers can be used for indicating that the girl was being pushed and that the boy was the pusher.

Through analogical reasoning over event structures, agents can then generalize these markers to become more abstract. In the experiments, we typically see the emergence of agent-like and patient-like case markers, as in the following sentences:

1. jack-fuitap walk-to jill-ginah
   Lit.: jack-sem-role-6 walk-to jill-sem-role-26
   ‘Jack walks to Jill.’
2. touch jill-fuitap house-payis
   Lit.: touch jill-sem-role-6 house-sem-role-29
   ‘Jill touches the house.’
3. house-woechen move-inside boy-fuitap
   Lit.: house-sem-role-10 move-inside boy-sem-role-6
   ‘The boy moves inside the house.’

Team: Luc Steels, Remi van Trijp

Our experiments require that the agents are fully grounded in the world through a sensori-motor apparatus and that they can conceptualise the world. So we are carrying out research into vision and motor control to build up and maintain world models usable by the language systems, and we are developing a framework called IRL for the automated planning of complex conceptualisations. IRL has been released through emergent-languages.org

Team: Michael Spranger

Collaboration: Joris Bleys (VUB), Martin Loetzsch

In order to conduct experiments in language evolution it was necessary to develop a solid flexible formalism for the representation of linguistic knowledge and for orchestrating the parsing, production and learning processes necessary for language interaction. In collaboration with the VUB AI laboratory we have developed Fluid Construction Grammar (FCG). FCG is within the tradition of feature structure based, unification oriented grammars. It has some unusual features including bidirectionality, scoring of rules, and special operators (particularly the J-operator) for dealing with hierarchy. FCG is fully operational and can be downloaded from www.fcg-net.org.

Collaboration: VUB AI LAB (Joachim De Beule, Joris Bleys, Pieter Wellens, Martin Loetzsch)

This research is concerned with the elaboration of a unified theory about the following three fundamental questions concerning the origins of the sound systems of human languages:

1. Origins of combinatoriality anf phonemic coding: how do we get from holistic to digital speech, in which syllables are composed of re-usable parts ?
2. Origins of phonotactics: how do we explain the structural regularities of phoneme inventories ?
3. Origins of a culturally shared speech code: How can a society of agents develop a shared system of vocalizations ?

The hypothesis that is explored is that the origins of speech can be understood only as the result of complex dynamical interactions between speaking and listening individuals, each of them being a complex system in which the vocal tract, the ear, and the neural system that connects them are coupled.

The dynamics of complex systems is difficult to understand, and one of the best tools to study them is computer modeling. This is why societies of artificial agents were built. Agents were endowed with artificial vocal tracts, ears and brains, in order to explore the possible mechanisms that could explain the origins of shared speech systems.

Architecture of the coupling between sensorimotor modalities
and between agents

Through these computational experiments, we have shown that from a minimal neural kit for vocal replication, a shared combinatorial speech code with structural regularities and diversity can spontaneously self-organize in a population of agents. This allows to understand that the evolutionary step from vocal replication systems to modern human speech systems might have been rather small.

The aim of these experiments is primarily exploratory. It is not supposed to prove directly what mechanisms were used for humans, but rather develop our intuitions and help structuring the research debate. In particular, we think that building artificial systems allows us to shape the search space of possible answers, in particular by showing what is sufficient and what is not necessary.

Links: More details on computational modeling of the origins of speech sounds.

Olafur Eliasson is an artist who emphasizes process instead of final form in a lot of his work. This makes his art a perfect fit with the research of Luc Steels, whose work on the origins of language has always focused on the dynamic processes from which linguistic forms originate and change. Eliasson and Steels therefore collaborated on the “Look into the Box” installation. It was first shown at the Musee d’Art Moderne de la Ville de Paris in 2002 and came out of discussions held at the Bridge the Gap symposium in 2001 in Kitakyushu (Japan) between Steels and Eliasson.

Team: Luc Steels, Nicolas Neubauer

Question: What are the mechanisms needed to learn the meaning of new words in natural social contexts? How are the social regulation mechanisms involved in language learning? How can one draw the attention of a robot towards particular aspects of their environment? What are the interactions between acquisition mechanisms and language evolution?

We investigate the mechanisms that enable humans and robots to learn new words and to use them in appropriate situations. We have built a number of robotic and computational experiments studying the mechanisms of concept formation, joint attention, social coordination and language games, and articulating the roles of learning, physical and environmental biases in language acquisition. The unifying theme of all these experiments is development: we explore the hypothesis that language can only be acquired through the progressive structuring of the sensorimotor and social experience. These experiments are described in the papers below.

Interaction between an AIBO and its human trainer in the
AIBO’s First Words experiment.

Participants: Luc Steels, Masahiro Fujita [Sony DCL Tokyo], Frédéric Kaplan, Angus McIntyre, and Pierre-Yves Oudeyer

Question: How do language users develop a shared communication system?

The Talking Heads experiment studied the evolution of a shared lexicon in a population of embodied software agents. The agents developed their vocabulary by observing a scene through digital cameras and communicating about what they have seen together. To add an extra level of complexity to their task, agents were able to move freely between different computer installations located in different parts of the world. Members of the public were able to influence the course of the experiment by logging on to the Talking Heads website to create and teach their own agents.

Links: The Talking Heads Experiment

The ‘naming game’ experiments used computer simulations of communities of language users to explore the emergence of shared lexicons in a population. In the naming game, software agents interact with each other in a stylised interaction (termed a ‘language game’). Repeated interactions lead to the development of a common repertoire of words for naming objects. By varying experimental parameters, it is possible to explore the effect of environmental factors such as noise and uncertainty, memory limitations, and contact between different language groups. (more…)