click here for a movie narrative AVI presentation (15 Mb !):
E-Sparks in synthesys
E-Sparks is a creative exploration of an artificial society that autonomously develops the own language structure through interaction with the human beings. A digital creatures population is called to learn from the humans through spoken dialog interaction. People can teach them sentence relationships, i.e. some insight about semantic networks. The creatures answers are a mosaic of sentences learned from humans, sometimes incoherent, sometimes stimulating and provocative. A social language emerges after a number of interactions, adapting and reflecting the visitors culture. The role of the artificial becomes an inter-humans cultural transmitter reflecting the roots of our mind and society.
go here for a description of the E-sparks theoretical framework -> The Artist Statement
The E-Sparks interactive installation
E-Sparks is a metaphor of a digital society of beings represented as “real living” beings, able to evolve and to develop a language interacting with humans. Through interaction, people can play with the creatures and share communicative experiences.
In the installation, three-dimensional "digital creatures" move inside a "real-virtual hybrid world" projected on a wall screen. The creatures, modelled as a real existing specie of plankton, appear as three-dimensional shapes pulsating and fluctuating with movements similar to the real plankton, animated by social behaviour visually similar to swarm formations.
In front of the screen, the visitors can interact through gestures and voice. The creatures are equipped with "sensors" that make them able to see and listen to the visitors, and with an “artificial brain” allowing them capable to process visual and sound information, to control the movements, to learn words by the visitors and to produce vocalizations. The “eye” is attached to a video-camera that captures the shadow of the visitors projected over the screen. The “ear” is attached to a microphone where the visitor can speaks. The brain of the creature is built using associative memory and neural networks, the base of learning capabilities.
Interacting with the visitors, the creatures obtain the energy necessary to live and to strengthen their behaviour. The creature can learn two basically ways of interaction with the humans. The first kind of interaction is the tracking of the visitor shadow projected on the screen. This is obtained through a continuous “trail and error” adaptation of their neural network controlling the creature movement. Moving towards the shadow, where energy is released by the visitor, is their task. This task is quite simple and convergence is obtained after few hours of interaction.
The most intriguing interaction is realized through spoken sentence exchange between creatures and humans. When a visitor speaks into the microphone, the creature memorizes his words and sentences organizing a sort of free association of semantic units. After a voice stimulus, the creature closer to the visitor try to formulate an answer. The first step is the classification of the sentence in respect to the units in the memory (“elastic matching”). The second step is to search for eventual semantic associations with the already memorized units, otherwise an emulation principle is applied.
The answer is a mosaic of sentences learned from other previous visitors. None predefined vocal material is inserted at the begin of a digital society, making the installation able to manage different languages. The answers are generally incoherent, especially during the first phases of the evolution, but after some run they results stimulating and provocative in respect to the visitor sentence. The play is unconsciously driven by the visitor which uses often irony and personification of the digital creatures.
The exchange generally assumes the form of a theatrical fiction where creatures and humans dialog exploiting the intriguing fashion of language, a sort of free association game similar to what was already discovered by surrealists with the famous play of the “cadavre exquis” (1925). In that play, artists like Dali, Duhamel, Prèvert, Tanguy, Ray, Morise, Mirò, Eluard, Picasso took inspiration from the apparently free sentence combination produced by the play, in a predefined semantic context.
During the E-Sparks play, the creatures continue to establish links between “semantic units” through a statistical reinforcement of words/sentences repeated by several visitors. Furthermore the digital creatures develop their social behaviour i nteracting between them, exchanging information about the individual semantic networks weighted by their ability to interact with the people (the “stronger” creature is partially emulated by others). Although each creature is different from others, they tend to share a common base of words/sentences and semantic relationships network.
After some weeks of interaction, the ability of the creature to give “inherent” answers increases, and the play is still more stimulating for attending audience.
Growing, Adaptation, Human Reflections
Each run of the installation starts from the data of the previous run. In this way the population evolve their level of development in time. Therefore the society increases intensity and experience value in the time, absorbing words and semantic links.
Their "cultural growth" is documented in archives of information. It is possible follow the growth of the creatures in the time, and navigate in their semantic network, discovering good association and creative errors.
One of the most interesting aspects of the installation is the continuous adaptation of the creatures to the language of the visitors. A mechanism has been inserted promoting the words/sentences which more frequently/recently repeated and more linked to other semantic units. This position is based on a recent theory of the language (2002) revisited as a “small word network”, based on some keywords called “semantic hubs”.
Changing the language of the visitors (i.e. from italian to english) the creatures have to progressively erode the old words and semantic links substituting them with the new instances. In this adaptation process the creatures continuously reflect the language changes and the attitude of the visitors to respond to artificial “personified” entities.
The interesting paradox in the E-Sparks installation is that apparently the visitor is speaking with artificial creatures, where often the visitor is speaking with many other visitors who left a “trace” in the artificial society (eventually long time before). This dialog is possible through the mediation and recombination of the artificial creatures. This paradox reflects one of the most interesting role that the “artificial entities” could have in our future: a powerful vehicle for the inter-cultural transmission between humans.
not enough ? still information, images and movie on E-Sparks ?
do you want know the curious long story of the E-Sparks installation ?
If you have ideas or comments whit this project, please don't hesitate to contact us at plancton@plancton.com