Virtual Marionette

[vimeo 78027358 w=580&h=326]

INVERSUS – artistic installation that explores interaction by using every-day objects

Why should a lamp be used only to illuminate ? … a speaker to play a sound ? ….or a fan to produce wind ?

This every-day objects are used commonly as transmission devices (light, sound and wind).
I knew, before starting this project, that a speaker could be used as a microphone.

My proposal was to find out if a transmission device could be used as a reception device for interaction purposes.

The Lamps (in this case: Leds) where used as light sensors, the speakers (in this case: piezzos) where used as pressure sensors and the fan was used as a blowing sensor.

The main concept of inversus was to invert the meaning of transmission devices into reception devices. A sensitive machine that capture human interaction to produce sound and visual kinetics. A performing instrument that gives life to a mechanic flower that spins when someone blows into the machine producing an animated shadow like in shadow puppetry. There is also a virtual marionette inside the machine that reacts to the pressure of the pads, this marionette is rigged with bones that are mapped to the pads that make them squash and stretch producing animation.
This instrument was made from a washing machine changing the meaning absorbing the spinning metaphor as a mixing machine. To extend the mixing metaphor i used a spinning wheel that creates a hypnotic effect.

Oct.2013
+ info at http://www.grifu.com/vm/?p=883#more-883
or http://www.virtualmarionette.grifu.com/

Virtual Marionette project was mentioned in the monthly news of Instituto Telecomunicações (IT)

“(…)

Instituto de Telecomunicações has participated in the science show that was held at the Pavilhão do Conhecimento – Ciência Viva in Lisbon. A team from Porto Interactive Center presented three demos: the project LIFEisGAME, which explores the possibility of teaching people with Autism Spectrum Disorder to recognize facial emotions, using real time synthesis and automatic facial expression analysis; Virtual Marionette, a research on digital puppetry via an interdisciplinary approach that brings the art of puppetry into the world of digital animation, and 3D Scanner, a system to create a high-quality human like 3D facial avatar by drastically improving the quality of 3D avatars and reducing the time spent in the process. From IT at Lisbon came BITalino, described in page 1, and a couple of demos of the Internet of the Future concept based on RFID. The stand from IT attracted more than 500 very interested visitors.”

link to the paper

SIC National Television broadcasted a news report from Porto Interactive Center showing our projects.

Although the focus was Life Is Game project, many other projects were shown, and Virtual Marionette also appear in this report.

Circus on the Strings is a very interesting puppetry play showing the great manipulation skills from Viktor Antonov.

Although FIMP (Porto International Marionette Festival) doesn”t focus too much in traditional marionettes, Circus on the Strings was that kind of show that fulfill the expectation of a more traditional audience.

Each time that i a see great puppeteer manipulating with great precision string puppets, i just wonder how they achieve this level of manipulation. It´s very difficult to handle so many strings with such precision giving life to puppets in a expressive way.

Viktor brought to FIMP this amazing work, a classic puppetry play with superb details.

The puppet mechanics were simply amazing; puppets that move their heads, mouths, eyes, even the color of their heads. To control all this parts, the manipulation controllers were very sophisticated.

Ellen Fullman started to develop the Long String Instrument in 1981.
An instrument with dozens of metallic strings with more then 15 meters.

Ellen plays with the fingers covered with rosin-coated producing a chorus of organ-like partials.

I had the privilege to participate in Ellen´s workshop at FIMP (Porto International Marionette Festival)

Beyond the magnificent experience i realized the connection between manipulating a marionette and a instrument like this.

Ellen must have full concentration to manipulate dozen of strings with great precision.

Her body needs to be in a perfect balance moving from one place to another producing the different tones.

In this picture it is possible to view the resonance wooden boxes and Konrad Sprenger with his guitar.

Konrad uses a guitar as an interaction interface for making music without playing it directly.

He uses solenoids in each string to produce sound in an electric guitar. With servo motors attached to the tuners of the guitar he can control and manipulate the tone of the strings. All the manipulation is made inside the computer in a MaxMSP patch.

VIDEO: Konrad guitar

INVERSUS – The Sensitive Machine

Inversus is the sensitive machine that makes no sense. An artistic installation exploring interaction with common objects.

Why should a lamp be used only to illuminate?

Lamps, speakers or fans are usually used as output interfaces, what would happen if we turn the output into an input interface? Inversus explores this inversion by using Lamps as light sensors, speakers as pressure sensors and fans as blowing sensors. The main concept of Inversus is to invert the meaning of transmission devices into reception devices. A sensitive machine that capture human interaction to produce sound and visual kinetics. A performing instrument that gives life to a mechanic flower that spins when someone blows into the machine producing an animated shadow like in shadow puppetry. There is also a virtual marionette inside the machine that reacts to the pressure of the pads; this marionette is rigged with bones that are mapped to the pads that make them squash and stretch producing animation. Virtual and real animation is generated based on human interaction. Made from a washing machine this audiovisual instrument makes the sense of things changing by spinning a colorful wheel.

There are 3 different types of interaction

1. by touching 4 color pads that produce sounds in the same manner as when playing a drum (pressure sensitive), the virtual marionette reacts to this interaction by moving the arms and legs

2. by passing with the hands above 3 leds which produce sound like an organ (sound keeps playing until the hand moves away from the led)

3. by blowing a fan the frequency of the sound changes and the flower starts to spin.

Inversus ( Transmission -> Reception)

This is a 3D picture simulating the appearance of the installation

Inversus – interactive installation 3D simulation

Picture from “CHEIA” exhibition at Póvoa de Varzim – Portugal (from 5th till 31th October)

Inversus interactive installation Póvoa do Varzim Oct. 2013

(more…)