Virtual Marionette

Inversus is a digital interactive installation that explores the relationship between user actions with common objects: lamps, speakers and fans. It is an interactive machine that shifts the conceptual understanding a user traditionally has about a specific object, making people wonder: why should a lamp be used only to illuminate?

[vimeo 126317214 w=580&h=326]

Well, sometimes you feel that you should share your presentations with the community and not just with the people that attended the conference. The problem is that my presentations are a combination of images and oral explanations, and by sharing a PDF with just the images you can not understand what it means. So, I have decided to create an animated presentation with a synthesized voice to help.

This video was made from the paper presentation at ACHI (February 2015)
It explains the concept, the process and the framework.

A Multimodal digital Ecosystem developed for live performances.

A solution for real-time virtual object manipulation, for show control, and for video routing.

for more information about the performance and the POEX project: http://po-ex.net/taxonomia/materialidades/performativas/retroescavadora-sem-titulo

[vimeo 126071525 w=580&h=326]

When you develop contents for live performances relying one just one application you might come across some difficulties in creating specific effects witch are easy to create with other tools.
You might find yourself asking, why can I use multiple applications in a live performance ?
In general the problem relies on how to handle all the video outputs from the different applications without having to switch outputs during the performance and how to control them in a seamless away.

This framework integrates multiple applications sharing one output providing a simple solution to interact and handle data, video and sound. A flexible remote control system that provides scalability and provides the right path for creators to go beyond application limits. It can even handles application crashes because the output is independent from the applications.

Eco-System features
– Shared output
– Shared control
– Flexible remote control
– Scalable data-flow to connect video, audio, and data

This eco-system was developed for the “Untitled” POEX live performance
For this performance we choose to work only with free applications.
Pure Data (puredata.info) a visual programming environment, using chdh Egrégore (chdh.net/egregore_source_manual.htm) an audiovisual instruments interaction environment developed for the egregore performance. The series of patches provides us with a particle system for the creation of an audio visual organic life that would represent the placenta.
Unity (unity3d.com) gave us the right environment for simulating a physical space where the hands of the performer could interact with virtual objects. But Unity does not handle very well typography animation.
Emotion (adrienm.net/emotion/) A real-time animation environment that presents many interesting features for text animation.

Working on a macbook retina 2.4 i5.
NOTE: there is some performance issues during the video capture because further then all the eco-system running I had to open a camera and capture the screen. The sound from this video was not made during the simulation.

On the 2th of March we will give a lecture on the puppet play “Prometeu” at the school Clara de Resende in Porto.

Picture by Pedro Martins

Almost all the production team will be present at the lecture:

Puppeteer: Marcelo Lafontana

dramaturgy: José Coutinhas

Photography: Pedro Martins

Multimedia: Luís Grifu

Scenography: Sílvia Fagundes

Luís Leite presented in the 24th of February the Inversus publication at the Eighth International Conference on Advances in Computer-Human Interactions (ACHI) in Lisbon.

This publication is now available for download at the download section.

Marcelo Lafantona won the award for best male performance with Prometeu at the 16th International Puppetry Festival in Varna – Bulgaria.

The renewed Prometeu system was performed with great success by the hands of Marcelo. A great performance presented to the Bulgarian audience. The Juri although not knowing the spoken language as chosen Marcelo´s performance as the best male performance.

The new system is more stable and flexible helping Marcelo to be more focus in his performance. Marcelo is the one man show, controlling lights, sounds, video and animation and all the puppeteering. And although it was very complex to create all the stuff is changed by the puppeteer in real-time with just one button.

A very important recognition from one of the oldest Eastern Europe festivals with a great tradition in the art of puppetry.

Congratulations to Marcelo Lafontana.

Photos from TMPC – Varna, Stage Filial – Drama Theater Stoyan Bachvarov

Visting Varna in Bulgaria to the 16th International Puppetry Festival.

Lafontana Formas Animadas was invited to present the Prometeu shadow play at this festival. One of the oldest Eastern Europe puppetry festivals with 45 years old.

We built a new compact version of Prometeu which travels inside a flat case  developed to go around the world. It was our first challenge and it passed with distinguish, Marcelo Lafontana won the award for the best performance. We were very happy to present our show at Varna.

From the 1th Octobre to 7th October I have had the privilege to watch 11 great performances from puppetry companies from all over the world.

From string to direct manipulation, from the most classical puppetry to the most contemporary approaches the Varna festival presented a wide range of puppetry aesthetics  and puppetry methods for all audiences.

I would like to congratulate the organization for there professionalism handling with all cultural differences representing 10 different countries.

Here is a list of all the puppetry shows that I was able to see,  all of them contributed in some way to opening my eyes to this wonderful theatrical art.

La Mar de Marionetas – Madrid, Spain
SPEAK, YOU´RE NOT ALONE

Sofia Puppet Theatre – Sofia
TALE OF THE WANDERING KING

Batumi Puppet and Youth State Theatre – Batumi, Georgia
COLORED FOG’S SEASHORE

State Puppet Theatre – Varna
LITTLE WIZARDS

Harpe en Perigord & La Mar de marionetas – French-Spanish co-production
SHAKESPEARE’S DAUGHTER TELLS HER MIDSUMMER NIGHT’S DREAM

Puppet Theatre Haet-Sal – Suwon, Korea
HEUNG-BU JEON

Baruti Teatro – Athens, Greece
MOMENTOS

Sofia Puppet Theatre – Sofia
THE TEMPEST

State Puppet Theatre – Plovdiv
THE LONG JOURNEY OF THE LITTLE CROCODILE

State Puppet Theatre – Varna
SILENT LEGENDS

The Key Theatre – Tel Aviv, Israel
WHEN ALL WAS GREEN

Peregrinação – A live cinema approach for paper puppetry

Digital media as an interface between puppetry and film to enhance the traditional paper theater experience.

Traditional paper puppetry shows are usually presented in small places to a small audience because of their size. We present a miniaturized multi-camera studio set as a novel way to experience puppetry to a greater audience.
The puppeteer tells the story by performing with tangible puppets in a physical set and the audience can compare this performance with the final composed image in a screen as if they were behind the cameras.
We have developed a multimedia solution based on virtual “strings” connecting computer applications, devices, and real-time effects providing all the control to one single puppeteer – the one man show.
We overcome traditional puppetry limitations by exploring digital media technology, and by mixing puppetry with cinematic techniques we present the experience of the live cinema.
Coproduction Lafontana – Animated Forms / Teatro Nacional São João (TNSJ) from the original text of Fernão Mendes Pinto, with the interpretation of Marcelo Lafontana.

Inspired by the adventures of Fernão Mendes Pinto, a portuguese explorer, reported from the book “Peregrinação” , published in 1614, Marcelo Lafontana makes a journey through strange stories that are presented in a miniature world.

Through the intersection of puppetry, in particular, the expressiveness of paper theater with the cinematic language, this small world becomes a large space of illusion in which the narrative arises.

The stage, is transformed into a film studio, where scenarios and characters drawn and cut from cardboard are handle in front of video cameras. The images are captured by a multimedia architecture providing image processing, image mixing, sound effects, chroma key, performance animation, and all happens in real-time.  The final result is projected on a screen, like a sail of a boat that opens to the charms of a journey through the imagination.

In 24th of July of 2014 I will be presenting the paper Prometeu – Ilusão em tempo real at the Avanca Festival in Portugal.

Leite, L., Redondo, M. (2014) Prometeu, Ilusão em tempo-real. Conferência Internacional de Cinema, Arte e Tecnologia. Avanca.

Prometeu is a multimedia theater play inspired by the Indonesian shadow puppetry, Wayang Kulit. This multidisciplinary project narrows the boundaries between theater and film, past and present, reality and illusion. Merging different art forms brings new experiences to the artists and to the public; in particular the amalgamation of puppetry with film and other disciplines contributes to a new concept, what we can call the performance cinema or ” Live Cinema”.

Using multimedia technologies and cinematic language this project motivates a different paradigm from the traditional reading of the narrative sequence. It promotes the discussion about the boundaries of theater and film, and the role of actors, puppeteers, animators and directors in this particular art form.

Preserving the heritage of traditional shadow theater, Prometeu overcomes the typical physical constraints found in traditional puppet theatre through film and multimedia techniques. Real-time post-production, performance animation or interaction techniques provide a new mean of creation to artists.  By using sensors the artists can expand the interaction. By using live cameras the performer becomes the director creating the sequence in real-time. Techniques such as “luma-key”, procedural animation or gesture tracking enhance the live experience making the performer into a sort of magician.

In Prometeu one puppeteer, performs with virtual and real objects and controls all the show remotely.

In this paper we present the methodology and the implementation of our technology as well as the aesthetics and conceptual aspects. 

The Blackmagic pocket camera (BMPC) is small and powerful even when you operate this camera remotely.

For project “Peregrino” we searched for small cameras with changeable lens to frame our particular small size scenes. Zoom, Focus and Iris were  features that we searched for. In our first experiment we use a Gopro camera because of the size, but the image quality and the fixed lens were a problem, this camera doesnt provide zoom,  focus, or iris. We also experiment a Sony NEX-7 and we did like the way we could remotely control with WiFi using the Sony API, but the image quality and many other tiny things like the fact the camera works only in battery mode made us search for other solutions. In the other hand the Blackmagic Pocket Camera offers a great image quality with interchangeable lens with a 16mm sensor. We could also control the Focus and Iris remotely by using the LANC input.

LANCuage the syntax to control.
LANC – Logic Application Control Bus System or Control-L

LANC is a trademark protocol made by SONY. A bidirectional communication serial port where two devices can communicate with each other. For more information about LANC just click in the fowling link: http://www.boehmel.de/lanc.htm

Other video camera brands, like Cannon also implement this protocol to take advantage of the many remote control equipments in the market, but they call it “Control” port. LANC is kind of a standard remote protocol for video cameras. Blackmagic also implements the LANC protocol in their cameras, although just a few functions are supported.
Available LANC commands in BMPC: REC, Manual focus near/far; Manual iris near/far; Auto-Focus; Auto-Iris. We are using in our play all of this functions intensively except the REC functions.

Framework to control IRIS and FOCUS on the BMPC. We use a computer software to control the LANC via MIDI but you can use any midi device if you implement a midi interface into the arduino.

QLAB is a show control application for MAC which we use in our project. A show control environment that orchestrates all the media and data flow. Sam Kusnetz provides a very interesting writing about how to prepare a machine for live performance using Qlab, visit: http://figure53.com/notes/2013-10-29-prepare-execute-troubleshoot/

The workflow: Qlab sends MIDI messages to the Hairless-midi midware software that converts standard midi to serial midi. Then, the messages are interpreted by the arduino witch sends LANC commands (pulses) into the blackmagic cameras.

[vimeo 97412299 w=580&h=326]

You can download all the files above (press more)

(more…)