_PROJECT

VIRTUAL MARIONETTE

Virtual Marionette is an interdisciplinary research on digital puppetry that brings the art of puppetry into the world of digital animation. Inspired in the traditional marionette our intention is to study novel interfaces as an interactive environment for creating artistic contents based on computer animated puppets. The overall goal of this project is to research and deploy methods and techniques for the manipulation of articulated puppets in real-time to define an interaction model for digital puppetry.

Project Overview

REFERENCES

WebATIC V01 – The first Web site for work in progress like this one, with a blog (diary)
WebATIC V02 – The second Web site was build for opening and presenting the project to the entire world wide web community, supporting Portuguese and English languages.

METHODOLOGY

Sometimes we just want to tell a story with simple puppet animation, but then, we take so much time producing the animation that we forget the story line.

Digital puppet animation based on key-frame techniques is laborious and time consuming, especially for non-experts.

We propose to build an interaction model for digital puppetry using traditional marionette methods as a starting point. Seeking new ways for users to interact with performing objects in real-time, which can be used as a storytelling technique in educational and entertainment environments.

Traditional puppetry techniques present an intuitive interaction between the manipulator and the puppet. With digital puppetry the animation is produced in real- time focusing the improvisation of the performer making it more spontaneous, which is great for live shows and storytelling. This real-time technique presents an interactive way for working with narratives, turning story time into life.

The classic marionette model will be used as reference for building methods for the puppet interaction, because of the great advantages in handling multiple points for manipulation. Furthermore then the study of marionette mechanics the project will also seeks to highlight and recreate cultural and artistic aspects related to the marionette, like the puppet theater.

TARGET

Our main focus is the pedagogical field, developing a virtual puppet show prototype to be used as a learning tool for children. An application for storytelling that allows children explore the magical and fascinating world of puppets in a simpler and easy manner. A learning tool, that brings the creative and inventive world of puppets into the computer animation. Our intention is to measure and evaluate through experiments, how children respond to the interaction with the puppets and the success of these applications as learning tool.

A collaborative tool which let the users create their puppets (rigging, physics behavior) mapping them to any kind of interface using the interaction model.

PROJECT BACKGROUND

It is intended to follow up the previous studies, published in the Master’s thesis “Virtual Puppet” [1] with the prototype “ANIMATIC” 1 and thus further research in the areas of interaction and in the real-time animation.

1 Prototype of Virtual Marionette: ANIMATIC – http://www.grifu.com/animatic

This research began at 2004 with a similar project:
2004 – CBS | CARTOON BROADCAST SYSTEM: A real-time animation system for television.
This was the first project presented for the Masters in Multimedia at FEUP.
CBS.ZIP - Windows Prototype for download
CBS_APRESENTA.ZIP - Windows application with the presentation of CBS.

2006 – Animatic v0.1 |Digital Marionettes: Real-time animation with puppets.
New approach based on marionettes. The first prototype was just a rag-doll test with springs.
Animatic_v0.1 - ONLINE demo (shockwave player needed) build with Macromedia Director

2007 – Building and testing, releasing several versions, now using strings and a virtual controller.
Animatic_v0.4 - ON-LINE demo (shockwave player needed) build with Macromedia Director

2008 – Animatic final prototype. A Web and Desktop 3D application with physics (dynamics) for real-time animation of a virtual marionette with several interfaces (mouse, keyboard, game-pad, microphone) with a build in face animation for real-time lip-sync.
animatic_v08b_PC.zip - Windows Version: Full unlimited version (prototype)
animatic_v08b_MAC.zip - Mac Version: A limited release| Without support for game-pad
animatic_v08_internet.htm - Web Version: Very limited release | Just with mouse and keyboard support

2008 – Virtual Marionette Publication for Masters in Multimedia at FEUP – Nuno Correira as arguer
Marionetas_Virtuais.pdf - Master Thesis in Multimedia Technology at Faculty of Engineering of University of Porto

2009 – New Animatic release with wiimote support for interacting with the marionette. At this time there was a great effort for building a multi-touch system in Director for interaction with the marionette using the Myrion Xtra.
AnimaticPC_v09.zip - Windows Version: Full unlimited version (prototype) supporting the WIIMOTE
animatic_v10a03_NET.html - Web Version: Very limited release | Just with mouse and keyboard support

2010 – The project was adapted for the PhD program with a new direction

 

PUBLICATIONS

1. Leite, L.: Marionetas Virtuais – Animação interactiva em tempo real, Master Thesis in Multimedia Communication Technologies, Engineering Faculty of the University of Porto (FEUP), Porto, Portugal (2006)

2. Leite, L., Orvalho, V.: Anim-actor: understanding interaction with digital puppetry using low-cost motion capture. In Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (ACE ”11). ACM (2011)

3. Leite, L.: Virtual Marionette. In Proceedings of the 2012 International Conference on Intelligent User Interfaces (IUI ”12). ACM (2012).

4. Leite, L., Orvalho, V.: Shape Your Body – Control a Virtual Silhouette Using Body Motion. In Proceedings of the 2012 International Conference on Human Factors in Computing Systems (CHI ”12). ACM (2012). (Accepted publication)

ACKNOWLEDGE

João Paulo Seara Cardoso (The Puppet Master), Veronica Orvalho (Supervisor), Rui Torres (Co-Supervisor), Pimenta Alves (PdH Director), Marcelo Lafontane (Puppet Master), Sérgio Rolo (Puppeteer), Rui Pedro (Puppet Creator), Nuno Correira (my Master Thesis arguer), Helena Santos and Karin Wilkins (phd methodologies Professors)