Virtual Marionette

Marionette Zoo is a digital puppetry experience for LeapMotion!

It was one of the projects accepted to the Leap Motion Developer Program in early 2013 but it was finished in 2014.

The user moves the puppets with the hand that is mapped to a virtual controller similar to a traditional string controller. This controller is connected to the virtual marionette with strings. The puppets are made with physics to achieve a marionette style look.

3D was made with blender using the bullet physics.

The sound was developed with pure data.

The application was developed with Cinder.

Link to the >> Project <<

An incredible digital puppetry performance with the digital puppet in 1992 at SCES.

The actor was wearing a facial Waldo developed by The Character Shop and a 3D mouse best known has the flying mouse.

Brilliance

Robert Abel and Associates developed in 1985 the first television production employing motion capture technology with computer graphics for digital character animation. “Brilliance” was not just a television advertisement, it was one of the greatest challenges in the use of computer graphics for production, featuring techniques that had never been attempted or achieved before.

Live cartoon show for television developed by TV-animation in 1999.

An interactive television show that allows children to call into the show, appear in real-time as an animated cartoon character.

The Cartoon Broadcast System was a custom hardware/software  real-time animation system developed by TV Animation, which allows producers to have real-time lip sync of up to 16 simultaneous sources, live pan, zoom and tracking and electronic puppeteering.

Behind the scenes from Nelly Nuts Show in Portugal which was called “Rita Catita e o Ursinho Oops”. It was produced by Miragem a portuguese production company in a 30 minute daily show  and broadcasted by TVI a portuguese television channel.

Hugo was an  interactive television show created by a Danish company called Interactive Television Entertainment (ITE) in 1990. An interactive game show broadcasted all over the world (more then 40 countries) until 2007. Players at home could play with the character by calling the television show and using their telephone keys as controller. Hugo was developed on custom-made ITE hardware based on a Commodore Amiga 3000 with MIDI devices and DTFM digital converter  for control. Later it was ported to a PC-based system using motion capture for live animation. Body movement and face expression of the puppeteer was retargeted to Hugo virtual character. An early digital puppetry system with great success during 17 years in the television live entertainment.

Motion capture system used to control Hugo character

The ITE 4000 a PC-based system

A MIDI Control panel with a joystick and buttons to manipulate the ONLINE contents.

FlickerLab’s Cartoon Broadcast System (2012)

CBS is an update of the TV-animation Nelly Nut Show animation system.
It brings new features and provides more flexibility for live animation using new controls.
Lip-sync is real-time.

Here is an example of a show called “Café Central” made by a portuguese production house (HOP)

Johnny Bravo Live

In the year 2000, Cartoon networks presented and broadcasted a live cartoon show, Johnny Bravo was a 2D cartoon digital puppet animated in real-time. Johnny Bravo was a kind of “CJ” or cartoon jockey listening to request from telephone for nostalgic cartoons.

Kydara FilmBox On-Air was the first commercial solution to provide a generic real-time animation system using motion capture. It brought more flexibility to the broadcast industry because production houses or television channels could produce their own lives shows.
Producers or animators/puppeteers could trigger 3D/2D animation clip sequences in real-time.
This solution was designed to provide performance and versatility for live broadcasting.

Although the character is not fully live animated it can respond to real-time impulses creating the feeling of interaction with the audience. In a matter of fact, the only thing that is live performance is the lip-sync, it recognizes the sounds spoken by an onset actor and changes the image of the mouth to the most correspondent one.
The puppeteer just press buttons to trigger animations. The animations are previous built from dozens of animated sequences made with hundreds of frames.

The Cartoon Networks used a beta version of On-Air.

Reference: http://www.cgw.com/Publications/CGW/2000/Volume-23-Issue-8-August-2000-/Cartoon-Jockey.aspx

Medialab Paris was one of the very first production houses to explore digital puppetry.

Since 1989 Medialab team as built several dozens of real-time puppets and explore many different interaction techniques. MediaLab open a second house in LA, but what happen to all of this stuff ?

Here is a link to a very interesting documentary about real-time animation and Medialab´s production pipeline, a documentary directed by Pascal Tosi.

A music videoclip using the Wayang Kinekt from antonius (http://antoni.us/post/5155623291/nature-of-code-final)

An example of using digital puppetry in animation where novel technology meets the old traditional puppetry of Wayang.

Wayang Kinekt technology and animation by Antonius Wiriadjaja