Virtual Marionette

Patricio Gonzalez had a very interesting idea to track InfraRed dots using PingPong balls.

Then, he uses CCV which sends TUIO messages to a openFrameWork application that can be downloaded here

Leap Motion – A low-cost hand/finger interface

Similar to the Microsoft Kinect, Leap Motion presents a powerful hand interface that can be used as a controller device for digital puppetry or for many other applications like painting, modeling…

Users can define their own gestures and mapped them using the free SDK delivered with the piece of hardware.

“Leap represents an entirely new way to interact with your computers. It””s more accurate than a mouse, as reliable as a keyboard and more sensitive than a touchscreen. For the first time, you can control a computer in three dimensions with your natural hand and finger movements.”

Wrist-worn 3D hand tracker (Digits – “on the move” interface)

Microsoft Kinect 2 will be released in 2013 and probably one of the new features will be finger tracking. Until then there is a wearable hardware device presented in 2012 by the Microsoft research division in the UK that recognizes and track with accuracy the user´s hand movement.

Components:

1. Inertia Sensor (measurement unit)
2. InfraRed diffuse ilumination
3. InfraRed Camera
4. InfraRed Laser Line

“Our hands are extremely dexterous, making them the primary mechanism to manipulate and interact with the physical world. Understandably a considerable focus of HCI* research has been in transferring such ‘natural’ hand manipulations into the digital domain. However, current user interfaces rarely leverage the full dexterity of our hands,”

Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor (ACM paper)

http://kotaku.com/5950393/microsofts-new-gadget-lets-you-control-any-device-with-a-mid+air-gesture

Softkinect presented in CES 2012 a virtual puppet show to demonstrate finger tracking feature, a gesture control system to tout “near mode” . The firmware for its DepthSense 311 should detect finger movement from as close as 15cm (vs. Kinect””””””””””””””””s 50cm) and as far away as about three feet.

A “puppet show” app that let you control two cartoon puppets with ragdoll arms

The puppets could nod their head, twist around, and open their mouths when you un-balled a fist. The twisting mechanism didn””””””””””””””””t work perfectly, and touching your hands together resulted in rather weird behavior.

Link to a video HERE

other video

Microsoft Kinect

Kinect is based on software technology developed internally by Rare, a subsidiary of Microsoft Game Studios owned by Microsoft and range camera technology by Israeli developer PrimeSense, which interprets 3D scene information from a continuously-projected infrared structured light.This 3D scanner system is called Light Coding, employing a variant of image-based 3D reconstruction.

The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot and is designed to be positioned lengthwise above or below the video display. The device features an “RGB camera, depth sensor and multi-array microphone running proprietary software”, which provide full-body 3D motion capture, facial recognition and voice recognition capabilities.

Arduino duemilenove

Arduino is an open-source electronics prototyping platform, designed to make the process of using electronics in multidisciplinary projects more accessible.

A physical computing interface that helps becoming real our imagination

Playstation Move

PlayStation Move is a motion-sensing controller interface for the Playstation 3.

A handheld motion controller wand that uses the Playstation Eye camera to track the wand´s position, and the inertial sensors in the wand to detect its motion.

Nintendo Wiimote

Nintendo WIImote and WIImote plus

The Wiimote is a low-cost interface that contains a 3 axis accelerometer, a high resolution IR camera, a bluetooth connection and vibration surface.

This was already tested in the Animatic Project, with very good results

I´ll try the IR camera as a tracking system and the accelerometers for a direct manipulation

In the Animatic Project the main interaction interfaces were the keyboard, mouse, gamepad, microphone and in the last version the wiimote.

The WIImote was one of the best interfaces to control the virtual marionette because it was very near to a real marionette controller.
After this last prototype i was working with Myron Xtra for Director to motion capture globs with camera. The idea was to control the marionette with our own hands. But the result was not great.

Now, the next step is to try new approaches  with the WIImote and WIImote Plus, multitouch surfaces, simple body motion capture, Brain Computer Interface, the new Sony Move and the newest Microsoft Kinetic. This last one, as great capabilities for controlling a puppet in realtime.