pull

 

Marionette Programming Language

Pull The Strings is a digital puppetry visual node-based control environment for performance animation.
It acts as a middle-ware interface between puppeteers and puppets, between device drivers and driven objects, between input interfaces and applications.

[vimeo 158084097 w=580&h=326]

It is a marionette programming engine that works as a digital mediator between devices and applications, it provides the building blocks for the digital puppeteer to establish the manipulation semantics. It is a visual programming language inspired by the strings of the marionettes and by the patch cords that connect modules in the old analog video synthesizers. In this environment the signals are processed and sent to the network. Finally the data arrives to Remote Control for Unity which is a plugin that facilitates the mapping of OSC messages.

It was designed with a minimalistic but intuitive interface and was developed in C++ with Openframeworks making use of multiple add-ons to provide their main features.
A multidisciplinary enviorment (MMMM: Multi-plataform, Multi-modal, Multi-mediator, Multi-user).
An enviorment for artists with non-programming background. Design for digital puppeteers.

Pull the strings is a middle-ware, an interface between applications, devices, and performing objects. It can be consider a visual programming environment for remote control made for artists and designers.
Sometimes you find hard to use one environment or application that offers all those nice features that your are looking for, so why not use a multiple set of applications in real-time ?
The goal of Pull the Strings is to facilitate the use and control of all your resources found in your computer and on the network.
An abstraction from technology making use of generic signals and communication protocols.
Connects, and transforms signals from input and virtual devices into performing object controls. Map and orchestrate multimedia data using OSC, DMX, MIDI and VRPN
Is a remote control application, an interface that connects inputs to outputs and provides a series of functionalities that helps to control the animation. It is a OSC-based environment and all nodes work with OSC format.

Version Alpha 0.050316 (first release 6 March 2016)
Pull The Strings will become open source soon. It is completly free to use in your art, research, or professional projects.
Built with [openFrameworks](http://www.openframeworks.cc).
It is developed by Luís Leite (GRIFU) for the Digital Media PhD with the support of FCT – Fundação para a Ciência e a Tecnologia.

## Download

Download Pull The Strings:

[Mac OS X 10.9 (32 bits) only ]

## Demo video

## License

Copyright 2012-2016 Luís Leite

GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007

Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.

Preamble

The GNU General Public License is a free, copyleft license for
software and other kinds of works.

## Pull The Strings Interface

Pull The Strings allows you to create, save and load your projects.
You should start by always saving your project first which will create a folder to gather all the files needed.
To navigate through the canvas use the key, to open the menu use the and then start writing the operator name or use the mouse to navigate.
To delete an operator just use the to access to operators options click the <*> on the operator.
Connect nodes from the outputs to the inputs. Add new input nodes by clicking the <+ O> button.
You can create multiple graph windows.

### OSC Communication
Pull The Strings is based on OSC protocol. It receives OSC messages and sends OSC messages and all its interface communicate internally via OSC.
You can send several messages from one output port.
It supports bonjour so you can easely find Pull The strings with Bonjour compatible applications, such as TouchOSC.
It makes use of RemoteUI to remote control the operators with other applications or devices.

### Pull The Strings operators

input.number (to create a Float or Int inside the interface or with remote applications)
input.string (to create strings)
math.scale (a scalar function with a learning input range option)
osc.combine (combine several OSC messages into one, it allows message syncincing)
osc.expand (expands an OSC message into seperated out nodes, it allows to inspect the data types and choose the address)
osc.port.in (opens port for incoming OSC data)
osc.port.out (opens port for sending)
print (for debugging, it prints in the interface the incoming messages)
show.plot (makes a plot from the values)

## Version History

– Alpha 003 (March 5th 2016)

## Support

![Support](http://www.virtualmarionette.grifu.com)

## Other features
Pull The Strings as other operators that are not yet avaiable such as:
Timelines
LeapMotion support
Wiimote support
Math operations
Recording OSC performance

## Developed with
Pull The Strings is based on ofxDuct and makes use of many other ofxAddons such as:
ofxRemoteUI
ofxLeapMotion
Wiiuse
ofxBonjour
ofxAppGLFWindowMulti
ofxHistoryPlot
ofxUI
ofxTimecode
ofxMSATimer
ofxRange
ofxTimeline
ofxOsc
ofxGui
ofxMSAInteractiveObject

Remote Control for Unity (RCu) is a open source generic mapping interface for controlling and exposing properties and methods of object””s in Unity using remote applications and devices.

Remote Control Framework Scheme

Remote Control Framework Scheme

Remote Control for Unity was published in December 2015 and is available at GitHub

This Video shows the framework of this digital puppetry environment.
[vimeo 150416605 w=580&h=326]

Remote Control is open source and provides a GUI to facilitate the mapping procedure ideal for non-expert programmers, an interface that provides an easy way for controlling Unity parameters in real-time from devices such as a smartphone.

RCu makes use of Jorge Garcia””s UnityOSC to access to the Open Sound Control (OSC) functionalities and provides connectivity with all devices and applications that support this protocol.

Videos:
Pre-Release Video
[vimeo 135032229 w=580&h=326]
This video shows the pre-released version from June 2015.

Tutorial 1 [Simple]: Controlling the object´s position
[vimeo 150416957 w=580&h=326]
In this tutorial you will learn how to connect, map, and control objects position from remote.

Tutorial 2 [Intermediate]: Controlling Blend-Shapes
[vimeo 150417112 w=580&h=326]
In this tutorial you will learn how to control Blend-shapes of your character with a flexible interface.

Tutorial 3 [Complex]: Control Lights, Cameras and Activator

In this tutorial you will learn how to change light parameters, how to move cameras and how to activate them remotely to create an interactive performative environment.
[vimeo 150417214 w=580&h=326]

Published in Dec.2015
GitHub: https://github.com/grifu/RemoteControlUnity

Resurrection performance at Casa da Música (casadamusica.com) in the 7th of December of 2015.
A major production from ESMAE (esmae-ipp.pt) for the 30th anniversary of IPP (portal.ipp.pt) embracing all the school departments (Theatre, Music, Audiovisual, Multimedia).

Resurrection at Casa da Música - 7.December.2015

Resurrection at Casa da Música – 7.December.2015

Inspired by the 2th Symphony of Gustav Mahler, Lee Beagley wrote and directed “Resurrection – Life lines” based on 65 interviews that he made during one year in Portugal with people who needed a second change in their life””s.

This video shows interventions that were performed in several places of Casa da Música and were adapted from the theatrical project presented at Teatro Helena Sá e Costa in November 2015.

Because my intervention was mainly on the multimedia field, I give you a brief insights of the project setup.
[vimeo 150416124 w=580&h=326]
[HARDWARE]
– Lobby and main Foyer (two 10k Lumen video projectors for the walls)
– Sala Renascença (one 7k Lumen projecting onto a translucent screen on a window to Sala Suggia)
– Exterior Wall (one video projector for the south wall)
– Two Macintosh””s
– Two Microsoft Kinect””s

[SOFTWARE]
– Qlab (Video mapping, video routing, video effects)
– OpenFrameworks (Computer Vision + Image Processing)
– Syphon (Video routing)
– Open Sound Control (for remote control)

A project with the students of ESMAE.

CREDITS
Texto e Encenacão: Lee Beagley
Traducão: José Topa

Ass. Encenacão e Produção: Xana Miranda
Ass. Movimento: Vítor Gomes
Ass. Voz: Bernardo Soares, Gabriela Amaro

Direcção de Cena e Produção: Gonçalo Gregório, Mariana Silva
Cenografia: Inês Mota, Luís Mesquita, Miguel Costa, Vera Matias
Figurinos: Manuel de Faria, Mónica Melo, Samanta Duarte
Luz e Som: Alexandre Candeias, Sérgio Vilela

Interpretação: Aldair Pereira, Carlos Alves, Cláudia Gomes, Gabriela Brás, Gabriela Costa, Guilherme de Sousa, Hugo Olim, Inês de Oliveira, João Lourenço, Mafalda Canhola, Maria Inês Peixoto, Mariana Coelho, Mariana Santos Silva, Marta Dias, Marta Rosas, Nuno Granja, Raquel Cunha, Rita Fernandes, Sara Xavier, Teresa Zabolitzki, Xavier Miguel, Miguel Marinho (participação especial)

Músicos: Diego Alonso, Gorka Oya Diez, Bernardo Soares, Ricardo Casaleiro, Gabriela Amaro
Fotografia: Paula da Fonte, Joana Machado, Rui Sá
Multimédia: Grifu, João Gigante, Ricardo Couto
Docentes DAI: Luís Leite (Grifu), Marco Conceição

/// This project hopefully opens new dialogues and breaks down barriers ///

Session #04 | 4 April 2015 @ 17h00 > Text-Space [Installation, Performance, Spacial poetry].

Presentation by Rui Torres. Interventions by Nuno M Cardoso with Luís Grifu and Luís Aly.

Live Arquive is Anarchyve! Presentation and remix of the Digital Arquive of Portuguese Experimental Literature (www.po-ex.net)  by Retroescavadora – Artistic Intervention Collective: Ana Carvalho, Filipe Valpereiro, Luís Aly, Luís Grifu, Nuno Ferreira, Nuno M Cardoso, Rui Torres.

At Gato Vadio bookstore
Arquivo vivo é anarquivo

On the 2th of March we will give a lecture on the puppet play “Prometeu” at the school Clara de Resende in Porto.

prometeu

Picture by Pedro Martins

 

Almost all the production team will be present at the lecture:

Puppeteer: Marcelo Lafontana

dramaturgy: José Coutinhas

Photography: Pedro Martins

Multimedia: Luís Grifu

Scenography: Sílvia Fagundes

 

avanca

In 24th of July of 2014 I will be presenting the paper Prometeu – Ilusão em tempo real at the Avanca Festival in Portugal.

Leite, L., Redondo, M. (2014) Prometeu, Ilusão em tempo-real. Conferência Internacional de Cinema, Arte e Tecnologia. Avanca.

Prometeu is a multimedia theater play inspired by the Indonesian shadow puppetry, Wayang Kulit. This multidisciplinary project narrows the boundaries between theater and film, past and present, reality and illusion. Merging different art forms brings new experiences to the artists and to the public; in particular the amalgamation of puppetry with film and other disciplines contributes to a new concept, what we can call the performance cinema or ” Live Cinema”.

Using multimedia technologies and cinematic language this project motivates a different paradigm from the traditional reading of the narrative sequence. It promotes the discussion about the boundaries of theater and film, and the role of actors, puppeteers, animators and directors in this particular art form.

Preserving the heritage of traditional shadow theater, Prometeu overcomes the typical physical constraints found in traditional puppet theatre through film and multimedia techniques. Real-time post-production, performance animation or interaction techniques provide a new mean of creation to artists.  By using sensors the artists can expand the interaction. By using live cameras the performer becomes the director creating the sequence in real-time. Techniques such as “luma-key”, procedural animation or gesture tracking enhance the live experience making the performer into a sort of magician.

In Prometeu one puppeteer, performs with virtual and real objects and controls all the show remotely.

In this paper we present the methodology and the implementation of our technology as well as the aesthetics and conceptual aspects. 

 

FestivalESMAE14

Been invited to give a MasterClass  April 7th, at Festival ESMAE 2014.

ESMAE-IPP – Escola Superior de Música, Artes e Espectáculo do Instituto Politécnico do Porto | 4 – 16 of April 2014

In this MasterClass, i have presented my research on performance animation with digital puppetry methods and an overview on HCI. I also made some live demonstrations using novel digital interfaces to control animation, sound and music.

full program (PDF 700k) FestivalEsmae2014a

Exhibition of Virtual Marionette interactive installations at Teatro Municipal de Vila do Conde from 14th December till 31th December 2013.

I was invited to participate with my virtual marionette work as part of my contribution to Lafontana Formas Animadas (LFA) puppetry company. This exhibition gathers the work of 15 years of puppetry from LFA. My research and work contributes to new approaches of puppetry.

I invite the participants to interact with Inversus and bePuppit installations and to send me some feedback to virtual.marionette@grifu.com

 

ViladoConde

vila6

more pictures, click the more button.

(more…)

The Virtual Marionette installation “BePuppit” in a puppetry exhibition at Teatro Municipal de Vila do Conde.
We invite the audience to participate in this play which recreates a traditional puppet booth.

“PUPPIT” is an interactive experience that explores different ways to control and animate virtual puppets in live environments using digital puppetry methods as a storytelling tool.
With “Body-PUPPIT” the users body is the puppet controller, producing performance animation with body motion. We challenge the participants to explore their bodies to give life to puppets that are different from human morphology as if they were using hand shadows to create imaginative figures.
By playing with puppets in a dramatic way the user can create the illusion of life and build a narrative.

PUPPIT consists in a several prototypes that are the work-in-progress of a PhD research called “Virtual Marionette – Interaction Model for Digital Puppetry”. The goal of the prototypes is to explore and evaluate different rigging, mapping and interaction methods, and to develop tools for artists and non-expert artists that can be used in as collaborative environment for storytelling – making live narratives.

The exhibition is open every day since the 14th till 22th December except mondays.

www.virtualmarionette.grifu.com
December 2013

logo

Presenting the digital shadow puppetry “Shape Your Body ” at the Computer Human Interaction conference CHI 2012 at Austin, Texas.

chi-austin-18

In this picture it is possible to identify the Shape Your Body Poster

Made many interesting contacts and received important feedback´s about my work.

I´ve made a presentation to PIC (Porto Interactive Center)  to share with my colleagues this great experience.

Feel free to download the PDF file: chi-austin 2012