Interaction: Real-Time Media Systems


Interactivity has been an essential and integral component of Troika Ranch’s work since the company’s inception. Interactivity is the energetic linkage of human action to digital content and the feedback loop that is formed by this relationship – performer responds to system, system reacts to performer, performer makes informed choice to respond to system, etc. This linkage is achieved by using cutting-edge computer technology in tandem with custom sensory technology that tracks the movement and vocalization of performers on stage or viewers in an installation. By using these systems a bending arm can warp a video image or the kick of a leg can recall a musical phrase. This interaction allows a performer or viewer to follow their instincts from moment to moment, making improvisational choices that subtly or profoundly change the visual and aural content of a work. Coniglio and Stoppiello want all of the digital media in their work to have the same sense of dynamism, vitality and “liveness” as the performers themselves.


Tools like video, digital media software packages and tele-presence are important to us because art, to be relevant in contemporary society, must simultaneously embrace and examine contemporary culture and currency. Integrating emergent media keeps live theater vital in a world where the most widely experienced channel of aesthetic expression is broadcast media. But, most importantly, it imparts a real and necessary density that mirrors the media intense world in which we live.


Isadora® is the software “brain” behind Troika Ranch performances and is the main technology used to generate the imagery and sound for Troika Ranch’s current projects. Created by Co-Founder Mark Coniglio, Isadora is a graphic programming environment that provides human control over digital media, with special emphasis on real-time manipulation of digital video. Isadora gathers movement information from various sensory devices and uses that information to control and manipulate digital video, music synthesizers, sound modulation devices, theatrical lighting and robotic set pieces. (Learn more at TroikaTronix website.)


A typical setup for a performance or installation includes one or more movement sensing devices that communicate with an offstage computer. A sensing device is a piece of equipment programmed to measure the movements or vocalizations of performers. One sensing device Troika Ranch developed and implemented is the MidiDancer; a bodysuit outfitted with plastic fibers that measure the flexion and extension of the major joints on the body. When a joint moves, it sends a signal to the computer. This signal is then transformed into code that manipulates other theatrical equipment – video cameras or digital sound devices. Video and sound are thereby manipulated directly in relation to a specific movement. For example, when a performer’s arm is straight, the sound is very soft; as the performer bends her arm, the sound amplifies. This system was first used while Coniglio and Stoppiello were students at CalArts in 1989. The two went on to create several works that feature the MidiDancer including their seminal work In Plane (1994), a duet for a dancer and her video representation. Using the MidiDancer, the performer controls the generation of music, the recall of video images, the manipulation of theatrical lighting and the movements of a robotically controlled video projector. This work has been analyzed and written about in numerous academic papers and journals.


Since the late 1980's Troika Ranch has continued to advance and improve their use of sensory devices. One recent sensory device seen in 16 [R]evolutions (2006) uses a single video camera placed on stage. This video camera is connected to a computer that runs specially designed software EyesWeb that imposes a 12-point virtual skeleton onto the body as seen by the camera. The points of the skeleton are then linked to Isadora, which generates imagery in real-time response to the movement of one or all of those points. Troika Ranch has also used Infrared light as a way to be able to track bodies and project images into the same space. By flooding the Òback wallÓ with infrared light, an infrared camera can ÒseeÓ a black silhouette of a performer in front of a white wall and ÒignoreÓ any video projected onto that wall.


Troika Ranch's current investigations are on how we can use the computer to disturb the choreographic process itself. For our recent work loopdiver (2009), we began by creating a special tool in Mark Coniglio's Isadora® software that allowed us to compose highly complex looping structures and impose them on any digitally recorded material, including (but not limited to) digital video files and audio files. We created a five-minute long "base material" consisting of real-time movement, music, video and theatrical lighting, then imposed the looping patterns on a videotape of the base material. Because of the repetition, the five minutes was transformed into 40 minutes. The resulting video became the choreographic "score" that our performers memorized and realized in their bodies -- a task whose difficulty cannot be underestimated. The music, video projections and theatrical lighting maintain the absolute precision and perfection of the computer loops, while the choreography performed by the dancers is necessarily imperfect due to human interpretation. When placed together on stage, we see the performers in a constant struggle to adapt to a relentless, externally imposed machine rhythm.