The Four Senses 2002

In April 2002 a collaboration between artist/researcher Tony Brooks (EU) and artist Raewyn Turner (NZ) resulted in a series of multi-sensory performances in Auckland New Zealand at the Dorothy Winstone Theatre called The Four Senses.

 

Performed with the Aotea Youth Symphony, The Four Senses was a concert for the deaf encompassing sight, sound, smell and touch. It included an ability impaired dance group, Touch Compass, as well as a deaf signing choir and a sight-impaired singer.

The performances were an improvisation in light and smell with sound and movement by the orchestra used as stimulus. With the aid of Martin automated fixtures, Tony and Raewyn created a real- time translation of sound (and the movement in making that sound) into light, color, and multiple layers of smell.

Tony utilised sensors and other leading edge technology to capture body part movement and translate it into painting with colored light. The light collage was a play of interaction between live video feeds and sensors, and colored light pre-programmed to an interpretation of sound, one affecting the other in a dynamic visual loop.

Tony enabled the conductor to "paint" the scene through his movements, for example the velocity of a contra bass string being plucked, a timpani roll, or other such captured moment.

Martin lights, and how Tony controlled them with just an arm, foot or head movement, was center stage. Martin lighting products involved in the event included 10 MAC 500s, 10 MAC 600s, 2 MAC 2000s and a LightJockey controller. Kinderdine Electrical Ltd. of Auckland, New Zealand supplied all Martin fixtures.

Meanwhile, Raewyn translated music into drawings and subsequently into colored light and olfactor stimulation (i.e. smells). Her translations were realised through her assistant spraying aerosols on cue into the AC system. The timing had previously been calculated into when the effect would reach the audience members so as to coincide with the chosen musical experience. Tony’s role was to use Raewyn’s interactive system to create the environment visuals and effects relative to the music and the movement entailed in creating this music. Tony used 7 large projectors, 5 computers and an array of static & roving cameras to feed his program and then linked into Raewyn’s design and manipulation in real-time.

Best if Tony himself explains, “The sensor side was multiple headed Soundscapes infrared sensors which sourced movement info from the conductors hands enabling him to “paint” the sceneography. Three sensors were set up so as to enable an opening of the Red, Green and Blue filters of an image sequence that I had created that drove in black. This means that there had to be movement within the sensor field [i.e. the Virtual Interactive Space-VIS - (© Tony Brooks )] before anything could be seen. Movement within each beam opened the related color filter and thus a full spectrum of color was available to the sequence that was projected onto the stage area. Options for real-time control was to switch RGB top HIS or HSV color synthesizers. It was also possible to blend and layer in real-time additional images to the previously made image sequence. In addition a sensor was set up for a disabled dancer to manipulate sounds via subtle hand movement. This gesture mapping of movement to sounds and images is the basis for my concept and methodology Soundscapes and the European IST funded project CARE HERE.

“The two spectrums of my projected images and Raewyn´s lights integrated well and it was a huge success with a 30 minute feature TV documentary (June release) made as well as the prime time TV & radio & press that we had while there.”

Raewyn Turner has an extensive background in working with colored light and contemporary music. She is currently developing intersensory correspondences in works that combine video, color, smell and sound in multi-sensory works and live performance. She investigates the psychological effects of color on the skin and the effect of smell on the emotions, of perception, ideation, imagination, and the compensatory mechanisms in the blind and deaf persons through which they see and feel the world. She is a founding member of Bad Mothers, Melbourne and exhibits regularly in New Zealand and Australia.

Lighting Contractor - Kenderdine Electrical, Auckland N.Z.
Production and stage manager - Peter "Zeb" Mansell
Technical and programming - Glen Stewart
Programming and show operator - Chris Just
Sound contractor - Paul Jefferies, Oceania Audio