The Dance of Communication by Bob Cotton:
A New Sensorama - part one
A New Sensorama - part one
Unfinished notes about the ongoing work in progress on an ideal human-computer-media interface...
Morton Heilig’s 1957 Sensorama - used movies, movement, smells, wind, sound and haptic control to simulate a motorcycle-ride through Manhattan... courtesy Morton Heilig.com
Over 50 tears ago, the Future Cinema philosopher, cinema technician and inventor Morton Heilig manufactured an interactive, multi-sensory personal cinema experience he called Sensorama.
Inspired by the 146 degree immersive wide-angle of Cinerama (1950), Heilig’s 1955 paper 'The Cinema of the Future' began to outline his proposals for ‘experience cinema’ - ways of incorporating interactivity, and multi-sensory immersive experience into the cinema/viewing experience. Two years later he launched Sensorama. Sensorama was an electro-mechaniucal-optical machine based on a back-projection film screen, and Heilig made several special film experiences for it, including one described at some length by Howard Rheingold in his 1991 book Virtual Reality, which simulated a motorcycle ride through Manhattan.
Even earlier than Sensorama, and a decade before Ivan Sutherlands Head-Mounted Display, Heilig developed a head-mounted 3d stereo-movie display he called the Telesphere Mask.
Morton Heilig: Telesphere Mask c1955 courtesy Morton Heilig.com
A New Sensorama:
Heilig is an exemplar of the artist-theoretician-inventor-film-maker that I think punctuates the history and development of Film. Heilig is interesting for many European artists because he worked outside the academic research world and was intent on developing a commercial entertainment product (much like Ted Nelson’s projection of his Xanadu networked hypertext idea as a franchisable service - a McDonalds of Cyberspace...). Sensorama was a coin-op, arcade-’game’, and remains a little-known prototype of a ‘cinema of the future’.
http://www.iwaswondering.org/cynthia_video.html
I was thinking about Sensorama and other experiments that envisaged the future of film, when I heard a BBC World Service programme interviewing Cynthia Breazeal - the MIT Affective Robotics pioneer and inventor of the Kismet emotive Robot (from 1999). She was talking about the kind of human-robot interface she was developing that attempted to simulate the kind of dance of communication that two humans can have when they use words, intonation, sight, sound, gesture, expression, body language, movement, shared memories, metaphor, allusion, reference, eyeball contact, gaze-tracking and empathy to communicate together.
This is the space of conversation. The best conversations, according to Cynthia Breazeal, are a dance of communication, a two-way engagement of voice, eyeball contact, gaze tracking, gesture, expression, memory, taste, interests, movement, attention, empathy, and emotions (etc). The conversation ‘dances’ between these channels, within these shared spaces of memory, opinion, tastes, passions and style. We can all reflect on those wonderful ‘marriage of minds’ - those special conversations that often last for hours and hours, constantly exploring and remembering, rejoicing and reflecting, expression and loquacity, in a sustained flow of engagement, a dance of communication.
OK so I got to thinking how, like Heilig, we might simulate this dance of conversation using the latest developments in biometric and position sensors, and other image-recognition and position-sensing techniques, and of course on the state-of-the-art of the toughest problem of all - creating an integrative AI ‘brain’ that could read, recognise, integrate, model and respond empathically to such input. In other words: how do we create a media-machine that can talk in sound and film clips, literary quotes, clips of recorded interviews, speeches, dialogue? That could illustrate a point in realtime with examples of photographs, music, drawings, paintings, engravings? That would simulate the experience of the best conversations, that could use the multi-sensory apparatus implied in dance of communication to link us seamlessly to the ‘omnimedia cloud’ that is our developing online archive of all media - all books, all films, all sound-recordings, all games, all facts, all stats, all data, all art, all ethnographical, scientific and cultural museum and archive artefacts.
This would amount to an ideal of human-computer interface design, making the media-machine an ideal teacher, companion, guide, mentor, baby-sitter, carer, counsellor and buddy.
So this article is a rather cursory sketch of the state-of-play in this fascinating research sector....
Will we ever have this kind of interlocutive dance with our media? Our books, films, records and pictures? What do we need to enjoy a dance of communication with the filmic media for example?
This article proposes some investigations that will be necessary to achieve really smart media - the kind of ‘intelligent’, knowledgable media that we can have a proper, life-enhancing conversation with.
"and what is the use of a book," thought Alice, "without pictures or conversations?' (Carroll: Alice’s Adventures in Wonderland)
more to come...
