3.1.1. Description of the ‘M.U.S.H.’ project
M.U.S.H.’ [217] (Multi-User Sensory Hallucination), by Joachim Montessuis and Eleonore Hellio (FR), was developed and produced in collaboration with V2_Lab. The text below describes the artists’ backgrounds, the conceptual and technical background, and the numbers after their names refer to the aRt&D Triangle below. The project description is written in close collaboration with the artists and the involved V2_Lab team members.
(Collage 1. Impressions of M.U.S.H. by Joachim Montessuis and Eleonore Hellio; visuals by the artists and Stock/V2 Lab)
(Collage 2. aRt&D map M.U.S.H.)
‘M.U.S.H.’ is a dual site telematic installation with one room in each location. Each visitor enters a closed room and faces a black image on a video screen. The person is provided with a wireless ‘M.U.S.H.’ device, equipped with accelerometer sensors, when they enter. The room is quiet but as soon as the person starts to move around the ‘M.U.S.H.’ device, their movements control the sounds and visuals. This exploration of the space causes a sound composition and works simultaneously as a synchronisation tool for telecommunication between the two sites. The system captures each person's movements to generate a sound and visual ‘partition’ and ‘orchestration’ in real time that becomes the ‘carrier wave’ for telecommunicated experience. ‘M.U.S.H.’ is a digital collision space where a chance is given to share an experience together (or alone). This will first depend on one's ability to fully explore and interact in its multi-sensory environment. The synchronicity of behavioural patterns is what may trigger the appropriate feed-back for a tangible link to occur. If both participants ‘excite’ the ‘M.U.S.H.’ room at the same time, in the same manner, they will increase their chances of meeting each other through virtual space. The sound acts both as a subversive and immersive element. The image is considered as both place and language. A visual dialogue that doesn't follow the usual rules of videoconferencing will define the feeling of tele-presence or tele-absence.
The sound environment consists of different layers of sinusoids, infra-basses, drones and granulated samples that react to the movements of the device in terms of intensity, frequency and loudness. ‘M.U.S.H.’ offers the operators a complete stage for jamming. As one conquers the system, the flickering image may become more precise and give a chance to the other operator to appear on the screen.
The main conceptual research objective in this project dealt with various aspects of the real-time telematic user experiences. The artistic research is informed by theory and concepts for real-time performance, as has been topic of research in the area of music instruments in Ircam in Paris (FR) and Steim in Amsterdam (NL). Michel Waisvisz’s [218] research into gesture controllers for electronic music is most closely related to the interface concept of ‘M.U.S.H.’ Furthermore, ‘M.U.S.H.’ has been informed by concepts of space–time discontinuity in cybernetics, a re-occurring artistic research topic, which lead to networked experiments by numerous artists over the years, among them earlier telematic art pioneers like Roy Ascott with his slowscan telematic videowork ‘Organe et Fonction d’Alice au Pays des Merveilles’ [219] (1985) and in 1991, Paul Sermon with his Telematic Dreaming [220] project where voyeurism and a telepresent body and reality were mirrored over each other via ISDN telephone lines and a video conferencing system. The artists themselves refer to more recent French art theory on time-space discontinuity in telematics:
‘...Le temps réel ne résulte pas seulement de l’augmentation de la vitesse de circulation des informations. Il correspond à la façon dont des réalités autonomes developpent en leur sein et dans les échanges qu’elles entretiennent avec leur environnement, des processus d’adaptibilité, de transformation, d’exploration dans la recherche de situations d’équilibre toujours suceptibles d’être modifiées...Il ouvre sur des temporalités multiples... Il consiste au contraire dans l’ouverture, le tâtonnement, le surgissement du nouveau...’ Jean Cristofol, Fabrice Gallis, Guillaume Stagnaro. [221]
The Beta version of ‘M.U.S.H.’ was presented during DEAF04. In an interview about the work, Eléonore Hellio explains the project from the conceptual, user-oriented angle. In the same interview, her collaborator formulated the question or problem in relation to the technical development of the interface device:
‘We wanted an interactive wireless device, that was the main goal of this project, to have a tool completely as a wireless system. We worked with Stock, the engineer who created this device. He worked for 6 months to create this and I worked for six months on the most important part of the sound patch. We just worked with this device and all this interactivity problems.’ (J. Montessuis) [222]
The artwork becomes alive through the participant’s movements with the device. The first version of ‘M.U.S.H.’ demonstrated the potential of the device’s interactive vocabulary. The next version of the wireless device is developed by V2_ in the context of a major Dutch National research program; MultimediaN.[223] The directors of the program received the demonstrator of the recent version very positively. In particular the ‘M.U.S.H.’ device’s features for training gesture and movement interpretation via a neural network, based on the indicators from the artists, were seen as a valuable user-oriented technology innovation. V2_Lab continues to adapt this interface-system as a generic device for various other applications and artworks.
The technologies used are high bandwidth videoconferencing facilitated by Polycom Ipower PT 680/685 Codec’s, MAX/MSP, Jitter and the Very Nervous System (VNS). Joachim Montessuis programmed the Max/MSP patches and Jitter in collaboration with V2_Lab. Stock, V2_Lab’s hard- and software engineer, developed the wireless ‘M.U.S.H.’ handheld device that is based on two accelerometer sensors to measure acceleration of movement in 3 dimensions. The prototype device in the first version includes a basic-stamp and uses the Bluetooth communication protocol to transmit the measurements to a computer. This was developed and used as technology in other projects in the past, and although it was known to have considerable shortcomings, it provides a working prototype for testing with a live audience. In Max/MSP, the device's measurements are decoded into a set of vectors representing the device's perception of the Earth's gravity, and its relative movements. This M.U.S.H. device is tested and improved in prototype version 3. This interface employs a neural network as the core gesture-recognition element. This version of the device is based on the Logitech Cordless Rumblepad2 game-controller. For more details and evaluation see ‘The M U S H environment Prototype Version 3’ by Stock Plum and Michel van Dartel, V2_Lab.[224]
