The World Opera Project is a distributed, real-time live opera performance planned to take place simultaneously in several Canadian, U.S. and European cities. At the Shared Reality Lab, we had the opportunity to run a series of experiments aiming to look into "how audio and video transmission delays affect the emotional connection between singers", "the choice of strategies singers use to compensate for these delays" and "the effect of orchestra placement on the resulting performance".
The set of experiments was a collective effort or our lab, in collaboration with Niels Windfeld Lund of the University of Tromsø, the founder of the project. Our results demonstrated that while the delay conditions often led to undesired changes in tempo, these might be overcome, in part, through rehearsal. Moreover, the role of the conductor was found to be important in maintaining the flow of the musical piece and preventing the singers or the pianist from following each other, which would have resulted in the progressive tempo reduction, typical of telematic performance conditions. Nevertheless, we observed that the difficulty in judging how to compensate for the audio and video delay precluded any adjustment by the conductor for musical expression. Both the conductor and pianist soon recognized that their best latency-coping strategy was to ignore the audio feed from the singers. As the conductor stated: "I do not correct because I can't judge."
Responses from participants to the questions related to their emotional connection with the remote singer indicate that latencies in the range we were testing had little or no impact on perception of presence, even though these may have been significant to the musical result. Rather, we found that the biggest factor influencing presence was the singer's familiarity with the technology. The longer the experiments proceeded, the less the singers indicated distraction from the camera-monitor mediated experience and a greater sense of emotional connection to the other musicians, regardless of latency.
Olmos, A., Brulé, M., Bouillot, N., Benovoy, M., Blum, J., Sun, H., Lund, N.W., and Cooperstock, J.R. (2009), Exploring the role of latency and orchestra placement on the networked performance of a distributed opera, 12th Annual International Workshop on Presence, Los Angeles, Nov. 11-13, 2009.
Videos are available of dancers on the Holo screen and the plasma display. Video segmentation was particularly challenged by the lighting conditions and remains an area of active research.
Cooperstock, J. R. Multimodal Telepresence Systems: Supporting Demanding Collaborative Human Activities. IEEE Signal Processing Magazine, special issue on Immersive Communications, 28(1): 77-86. January 2011.
Download more photos or a high-definition video clip.
Can opera be performed if the opera singers are standing on different stages in different time zones in different countries?
This question is at the heart of the World Opera Project, a planned joint, real-time live opera performance to take place simultaneously in several Canadian, U.S. and European cities. The project is envisioned as a worldwide opera house located in cyberspace.
Although simultaneous music performance trials over the Internet have been run previously, opera is one of the most demanding musical forms. It not only involves music, but demands that the singers create and convey emotional tension, a significantly more difficult task when performers, conductor and orchestra may be in different cities.
The McGill experiment is designed to test many factors which make networked performances unique. What are the limits at which the singers can cope with delay, and how does the delay affect the emotional connection between characters that is vital to a successful performance? Can the conductor help the singers compensate for this delay? Does the location of the orchestra influence their ability to synchronize?
Researchers from McGill's Centre for Intelligent Machines and McGill's Centre for Interdisciplinary Research in Music Media and Technology (CIRMMT) are attempting to answer some of these questions. Vocal performers from McGill's voice program will be located in separate rooms, and will perform with internationally acclaimed conductor Niels Muus, accompanied by pianist Michael McMahon. The musicians see and hear each other through video monitors and speakers, but the delay between the rooms will be adjusted to simulate being networked across the country and half-way around the world.
These experiments are the result of a collaboration between researchers and musicians at McGill and researchers at the University of Tromsø in Norway, initiated by The International World Opera association and funded in part by The Norwegian Research Council.