Ultra-Videoconferencing Overview
The Ultra-Videoconferencing project at McGill University has developed a flexible, low-latency IP transport system for audio, video, and vibrosensory data. We have used this system for a range of demanding applications including live concert streaming (1999), remote mixing (2000), collaborative performance (2001), distance masters classes over SDI (2002), and remote video interpreting of sign language using three simultaneous DV streams (2003).
Our software supports high-resolution multichannel audio and video from either frame grabbers using v4l or v4l2 interfaces (e.g., Bt878 chipset), digital video (DV) cameras, DCAM (raw, uncompressed video) 1394 devices, and standard or high-definition Serial Digital Interface (SDI). We are also working on mosaicing of video displays with minimal latency and moving toward super-resolution techniques to create video displays beyond the quality of high-definition.
Development began with a project to stream Dolby AC-3 audio (5.1 surround sound) over the Internet from Montreal to New York, as part of the 1999 Audio Engineering Society Convention. The following year, the software was rewritten to support the streaming of 12 channels of 24bit/96kHz PCM audio between Montreal and Los Angeles.
Since then, the software was expanded to include analog, digital, and SDI video, and further updates reduced the transport latency to levels enabling effective interaction at a distance, as used for a real-time violin duet, a cross-continental jazz jam, remote sign-language interpreting, and distance Masters classes with Maestro Pinchas Zuckerman.
The development of this software was supported largely by research grants and contracts from Canarie Inc. and Valorisation Recherche Quebec.
Related Links
Interactive Cross-Continental Jazz
On October 24, 2002, musicians from Stanford University and McGill joined in a highly successful evening of cross-continental jazz, playing together in surround sound and full-screen video, over the Internet, using our Ultra-Videoconferencing system in analog video mode.
Despite the performers being at opposite sides of the continent, the system maintained a delay (during the good moments — as problems with an intervening router caused periodic delays that were almost double) below 50 milliseconds, which feels to the musicians almost like being on the same stage.
The actual first attempt at this experience took place on June 13, 2002. Running the entire McGill end from a single machine, the system latency did introduce a perceptible audio echo, which was removed at the end of the evening by transmitting audio-only. Nevertheless, the event was a major success in demonstrating the feasibility of Internet support for highly interactive events across the continent.
Remote Masters Classes with Pinchas Zuckerman
Distributed Violin Duet
On November 9, 2001, a violin duet was performed over a wide area, high speed computer network in full-screen video as part of the RISQ 2001 Conference. To our knowledge, this marked the world’s first demonstration of such a high quality and low-latency system, using internet protocol communications.
Although separated by several kilometres, the two performers were able to hear each other in “near-real-time,” (approximately 20 ms delay) allowing them to synchronize their playing as if standing together on the same stage.
A one-way video transmission accompanied the audio — problems with the second video card prevented this from being a bi-directional video demonstration.