Shadow Removal from Multi-Projector Displays Modeled via Multiple View Geometry

This project involved prior contributions by M.Eng. students Daniel Sud and Nadia Hilario, and was brought to its present form by the M.Eng. research of Samuel Audet.

Overview

Before shadow removal.
Figure 1(a). Before shadow removal.
After shadow removal.
Figure 1(b). After shadow removal.

Multi-projector displays are often exploited for many installations where large display surfaces are desired. A seminal example is the CAVE (Cave Automatic Virtual Environment), where projectors display images from behind the screens traditionally with the help of mirrors, a technique known as back projection. Unfortunately, it is not practical to mount and unmount a CAVE whenever needed, and consequently room space usually needs to be reserved. This drawback limits the widespread adoption of the CAVE in more fields. An alternative to back projection is front projection, such as used in the Office of the Future. However, front projection has disadvantages as well. Particularly, when a person stands between a projector and a display surface a shadow occurs. Other shadow removal methods either cannot work with dynamic video playback at 60 frames per second or require specialized hardware, but we wanted a method that could work with commodity hardware.

Our method uses a set of cameras and projectors geometrically calibrated with respect to the floor and the flat display surfaces. For calibration, we adapted existing geometric calibration methods. The cameras are used in pairs with a stereo correspondence algorithm. The recovered depth information is reprojected in the direction of the normal of the floor. Since we assume that people stand vertically on the floor, it can cope with occlusions. Instead of depth information, our system can also use tracking information of people. One way or the other, knowing the position of people on the floor, our algorithm finds in the image of a projector the region that is occluded, as depicted in Figure 2. Another projector can then automatically fill the region with equal intensity and identical color to that of the occluded projector.

We tested our method using a stereo camera from Tyzx and four projectors on one wall. We found that the recovered calibration was accurate and that the system could correctly and efficiently perform shadow removal, providing a more appealing visual experience to users of multi-projector displays. As main limitations and material for future work, the current projector arrangement does not allow for effective shadow removal of more than one user. Also, the stereo algorithm causes undesirable flickering and is not sensitive enough to work well with relatively thin objects such as arms.

Model Figure
Figure 2.
Rendered image of the model of the room with cameras, projectors and a person, where the floor is in dark gray, the wall, in black, and its chosen display surface in white. The lines on the wall denote regions that cameras can see and that projectors can cover, drawn in their corresponding colors.

Papers

Videos



Last update: 17 July 2008