The research is being undertaken initially within a three-year (2008-2011) joint European project with partners at Université Paris VI Pierre et Marie Curie (France), University of Verona (Italy), INRIA-IRISA (France), and Aalborg University (Denmark), and local collaborators including B. Giordano and S. McAdams of the Music Technology area at McGill University. Funding is provided by a generous grant from the Ministère du Développmeent, économique, de l'Innovation et de l'Exportation (Québec) in coordination with European Commission Seventh Framework FET-OPEN project NIW: Natural Interactive Walking (no. 222107). Additional support was provided by a CIRMMT student research project award from 2006-2008 (Y. Visell, supervised by J. Cooperstock, M. Wanderley).

The Natural Interactive Walking project investigates new technologies for multimodal interaction with virtual ground surfaces. Specifically, this involves research into the integrated and interchangeable use of the haptic and auditory modality in floor interfaces, and for the synergy of perception and action in capturing and guiding human walking. The project is developing efficient techniques for closed loop interaction via walking in real, augmented environments. Validation is carried out through experiments dealing with fundamental aspects of human multisensory perception of materials via contact interactions in walking, and usability of the specific virtual interaction methods developed. The aim is to enable the transfer of skills that have been learned previously in everyday tasks associated with walking to new interactive settings, where multi-sensory feedback and sensory substitution can be exploited to create unitary multimodal percepts of virtual ground surfaces.

Our more recent work has investigated device compensation for lower limb effects, multimodal rendering of foot-water contact effects, use of the floor surface as a multitouch input device for the feet, and possible strategies for integrating dynamic friction.

Video clips

A multimodal floor for virtual environments	The Beach
Bubbles	Control Surface

Recent Publications

• Vibrotactile Rendering of Splashing Fluids. Cirio, G., Marchal, M., Lécuyer, A., and Cooperstock, J. R. Transactions on Haptics, 6(1):117-122. 2013.
• Modeling of Lower Limbs for Vibrotactile Compensation. Otis, M., Millet, G, Beniak, S, and Cooperstock, J. R. (2011). Canadian Medical and Biological Engineering Conference, Festival of International Conferences on Caregiving, Disability, Aging and Technology.
• Initial Development of a Variable-Friction Floor Surface. Millet, G., Otis, M., Chaw, G., and Cooperstock, J. R. (2011). In Canadian Medical and Biological Engineering Conference, Festival of International Conferences on Caregiving, Disability, Aging and Technology.
• Vibration Influences Haptic Perception of Surface Compliance During Walking. Y. Visell, B. Giordano, G. Millet, J. Cooperstock (2011), PLoS ONE 6(3: e17697).
• Audio-Haptic Identification of Ground Materials during Walking. B. Giordano, Y. Visell, H.-Y. Yao, V. Hayward, J. Cooperstock, and S. McAdams, Journal of the Acoustical Society of America (in revision).
• Interaction Capture in Immersive Virtual Environments via an Intelligent Floor Surface. Visell, Y., Law, A., Ip, J., Smith, S., and Cooperstock, J. R. (2010). In IEEE Virtual Reality (VR), 313-314, Waltham, MA, USA, March.
• Contact Sensing and Interaction Techniques for a Distributed, Multimodal Floor Display. Visell, Y., Smith, S., Law, A., Rajalingham, R., and Cooperstock, J. R. (2010). In IEEE 3D User Interfaces (3DUI), 75-78, Waltham, MA, USA, March.
• Probabilistic Tracking of Pedestrian Movements via In-Floor Force Sensing. Rajalingham, R., Visell, Y., and Cooperstock, J. R. (2010). In Seventh Canadian Conference on Computer and Robot Vision (CRV), 143-150, Ottawa, ON, Canada.
• Design of a Vibrotactile Display via a Rigid Surface. Visell, Y., and Cooperstock, J. R. (2010). In IEEE Haptics Symposium, 133-140, Waltham, MA, USA, March. (Best Paper Award)
• Multimodal floor for immersive environments. Law, A., Ip, J., Peck, B., Visell, Y., Kry, P., and Cooperstock, J. R. (2009). In ACM SIGGRAPH Emerging Technologies, ACM, New Orleans, LA, USA, August.
• Touch Is Everywhere: Floor Surfaces as Ambient Haptic Interfaces. Visell, Y., Law, A., and Cooperstock, J. R. (2009). IEEE Transactions on Haptics, 2(3):148-159, July.