Using Spatialized Sound to Enhance Self-Motion Perception in Virtual Environments and Beyond: Auditory and Multi-Modal Contributions

Abstract

Embodied self-motion illusions (“vection”) have long fascinated both researchers and laypeople. With the increasing quality and affordability of immersive virtual reality and tele-operation/tele-robotics interfaces, there is also increasing interest in providing compelling sensations of self-motions to create more life-like and convincing experiences. Whereas most research on self-motion perception focuses on visual and vestibular contributions, auditory can also play a relevant role. Here, we will provide an overview on research indicating how spatialized sound (moving sound fields) can both induce self-motion illusions in blindfolded listeners and enhance self-motion illusions induced by other modalities. Auditory vection by itself can be enhanced by a number of factors, including increasing the number of moving sound sources and employing sound sources that are more likely to be interpreted as originating from stationary objects (“acoustic landmarks” such as church bells) than artificial sounds or sounds associated with moving objects such as footstep sounds or car sounds. Although auditory cues alone provide a much less compelling self-motion sensation than visual cues or biomechanical cues (e.g., from walking on a circular treadmill), they can significantly enhance vection induced by other modalities as well as enhance presence and immersion in virtual environments. These findings will be discussed both in the context of multi-modal cue integration and self-motion simulation applications such as Virtual Reality, where high-quality spatialized sound could often be included at relatively low cost.