Using Auditory Display Techniques to Enhance Decision Making And Perceive Changing Environmental Data Within a 3D Virtual Game Environment

Abstract

When it comes to understanding our environment, we use all our senses. Within the study and implementation of virtual environments and systems, huge advancements in the quality of visuals and graphics have been made, but when it comes to the audio in our environment, many people have been content with very basic sound information. Video games have strived towards powerful sound design, both for player immersion and information perception. Research exists showing how we can use audio sources and waypoints to navigate environments, and how we can perceive information from audio in our surroundings. This research explores using sonification of changing environmental data and environmental objects to improve user's perception of virtual spaces and navigation within simulated environments, with case studies looking at training and for remote operation of unmanned vehicles. This would also expand into how general awareness and perception of dynamic 3D environments can be improved. Our research is done using the Unity3D game engine to create a virtual environment, within which users navigate around water currents represented both visually and through sonification of their information using Csound, a C based programming language for sound and music creation.