A 3D-auditory environment for hierarchical navigation in non-visual interaction

Abstract

This report describes the development of a generic, reusable spatial auditory environment, which combines a spatial auditory display derived by processing sound sources with head-related transfer functions, with gesture and speech-based input techniques. The spatial auditory environment is structured in a "ring topology", where the user is surrounded by a virtual "ring" containing auditory interaction objects. Auditory objects can be reviewed and selected by using 3D-pointing, hand gestures or speech recognition input, providing a multi-modal direct-manipulation method for the exploration of auditory interaction objects. Interaction processes are supported by different classes of spatial non-speech and speech audio cues that provide an immediate auditory feedback on different user activities. This auditory environment is used to implement a hierarchical navigation dialogue in a multimedia non-visual interaction toolkit that is currently under development.