Environmental Sounds as Concept Carriers for Communication

View/ Open
Date
2010-06Author
Cook, Perry R.
Fellbaum, Christiane
Ma, Xiaojuan
Metadata
Show full item recordAbstract
Sonification, the use of nonspeech audio to represent data and
information, has been applied to industrial systems and computer
interfaces via mechanisms such as auditory icons and earcons. In
this paper, we explore a different application of sonification,
which is to facilitate communication across language barriers by
conveying commonly used concepts via environmental auditory
representations. SoundNet, a linguistic database enhanced with
natural nonspeech audio, is constructed for this purpose. The
concept-sound associations which are building blocks of
SoundNet were validated through a sound labeling study
conducted on Amazon Mechanical Turk. We determine the factors
that cause a sound to evoke a concept. We examine which aspects
of the proposed auditory representations are evocative, and what
kinds of confusions may occur. Our results show that sounds can
effectively illustrate some concepts, especially those related to
concrete entities and actions, and thus can be utilized in assistive
communication applications.