Soundview: Sensing color images by kinesthetic audio

Abstract

An experimental system called SoundView has been developed, which allows the exploration of a color image through touch and hearing. The image is mapped onto a virtual surface with a fine-grained color dependent roughness texture. The user explores the image by moving a pointer device over the image. The pointer acts like a virtual gramophone needle, and the sound produced depends on the motion as well as on the color of the area explored. The roughness texture is obtained by constructing a mapping of three dimensional color space onto a three dimensional sound space. The mapping tries to achieve maximal alignment of the color and sound spaces by preserving the perceptual metrical and topological structure of color space, as well as by incorporating common associations between sound and color.