Sonimime: Sonification of fine motor skills

Abstract

This paper describes the design of SoniMime, a system for the soni cation of hand motion. Among SoniMime's applications is the use of auditory feedback to re ne ne motor skills in a wide variety of tasks. Our primary soni cation method involves mapping movement to timbre parameters. Speci cally, we explore the application of the tristimulus timbre model for the soni cation of gestural data, working toward the goals of assisting a user to learn a particular motion or gesture with minimal deviation. We also explore real-time timbre shaping and its possible use as an intuitive soni cation tool related to formant synthesis and human vowel recognition. SoniMime uses two 3-D accelerometers connected to an Atmel microprocessor which outputs OSC control messages. Data ltering, parameter mapping, and sound synthesis take place in Pd running on a Linux computer.