PC-Based Real Time Sonification of Human Motion Captured by Inertial Sensors
MetadataShow full item record
This paper presents a low latency system for real time sonification of human motion captured by inertial sensors. Exemplarily the position of the wrist estimated by two inertial sensors located at upper arm and forearm is transformed into a continuous synthetic sound. The body segment orientation is captured by Xsens MTx sensors and used to compute the position of the wrist relative to the shoulder joint. The accessible motion parameters provided by inertial sensors are three axial segment orientations, three axial accelerations and angular rates, and all derived quantities like position in three dimensional space. Motion data sonification is performed by the Sound Synthesis Toolkit (STK), a set of C++ classes for audio signal processing and sound synthesis. The proposed framework enables future research in continuous real time sonification of human motion to improve the process of motion learning during stroke rehabilitation. Through software profiling the proposed framework is benchmarked in terms of latency induced by signal transmission and processing to evaluate maximal process- able sampling rates and inertial sensor counts.