Interactive visualization and sonification for monitoring complex processes

Abstract

This paper introduces AVDisplay, a versatile auditory and visual display for monitoring, querying and accessing information about modules or processes in complex systems. In the context of a collaborative research effort (SFB360, artificial communicators) at Bielefeld University, a cognitive robotics system for humanmachine interaction is being developed. The AVDisplay provides the central interface for monitoring and debugging this system, currently involving about 20 computers hosting more than 30 complex processes. The display is designed to provide a summary over the system's activities combining visualization and sonification techniques. The dynamic visualization allows inference of correlated activity of processes. A habituation simulation process automatically sets a perceptional focus on interesting and relevant process activities. The sonification part is designed to integrate emotional aspects – if the system suffers from poor sensory quality, the sound conveys this by sounding uncomfortable.