Sonifyd: A graphical approach for sound synthesis and synesthetic visual expression
This paper describes Sonifyd, a sonification driven multimedia and audiovisual environment based on color-sound conversion for real-time manipulation. Sonifyd scans graphics horizontally or vertically from a scan line, generates sound and determines timbre according to its own additive synthesis based color-to-sound mapping. Color and sound relationships are fixed as default, but they can be organic for more tonal flexibility. Within this ecosystem, flexible timbre changes will be discovered by Sonifyd. The scan line is invisible, but Sonifyd provides another display that represents the scanning process in the form of dynamic imagery representation. The primary goal of this project is to be a functioning tool for a new kind of visual music, graphic sonification research and to further provide a synesthetic metaphor for audiences/users in the context of an art installation and audiovisual performance. The later section is a discussion about limitations that I have encountered: using an additive synthesis and frequency modulation technique with the line scanning method. In addition, it discusses potential possibilities for the future direction of development in relation to graphic expression and sound design context.