Photone: Exploring modal synergy in photographic images and music
Abstract
We present Photone, an interactive installation combining photographic
images and musical sonification. An image is displayed,
and a dynamic musical score is generated based on the overall
color properties of the image and the color value of the pixel under
the cursor. Hence, the music changes as the user moves the cursor.
This simple approach turns out to have interesting experiential
qualities in use. The composition of images and music invites the
user to explore the combination of hues and textures, and musical
sounds. We characterize the resulting experience in Photone as
one of modal synergy where visual and auditory output combine
holistically with the chosen interaction technique. This tentative
finding is potentially relevant to further research in auditory displays
and multimodal interaction.