Real-time smile sonification using surface EMG signal and the evaluation of its usability
Date
2015-07Author
Nakayama, Yuki
Takano, Yuji
Matsubara, Masaki
Suzuki, Kenji
Terasawa, Hiroko
Metadata
Show full item recordAbstract
We propose a real-time, interactive system for smile recognition
and sonification using surface electromyography (EMG) signals.
When a user smiles, a sound is played. The surface EMG
signal is mapped to pitch using a conventional scale. The timbre
of the sound is a synthetic sound mimicking bubbles.
In the experiment, eight participants evaluated the effects of
smile-based sonification feedback. Participants expressed smiles
in a condition that there was feedback or no feedback. We investigated
what type of effects the feedback had on smiling by analyzing
surface EMG signals and interviewing the subjects.
The results suggest that the sonified feedback could facilitate
the expression of spontaneous smiles. In addition, results suggested
that both the attack and release of a smile were similarly
perceived using visual or auditory feedback.