Audio-Visual Panoramas and Spherical Audio Analysis using the Audio Camera
MetadataShow full item record
Capturing a scene for later or contemporaneous display needs to capture the complex interactions between the source(s) in the scene and the environment. High order spherical Ambisonics and plane-wave analysis are powerful mathematical tools for such scene analysis. The spherical microphone array (and its embodiment in the Audio Camera) is a useful tool for capture and analysis of scenes. Further information about the environment is available from the visual scene. We present the audio-visual panoramic camera as a tool that greatly simplifies the task of processing audio visual information by providing one common framework for both modalities. Via the Audio Camera , we show that microphone arrays can be viewed as a central projection camera that can effectively image the audible acoustic frequency spectrum. We demonstrate a new device, the audio visual Panoramic camera that is composed of a 64 channel spherical microphone array combined with a 5 element video camera array. The combined sensor is capable of real-time audio visual panoramic image generation using state of the art NVidia Graphics cards. It also provides an order-7 ambisonic description of the scene.
Showing items related by title, author, creator and subject.
A Novel Approach to Streaming and Client Side Rendering of Multichannel Audio with Synchronised Metadata (demonstration) Paradis, Matthew; Pike, Chris; Day, Richard; Melchior, Frank (Georgia Institute of Technology, 2016-04)Object based audio broadcasting is an approach which combines audio with metadata that describes how the audio should be rendered. This metadata can include spatial positioning mixing parameters and descriptors to define ...
Adenot, Paul (Georgia Institute of Technology, 2016-04)In this tutorial, we will look at two different aspects of working with the Web Audio API. First, we’ll have a look into the performance characteristics of the different AudioNodes available, their performance profile, ...
Interactive Audiovisual Rendering of Recorded Audio and Related Data with the WavesJS Building Blocks Matuszewski, Benjamin; Schnell, Norbert; Goldszmidt, Samuel (Georgia Institute of Technology, 2016-04)This article presents a set of components for the interactive audiovisual rendering of recorded audio signals and related data streams (e.g. audio descriptors and annotations) together with a set of example applications. ...