File(s) under permanent embargo

Reason: This item is currently closed access.

Spatial synchronization of audiovisual objects by 3D audio object coding

conference contribution
posted on 11.10.2016, 13:15 by Banu Gunel, Erhan Ekmekcioglu, Ahmet Kondoz
Free viewpoint video enables the visualisation of a scene from arbitrary viewpoints and directions. However, this flexibility in video rendering provides a challenge in 3D media for achieving spatial synchronicity between the audio and video objects. When the viewpoint is changed, its effect on the perceived audio scene should be considered to avoid mismatches in the perceived positions of audiovisual objects. Spatial audio coding with such flexibility requires decomposing the sound scene into audio objects initially, and then synthesizing the new scene according to the geometric relations between the A/V capturing setup, selected viewpoint and the rendering system. This paper proposes a free viewpoint audio coding framework for 3D media systems utilising multiview cameras and a microphone array. A real-time source separation technique is used for object decomposition followed by spatial audio coding. Binaural, multichannel sound systems and wave field synthesis systems are addressed. Subjective test results shows that the method achieves spatial synchronicity for various viewpoints consistently, which is not possible by conventional recording techniques.

Funding

This work has been supported by the MUSCADE Integrating Project (www.muscade.eu), funded under the European Commission ICT 7th Framework Programme.

History

School

  • Loughborough University London

Published in

MMSP

Pages

460 - 465

Citation

GUNEL, B., EKMEKCIOGLU, E. and KONDOZ, A., 2010. Spatial synchronization of audiovisual objects by 3D audio object coding. IN: Proceedings of 2010 IEEE International Workshop on Multimedia Signal Processing (MMSP 2010), Saint Malo, France, 4-6 October 2010, pp.460-465.

Publisher

© IEEE

Version

VoR (Version of Record)

Publisher statement

This work is made available according to the conditions of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. Full details of this licence are available at: https://creativecommons.org/licenses/by-nc-nd/4.0/

Publication date

2010

Notes

Closed access.

ISBN

9781424481125

Language

en

Exports

Logo branding

Categories

Exports