sensors-18-02730.pdf (7.19 MB)

Robust fusion of LiDAR and wide-angle camera data for autonomous mobile robots

Download (7.19 MB)
journal contribution
posted on 01.11.2018 by Varuna De Silva, Jamie Roche, Ahmet Kondoz
Autonomous robots that assist humans in day to day living tasks are becoming increasingly popular. Autonomous mobile robots operate by sensing and perceiving their surrounding environment to make accurate driving decisions. A combination of several different sensors such as LiDAR, radar, ultrasound sensors and cameras are utilized to sense the surrounding environment of autonomous vehicles. These heterogeneous sensors simultaneously capture various physical attributes of the environment. Such multimodality and redundancy of sensing need to be positively utilized for reliable and consistent perception of the environment through sensor data fusion. However, these multimodal sensor data streams are different from each other in many ways, such as temporal and spatial resolution, data format, and geometric alignment. For the subsequent perception algorithms to utilize the diversity offered by multimodal sensing, the data streams need to be spatially, geometrically and temporally aligned with each other. In this paper, we address the problem of fusing the outputs of a Light Detection and Ranging (LiDAR) scanner and a wide-angle monocular image sensor for free space detection. The outputs of LiDAR scanner and the image sensor are of different spatial resolutions and need to be aligned with each other. A geometrical model is used to spatially align the two sensor outputs, followed by a Gaussian Process (GP) regression-based resolution matching algorithm to interpolate the missing data with quantifiable uncertainty. The results indicate that the proposed sensor data fusion framework significantly aids the subsequent perception steps, as illustrated by the performance improvement of a uncertainty aware free space detection algorithm.


Authors would like to thank Loughborough University for providing seed funding for the research carried out and for the doctoral studentship of Jamie Roche.



  • Loughborough University London

Published in

Sensors (Switzerland)






DE SILVA, V., ROCHE, J. and KONDOZ, A., 2018. Robust fusion of LiDAR and wide-angle camera data for autonomous mobile robots. Sensors, 18 (8), 2730.


MDPI © The Authors


VoR (Version of Record)

Publisher statement

This work is made available according to the conditions of the Creative Commons Attribution 4.0 International (CC BY 4.0) licence. Full details of this licence are available at: by/4.0/

Publication date



This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (