Object detection in drone imagery using convolutional neural networks

Drones, also known as Unmanned Aerial Vehicles (UAVs), are lightweight aircraft that can fly without a pilot on board. Equipped with high-resolution cameras and ample data storage capacity, they can capture visual information for subsequent processing by humans to gather vital information. Drone imagery provides a unique viewpoint that humans cannot access by other means, and the captured images can be valuable for both manual processing and automated image analysis. However, detecting and recognising objects in drone imagery using computer vision-based methods is difficult because the object appearances differ from those typically used to train object detection and recognition systems. Additionally, drones are often flown at high altitudes, which makes the captured objects appear small. Furthermore, various adverse imaging conditions may occur during flight, such as noise, illumination changes, motion blur, object occlusion, background clutter, and camera calibration issues, depending on the drone hardware used, interference in flight paths, changing environmental conditions, and regional climate conditions. These factors make the automated computer-based analysis of drone footage challenging.

In the past, conventional machine-based object detection methods were widely used to identify objects in images captured by cameras of all types. These methods involved using feature extractors to extract an object’s features and then using an image classifier to learn and classify the object’s features, enabling the learning system to infer objects based on extracted features from an unknown object. However, the feature extractors used in traditional object detection methods were based on handcrafted features decided by humans (i.e. feature engineering was required), making it challenging to achieve robustness of feature representation and affecting classification accuracy. Addressing this challenge, Deep Neural Network (DNN) based learning provides an alternative approach to detect objects in images. Convolutional Neural Networks (CNNs) are a type of DNN that can extract millions of high-level features of objects that can be effectively trained for object detection and classification. The aim of research presented in this thesis is to optimally design, develop and extensively investigate the performance of CNN based object detection and recognition models that can be efficiently used on drone imagery.

One significant achievement of this work is the successful utilization of the state-of-the-art CNNs, such as SSD, Faster R-CNN and YOLO (versions 5s, 5m, 5l, 5x, 7), to generate innovative DNN-based models. We show that these models are highly effective in detecting and recognising Ghaf trees, multiple tree types (i.e., Ghaf, Acacia and Date Palm trees) and in detecting litter. Mean Average Precision (mAP@0.5IoU) values ranging from 70%-92% were obtained, depending on the application and the CNN architecture utilised.

The thesis places a strong emphasis on developing systems that can effectively perform under practical constraints and variations in images. As a result, several robust computer vision applications have been developed through this research, which are currently being used by the collaborators and stakeholders.