AI-based defect and irregular pattern detection, classification and retrieval models enhanced with morphological feature engineering for enabling AI reasoning capability

The reliable detection and analysis of Defects and Irregular Patterns (DIPs) in image data is critical across various domains, including manufacturing, infrastructure monitoring, and medical diagnosis. Given the large volume of image data generated from inspection processes, advanced Artificial Intelligence (AI) techniques may be utilised to automate accurate and robust DIP detection and analysis. However, the adoption of AI requires interpretability—the ability to explain the reasoning behind model decisions in order to build appropriate trust. This research is focused on advancing AI capabilities for the precise identification of DIPs in images while also clarifying model decisions through characterisation and reasoning.

The first key contribution is the Image-Enhanced Mask R-CNN (IE-Mask-RCNN), a novel Deep Learning (DL) pipeline for automated wind turbine blade defect detection and classification. The pipeline uses image enhancement, augmentation techniques, and a Mask R-CNN architecture to precisely predict and identify defects. New defect prediction evaluation metrics are introduced, including prediction box accuracy, recognition rate, and false label rate. Experiments revealed that IE-Mask-RCNN achieved higher accuracy and lower error rates than other state-of-the-art DL algorithms, such as Mask R-CNN, YOLOv3, and YOLOv4. The pipeline also generates polygon masks that precisely localise defect regions.

The second key contribution is a new AI Reasoner framework for explaining AI model predictions. It extracts a comprehensive set of 38 Defect Characteristics (DefChars) to describe key properties of the input images. An ensemble decision tree model assesses the importance of each DefChar in influencing the AI model's predictions. The framework generates visualisations and textual suggestions based on the DefChar analysis to explain the model's reasoning. Another significant contribution is the integration of this interpretability methodology into a new Python toolkit called Forest Monkey. This enables the AI Reasoner to provide explanations for different AI models.

The third key contribution is extending the use of DefChars to a new application in image retrieval (ImR). The thesis introduces an ImR framework that extracts DefChars from images and uses them to search for similar DIPs across different datasets. Evaluations revealed that when utilised in the proposed ImR framework, DefChars enable more reliable ImR compared to when utilising other features such as resized images, Local Binary Pattern, and Scale-Invariance Feature Transform, even with imbalanced and small datasets. This demonstrates the cross-domain applicability of DefChars that goes beyond providing model explainability to applications that include ImR.