Automated Detection of Aircraft Surface Defects Using Deep Learning with Integrated Human Validation

Abstract

Visual inspection of aircraft surface is one of the many steps in the maintenance routines. Usually performed by operators, this procedure might last days to be accomplished. The use of automated process can help reduce time and results in accurate detection of surface defects on aircraft, as they are vital to maintain structural soundness and flight safety. This paper proposes a deep learning framework for automated defect detection based on Faster R-CNN with ResNet-50 Feature Pyramid Network (FPN) as the backbone model. This model was trained and validated on a sizable, labeled aircraft images with a maximum F1-score of 0.555 achieved in the test set. This is the result of preliminary study, where the authors aimed to detect all types of defects without classification. To further enhance reliability and allow for human input, a custom annotation validation user interface was implemented via Python, which allowed aircraft inspectors to view, edit, add, and acknowledge predictions made by the model in an attempt to hold onto precise level of annotation. This system also facilitated the management of annotations, visualization on irregular aircraft zones, and the creation of reports thus allowing for inspection workflows. The results show that combining state-of-the-art object detection with domain expertise in validation as route to reliable semi-automatic, standards-compliant aircraft defect detection is plausible. Future work will involve expanding the dataset, tuning for accuracy, and incorporating human feedback for enhancement of model utility over time.