Airport runways must be constantly controlled and monitored by air traffic control to detect security endangering objects or circumstances such as swarms of birds or fog. This process requires constant attention and concentrated working from the air traffic controllers to ensure safe landings and take-offs. The automation of the object detection can support air traffic controllers to identify potential risks on the runway.
The automatic object detection on airport runways requires the processing of image and video data in order to detect objects, such as birds, airport staff or baggage cars, and to classify them. This can be done by a machine learning model which recognizes patterns in image data based on shapes and colours.
The major challenge is to generate a training data set, that provides sufficient labeled data of good quality in order to cover all possible objects on runways and to achieve accurate and reliable outcomes, which would otherwise cause serious security issues. Moreover, the system needs a very high accuracy and shouldn't produce a high number of "false positives" either not to dilute trust of the air traffic controllers into the software.
For the detection and classification of objects in images and videos, an image segmentation and classification model has to be implemented, for which the state-of-the-art algorithms are convolutional neural networks (CNN), such as U-Net or Mask R-CNN architectures.
To reduce the training data and effort, pre-trained models, such as VGG-16 or ResNet50, can be used. For video data, a YOLO ("You only look once") architecture might be suitable.
Although the object detection part might be relatively straightforward to implement, the major effort lies in the assessment of situations on the runway security. For example, a swarm of birds flying towards the runway is more severe than one that is flying away from the runway.
Crossing a runway of vehicles might be allright, if prior permission was given by the air traffic control. Handling exceptions and incorporating air traffic control knowledge is therefore essential to be able to use the software in production.