Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network

Azimi, Seyed Majid; Kraus, Maximilian; Bahmanyar, Reza; Reinartz, Peter · 2021 · Crossref

DOI: 10.3390/rs13101953

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of Multi-Object Tracking (MOT) for pedestrians and vehicles in high-resolution aerial imagery, a domain characterized by unique difficulties such as moving cameras, low frame rates, heavy occlusions, and the small size of objects relative to the image scale. While Visual Object Tracking (VOT) has advanced significantly in ground-based imagery using Deep Learning (DL), aerial MOT remains underexplored due to the scarcity of suitable datasets and the distinct characteristics of aerial data, where objects like pedestrians may occupy fewer than 16 pixels. The authors aim to overcome these hurdles by evaluating traditional and DL-based Single- and Multi-Object Tracking methods and proposing a novel DL-based architecture, AerialMPTNet, designed to fuse appearance, temporal, and graphical information for robust tracking. The proposed AerialMPTNet is an end-to-end trainable, regression-based neural network that integrates three key modules: a Siamese Neural Network (SNN) for appearance feature extraction, a Long Short-Term Memory (LSTM) network for temporal information and movement prediction, and a Graph Convolutional Neural Network (GCNN) to model spatial and temporal relationships between adjacent objects. The authors also investigate the impact of Squeeze-and-Excitation (SE) layers and Online Hard Example Mining (OHEM) on performance, noting these are the first applications of such techniques in regression-based MOT. Additionally, they propose Euclidean Online Tracking (EOT), a method using Ground Sampling Distance (GSD) adapted Euclidean distance for object association. Experiments were conducted on three aerial MOT datasets: AerialMPT and the KIT AIS pedestrian and vehicle datasets, which feature varying object densities, movement patterns, and image qualities captured by airborne platforms. Results demonstrate that AerialMPTNet outperforms all previous methods on pedestrian datasets and achieves competitive results on vehicle datasets. The integration of LSTM and GCNN modules significantly enhances tracking performance by addressing challenges related to similar appearance features and dense object movement. However, the inclusion of SE layers and OHEM yielded mixed results, improving performance in some scenarios while degrading it in others. Furthermore, the study found that the L1 loss function generally produced better results than the Huber loss function across most scenarios. The proposed EOT method also outperformed tracking methods relying on Intersection over Union (IoU)-based association strategies. The significance of this work lies in its comprehensive evaluation of aerial MOT challenges and the introduction of AerialMPTNet, which provides a robust framework for tracking small, densely packed objects in aerial imagery. By demonstrating the effectiveness of fusing appearance, temporal, and graphical data, the paper offers valuable insights for future research in remote sensing and surveillance applications, such as disaster management and traffic monitoring. The extensive ablation studies and comparisons with traditional and state-of-the-art methods establish a baseline for aerial MOT, highlighting the specific architectural components necessary to handle the unique constraints of aerial video data.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• distraction detection algorithms