External Perception for Locomotives: Systems and Algorithm Development

Brown, Daniel; Jafek, Ben; Varney, Robert F. · 2020 · ROSA P / United States. Department of Transportation. Federal Railroad Administration

archive: archived pipeline: cataloged verified

Get this paper ↗ (full text — opens at the source; we link to it, we don't host it)

Summary

This report details the development of the External Perception for Locomotives (ExP-L) system, a vision-based operator augmentation tool sponsored by the Federal Railroad Administration (FRA) and developed by Aurora Flight Sciences. The research was motivated by the need to mitigate accidents caused by operator fatigue and distraction, such as signal violations and track obstructions, which the FRA estimates cause over 40 accidents annually. Unlike infrastructure-dependent systems like Positive Train Control, ExP-L relies solely on onboard sensors to detect and interpret signs, signals, and objects (SSOs), providing noncooperative situational awareness to locomotive engineers. The system was developed and tested within the FRA’s Cab Technology Integration Lab (CTIL), a comprehensive locomotive simulator. The methodology involved three phases: systems engineering, algorithm development, and human-machine interface (HMI) prototyping. For object detection, the team selected the YOLOv3 deep neural network architecture after a trade study comparing it against RetinaNet and Faster R-CNN. The model was trained on 30,740 images containing 103,067 annotated objects across eight classes, including cars, circular and elliptical signals, railroad crossings, station signs, mile markers, and track splits. To interpret the state of detected objects, the team employed traditional computer vision techniques, such as color classification for signals and optical character recognition for signs. The system integrated with the CTIL’s video output and web service to overlay urgent alerts onto the engineer’s display via a heads-up display (HUD). The results demonstrated that the YOLOv3 algorithm achieved 71 percent per-frame accuracy while operating at 10 frames per second, approaching real-time performance. This accuracy is significant because objects typically remain in view for 40–100 frames, allowing for multiple detection opportunities. The state interpretation algorithms performed with greater than 90 percent accuracy, with most exceeding 95 percent when tested on ground truth data. The system successfully demonstrated four key capabilities: alerting operators to vehicles on the track, identifying signal violations, detecting nonfunctional grade crossings, and maintaining location awareness through sign recognition. Incorrect interpretations were attributed to detection errors rather than failures in the interpretation logic itself. The significance of this work lies in its proof of concept for infrastructure-independent, vision-based safety systems. The ExP-L system provides a viable pathway for augmenting operator awareness without requiring extensive trackside infrastructure upgrades. The report concludes that the approach can be transitioned to operational locomotives and suggests that the CTIL-based prototype is suitable for further human factors research to refine the HMI. Future work includes testing in unconstrained outdoor environments and optimizing the system for real-world deployment to enhance railway safety.

Key finding

The ExP-L object detection algorithm achieved 71 percent per-frame accuracy at 10 frames per second, while state interpretation algorithms achieved greater than 90 percent accuracy.

Methodology

simulator

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via bulk_ingest_rosap on 2026-05-23 (6 acquisition events logged).

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• ehmi external hmi