Robust Detection, Association, and Localization of Vehicle Lights: A Context-Based Cascaded CNN Approach and Evaluations

Gopalkrishnan, Akshay; Greer, Ross; Keskar, Maitrayee; Trivedi, Mohan M. · 2023 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2307.14571

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of robustly detecting, associating, and localizing vehicle lights, a critical task for autonomous driving systems that rely on light states (e.g., turn signals, brake lights) to predict the future trajectories of surrounding vehicles. Existing methods often use single-stage detectors that decouple light identification from vehicle instances or rely on bounding boxes that fail to capture the irregular shapes of lights. The authors propose a context-based, cascaded Convolutional Neural Network (CNN) approach that predicts the four corners of a vehicle light given an upstream vehicle detection and an estimated light center. This method implicitly associates lights with their respective vehicles and provides more precise localization than standard bounding boxes. The study formulates the problem as a regression task, predicting four (x, y) coordinates relative to a known light center. The authors trained and evaluated their models on the LISA Lights Dataset, which contains over 40,000 cropped images of vehicle lights. To ensure robustness against errors in upstream center estimation, they introduced random offset noise to the ground-truth centers during training. They tested various CNN architectures, including ResNet and DenseNet variants, as well as Vision Transformers. The models were modified to output eight values (four corner coordinates) normalized via tanh activation, trained with a custom Mean Squared Error loss that ignores non-visible corners. The authors also compared two cropping strategies: one using the full traffic scene context and another using only the vehicle context. Experimental results indicate that ResNet-101 achieved the best performance, yielding an average distance error of 4.77 pixels, which corresponds to approximately 16.33% of the average vehicle light size. Models trained on ground-truth centers generally outperformed those trained with added noise, suggesting that accurate center inputs strengthen the model’s understanding of light location. The "Vehicle-Only Context" cropping approach significantly outperformed the full-scene approach by filtering out surrounding traffic noise and constraining predictions within the vehicle region. Data augmentation via horizontal reflection did not improve performance, likely due to the creation of artificial viewing angles. While left-front and left-rear light models performed best due to larger dataset sizes, the front-right model showed lower accuracy due to fewer training examples. Compared to prior taillight detection methods, the proposed approach achieved high mean Average Precision (mAP@50 of 84.15%) and demonstrated robustness across various lighting conditions and distances. The significance of this work lies in its modular, cascaded design that improves the precision of vehicle light localization while ensuring implicit association with vehicle instances. By moving beyond bounding boxes to corner-based regression, the method better captures the geometric diversity of vehicle lights. This approach provides a reliable foundation for downstream tasks, such as interpreting driver intentions and predicting vehicle trajectories, thereby enhancing the safety and situational awareness of autonomous driving systems.

Key finding

A cascaded CNN approach using ResNet-101 to predict vehicle light corners from estimated centers achieved an average distance error of 4.77 pixels, demonstrating robust performance across various lighting conditions and vehicle types.

Methodology

simulation_modeling

Sample size: 40000

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• vehicle conspicuity