Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

Rodrigues, Filipe; Azevedo, Carlos Lima · 2019 · Crossref

DOI: 10.1109/itsc.2019.8917451

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the lack of robustness in deep reinforcement learning (DRL) algorithms applied to traffic signal control, specifically regarding their performance under exogenous uncertainties. While DRL controllers often outperform conventional systems in stable environments, their reliability in dynamic urban areas subject to demand surges, incidents, and sensor failures remains underexplored. The authors aim to evaluate how these uncertainties impact DRL performance and propose design guidelines to mitigate their effects. To achieve this, the authors developed CAREL (CAllback-based REinforcement Learning), an open-source framework that integrates DRL controllers with traffic simulation environments. The study utilizes a single four-leg intersection modeled in AIMSUN Next, using real-world demand data as a baseline. The DRL agent employs a dueling deep Q-network architecture. Key design choices include a state representation based on maximum queue lengths per phase and elapsed time since the last green signal, which avoids reliance on high-dimensional or easily corrupted data. The action space involves either phase selection or time extension. The reward function is defined by the reduction in squared maximum queue lengths. Crucially, the neural network incorporates Dropout regularization to prevent overfitting and enhance robustness. The framework was tested against fixed-time and actuated baseline controllers across three uncertainty dimensions: demand surges (simulating special events), supply reductions (simulating incidents), and sensor failures. The results demonstrate that DRL controllers using phase selection significantly outperform traditional methods in base scenarios and maintain superior performance during demand surges. However, the performance gap narrows during low-demand scenarios. In supply variation experiments, the DRL controller effectively adapted to incidents, though its advantage over actuated controllers varied depending on the incident type. Most notably, the study found that Dropout regularization was essential for robustness against sensor failures; without it, the DRL agent’s performance degraded significantly when sensor data was noisy or missing. Additionally, the authors showed that transfer learning allows a pre-trained agent to adapt to new scenarios, such as post-event demand surges, much faster than training from scratch. The significance of this work lies in establishing a publicly available benchmark for evaluating the robustness of DRL in traffic control. It provides concrete empirical evidence that specific architectural choices, particularly state representation based on queue lengths and the use of Dropout, are critical for handling real-world uncertainties. The findings suggest that while DRL is promising, its deployment requires careful design to ensure stability under exogenous shocks, offering a pathway toward more reliable adaptive traffic management systems.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• mental model of traffic