Learning to drive from naturalistic trajectories: Offline reinforcement learning for safe speed guidance at signalized intersections
DOI: 10.26599/jicv.2026.9210085
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study addresses the challenge of optimizing speed guidance for connected and automated vehicles (CAVs) at signalized intersections within mixed traffic environments. Motivated by the limitations of rule-based strategies, which are often rigid, and optimization-based controllers like Model Predictive Control (MPC), which can be computationally prohibitive for real-time deployment, the authors propose an offline reinforcement learning (RL) framework. This approach leverages naturalistic driving trajectories to learn safe and efficient policies without requiring risky online exploration, thereby bridging the gap between simulation-based training and real-world deployment. The methodology employs Critic Regularized Regression (CRR), an offline RL algorithm that mitigates overestimation of Q-values by filtering high-quality actions from static datasets. The authors constructed a Markov Decision Process (MDP) using the UCF-SST-CitySim dataset, which contains real-world vehicle trajectories synchronized with Signal Phase and Timing (SPaT) data. The state space includes vehicle dynamics, distance to the stop line, and signal status, while the continuous action space controls acceleration. The reward function balances efficiency, comfort, and safety by incorporating travel distance, jerk (rate of change of acceleration), and Time-to-Collision (TTC). Experiments were conducted in the SUMO simulation environment, comparing the CRR framework against baselines including MPC, Behavior Cloning, Twin Delayed Deep Deterministic Policy Gradient (TD3), and Batch Constrained Q-learning (BCQ). Results demonstrate that the CRR-based framework significantly outperforms rule-based baselines in safety, comfort, and efficiency. Specifically, the average Time-to-Collision increased from 2.75 to 8.53 seconds, and jerk was reduced by over 50%, indicating smoother driving behavior. The system maintained a consistent time headway of approximately 1.68 seconds. Comparative analysis showed that CRR provided more stable performance than other state-of-the-art methods. Ablation studies and simulations involving communication losses, delays, and incomplete sensor data confirmed the framework’s robustness under realistic operational constraints. Furthermore, the method exhibited superior computational efficiency compared to MPC, requiring considerably less processing time. The significance of this work lies in its demonstration that offline RL can effectively learn from imperfect, naturalistic human driving data to produce robust autonomous driving policies. By integrating SPaT information and surrogate safety measures, the framework offers a practical solution for real-time speed guidance that enhances intersection safety and ride comfort. The findings support the feasibility of deploying data-driven RL agents in intelligent transportation systems, particularly in mixed-autonomy scenarios where ideal communication conditions cannot be guaranteed.
Key finding
The Critic Regularized Regression offline reinforcement learning framework significantly improves safety and comfort metrics compared to baseline methods, increasing time-to-collision from 2.75 to 8.53 seconds and reducing jerk by over 50%.
Methodology
simulation_modeling
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28 (2 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 3 | 2026-05-28 |
| archive | success | canonical_url | — | — | 1 | 2026-06-04 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | — | — | — | 1 | 2026-05-28 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Theoretical Contribution: computational model