From Simulation to Real World Maneuver Execution using Deep Reinforcement Learning

Capasso, Alessandro Paolo; Bacchiani, Giulio; Broggi, Alberto · 2020 · OpenAlex-citations

DOI: 10.1109/iv47402.2020.9304593

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of deploying Deep Reinforcement Learning (DRL) agents for autonomous driving maneuvers, specifically roundabout insertions, in real-world scenarios. The authors identify two primary obstacles to successful deployment: the lack of distinction between training and testing environments, which leads to overfitting, and the "reality gap" caused by differences between simulated and real-world data. To mitigate these issues, the study proposes a training pipeline inspired by supervised learning that incorporates distinct training, validation, and test datasets, alongside techniques to introduce uncertainty during training to improve robustness. The experimental design utilizes a Multi-environment System where active agents, responsible for the insertion maneuver, are trained simultaneously across four synthetic roundabout environments with varying shapes and lengths. A separate validation environment is used to select the best model weights, ensuring the agent does not overfit to specific training scenarios. Passive agents representing traffic are trained using Multi-agent A3C to cooperate and negotiate space. To bridge the simulation-to-reality gap, the authors inject artificial noise into the training process, including perception noise (errors in position, size, and detection of other vehicles) and localization noise (perturbations in the active agent’s path using Cubic Bézier curves). Additionally, entry lane shapes are modified periodically to expose the agent to varied navigable spaces. The results demonstrate that the proposed "Multi env&noise" approach significantly outperforms a baseline model trained on a single environment without noise injection. On unseen simulated test scenarios, the proposed method achieved a 99.1% success rate in completing insertions, compared to 90.7% for the baseline, while reducing crash rates from 9.3% to 0.9%. The model also showed superior generalization on a validation environment not used for weight updates, achieving a 99.4% success rate versus 92.0% for the baseline. Furthermore, the system successfully generalized to a different maneuver type, junction insertions, maintaining high performance. Crucially, real-world tests on a self-driving vehicle in Parma, Italy, confirmed that the noise-injected model could execute over 100 successful insertions in live traffic, whereas the baseline model failed due to instability caused by real-world sensor noise. The significance of this work lies in its demonstration that combining multi-environment training with explicit noise injection effectively reduces overfitting and enhances the robustness of DRL agents. By treating simulation training with the rigor of supervised learning—using validation sets and data augmentation via noise—the authors provide a viable pathway for transferring policies from synthetic environments to real-world autonomous driving applications. This approach ensures that agents remain effective despite the unpredictable uncertainties inherent in real-world perception and localization systems.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• simulator training transfer

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Theoretical Contribution: computational model