Integration of Reinforcement Learning Based Behavior Planning With Sampling Based Motion Planning for Automated Driving

Klimke, Marvin; Völz, Benjamin; Buchholz, Michael · 2023 · Crossref

DOI: 10.1109/iv55152.2023.10186736

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of deploying deep reinforcement learning (RL) for automated driving by decoupling high-level behavior planning from low-level motion planning. While prior RL approaches often rely on end-to-end architectures that output direct control commands, these methods struggle with the "simulation-to-real" gap and lack robustness in real-world deployment. The authors propose a hybrid framework where an RL policy determines cooperative behavior strategies, which are then translated into smooth, drivable trajectories by a separate sampling-based motion planner. This approach aims to leverage the strategic decision-making capabilities of RL while ensuring safety and drivability through established motion planning algorithms. The methodology employs a graph neural network (GNN) based RL policy trained in a simulation environment to manage cooperative maneuvers in mixed traffic. To bridge the gap between the RL policy’s instantaneous actions and the motion planner’s need for future trajectory specifications, the authors utilize a built-in simulator to roll out the traffic scene. The RL policy predicts the evolution of the traffic scene over a horizon, generating simulated trajectories that serve as motion planning objectives. These objectives include path specifications, speed profiles, and anchor points with specific timing constraints. The system supports two operational modes: single-shot planning, where the plan is generated once, and cyclic replanning, where the plan is updated every two seconds to react to dynamic changes or deviations by other road users. Experimental validation was conducted using a real Audi A6 testing vehicle in a vehicle-in-the-loop setup, alongside simulative analyses for more complex scenarios. In real-world tests involving a prioritized crossing vehicle, the system successfully coordinated intersection traversal. When the crossing vehicle behaved predictably, single-shot and cyclic planning yielded similar results. However, in challenging scenarios where the crossing vehicle drove slower than anticipated, single-shot planning failed to adjust, leading to trajectories that violated safety constraints and risked collision. In contrast, cyclic replanning successfully detected the deviation, adjusted the anchor times, and generated a safe evasive maneuver by delaying the ego vehicle’s intersection entry. Simulative results further demonstrated that cyclic replanning enables the management of complex multi-agent maneuvers that single-shot planning cannot handle. The significance of this work lies in demonstrating the feasibility of integrating RL-based behavior planning with traditional motion planning for real-world automated driving. By using simulation to predict future states and cyclic replanning to handle uncertainties, the approach effectively shrinks the simulation-to-real gap. The results indicate that this hybrid architecture allows for safe, cooperative driving in mixed traffic environments, offering a viable path for deploying RL policies in automated vehicles without requiring end-to-end control.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Theoretical Contribution: computational model