Autonomous Driving in Roundabout Maneuvers Using Reinforcement Learning with Q-Learning
DOI: 10.3390/electronics8121536
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the challenge of autonomous vehicle navigation in roundabouts, a complex driving scenario requiring precise lane selection, adherence to priority rules, and interpretation of other drivers' intentions. The authors argue that while supervised learning methods can predict variables, they often fail to maintain autonomous direction or avoid collisions, and unsupervised methods are unsuitable for decision-making. Consequently, the study proposes a reinforcement learning approach using the Q-learning algorithm to train an autonomous agent to execute smooth, efficient, and safe maneuvers. The primary motivation is to develop a tangible learning technique for sequential decision-making that allows the vehicle to learn optimal behaviors through trial and error, achieving a high success rate in navigating roundabouts both with and without surrounding traffic. The methodology employs the CARLA simulation environment, which provides a 600 m × 600 m urban map containing approximately 5 km of road and specific roundabout scenarios. The system is modeled as a Markov Decision Process (MDP) where the agent interacts with the environment through a finite state space, a set of actions, and a reward function. The agent’s actions include steering (angles between −40° and +40°), throttle, braking, and hand brake control. Perception is handled via RGB cameras with semantic segmentation, depth cameras, and GPS data, which provide information on road geometry, lane markings, and vehicle position. The Q-learning algorithm updates the action-value function $Q(s, a)$ based on immediate rewards and predicted future values, allowing the agent to converge on an optimal policy. The study utilizes naturalistic driving data for contextual speed information and trains the agent in two distinct scenarios: navigating an empty roundabout to learn lane tracking, and navigating a roundabout with traffic to learn safe entry, circumnavigation, and exit maneuvers. The results demonstrate that the Q-learning-based agent successfully learns to navigate roundabouts effectively. In simulations, the vehicle agent achieved smooth and efficient driving behaviors, correctly identifying entry and exit lanes and adhering to priority rules. The reward policy effectively penalized incorrect actions, guiding the agent toward optimal decision-making. The agent was able to determine the correct route regardless of the presence of other vehicles, successfully executing the approach, crossing, and exit phases of the roundabout maneuver. The study confirms that the reinforcement learning framework allows the autonomous vehicle to adapt its behavior based on environmental feedback, resulting in safe navigation without collisions. The significance of this work lies in its demonstration that reinforcement learning, specifically Q-learning, is a viable method for handling the uncertainty and complexity of roundabout navigation in autonomous driving. By moving beyond supervised prediction to autonomous decision-making, the approach ensures that vehicles can maintain correct orientation and avoid obstacles dynamically. The findings suggest that such learning techniques can be applied to other complex driving scenarios where uncertainty is high, contributing to the development of more robust and safe autonomous driving systems. The use of the CARLA simulator also highlights the utility of simulated environments for training and evaluating these algorithms before real-world deployment.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | OpenAlex-citations | — | — | 1 | 2026-06-20 |
| archive | success | openalex | — | — | 5 | 2026-06-26 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-20 |
| chunk | success | chunk | — | — | 1 | 2026-06-20 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-20 |
| promote | success | — | — | — | 1 | 2026-06-20 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-20 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Theoretical Contribution: computational model