Microscopic Traffic Simulation by Cooperative Multi-agent Deep Reinforcement Learning

Bacchiani, Giulio; Molinari, Daniele; Patander, Marco · 2019 · Crossref

DOI: 10.65109/bgaa3250

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of simulating realistic, interactive driving behaviors in microscopic traffic simulators, specifically focusing on implicit negotiation maneuvers such as entering roundabouts. Traditional simulators rely on hard-coded rules that result in overly cautious or rigid behaviors, lacking the ability to negotiate right-of-way or exhibit diverse driving styles. To solve this, the authors propose a simulator using Cooperative Multi-agent Deep Reinforcement Learning (DRL), enabling agents to learn interaction strategies through experience rather than explicit programming. The methodology extends the Asynchronous Advantage Actor-Critic (A3C) algorithm to a multi-agent setting where agents share a global network but operate in parallel environment instances. This design allows agents to sense and react to each other, fostering cooperative behaviors. The input architecture combines visual data (a sequence of four top-view semantic images representing navigable space, obstacles, and paths) with a numerical vector. This numerical input is crucial for encoding non-visual information, such as absolute speed, and for tuning individual agent behaviors, specifically aggressiveness and desired cruising speed. The network processes these inputs through separate pipelines before merging them to output longitudinal acceleration actions. The system was evaluated in a synthetic single-lane roundabout scenario. The results demonstrate that the multi-agent A3C approach successfully trains vehicles to navigate complex interactions without hand-coded rules. By sharing parameters while allowing behavioral variation through numerical inputs, the simulator can generate diverse driving styles, from calm to aggressive. The parallel execution of agents mitigates the instability often associated with multi-agent learning, as updates from different environment instances reduce data correlation. The agents learned to implicitly negotiate space, effectively balancing the reward for reaching goals against the penalty for collisions. The significance of this work lies in its potential to improve the training of autonomous vehicle modules by providing a realistic simulation environment that captures human-like negotiation and variability. Unlike rule-based simulators, this approach allows for the emergence of complex, adaptive behaviors, which is essential for testing autonomous systems in mixed traffic with human drivers. The hybrid input architecture also offers flexibility, allowing the same model to be adapted to various scenarios by adjusting the numerical parameters, thereby supporting more robust and trustworthy autonomous driving development.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified_with_issues.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• mental model of traffic

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Methodological Resource: tool software
• Theoretical Contribution: computational model