Deep Reinforcement Learning Based High-level Driving Behavior Decision-making Model in Heterogeneous Traffic

Bai, Zhengwei; Shangguan, Wei; Cai, Baigen; Chai, Linguo · 2019 · Crossref

DOI: 10.23919/chicc.2019.8866005

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of high-level driving behavior decision-making for connected vehicles (CVs) operating in heterogeneous traffic, where autonomous vehicles share the road with human-driven vehicles. The authors identify limitations in existing approaches, noting that many models rely on unrealistic assumptions about other vehicles' intelligence, output low-level control signals rather than high-level policies, or use raw sensor data that hinders generalization. To overcome these issues, the study proposes a deep reinforcement learning (DRL) model designed to learn optimal driving policies that balance efficiency and safety. The proposed model consists of three main components: a data preprocessor, a two-stream deep neural network, and a DRL network. The preprocessor maps raw Vehicle-to-Everything (V2X) communication data into a unified format called a Hyper-Grid Matrix (HGM), which decouples raw data from the neural network and normalizes vehicle speed and position. The neural network employs a parameter-sharing two-stream architecture to process both RGB camera images and HGM data, extracting hidden features. The DRL component utilizes a Dueling Deep Q-Network (DQN) with prioritized experience replay to determine discrete high-level actions: acceleration, deceleration, lane changes, or no action. The reward function is designed to encourage higher speeds and overtaking while penalizing collisions, dangerous maneuvers, and unnecessary lane changes. Experiments were conducted using a simulation environment built on the Unity ML-Agent framework, featuring a five-lane highway with dense traffic composed of conservative human-driven vehicles, aggressive human-driven vehicles, and normal CVs. The model was trained for one million steps and tested across five scenarios with varying ratios of connected to human-driven vehicles. The proposed model was compared against a non-intelligent baseline, Min’s method (a deep Q-learning approach), and a Coordinate method (which feeds raw V2X data directly without HGM mapping). The results demonstrate that the proposed model significantly outperforms the baselines. It achieved faster training convergence and higher average driving speeds compared to Min’s method and the Coordinate method. Furthermore, the model successfully learned to drive efficiently through heterogeneous traffic with fewer unnecessary lane changes and a higher number of overtakes. The study concludes that the combination of the HGM data format and the two-stream Dueling DQN architecture effectively handles the complexity of heterogeneous traffic, providing a robust solution for high-level driving policy determination in connected vehicle systems.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Theoretical Contribution: computational model