Area-wide traffic signal control based on a deep graph Q-Network (DGQN) trained in an asynchronous manner

Kim, Gyeongjun; Sohn, Keemin · 2022 · OpenAlex-citations

DOI: 10.1016/j.asoc.2022.108497

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of jointly controlling traffic signals across a large transportation network using reinforcement learning (RL). Traditional multi-agent RL approaches struggle to guarantee a global optimum and often fail to coordinate actions effectively, while single-agent deep Q-networks (DQN) suffer from an exponential explosion in the action space as the number of intersections increases. Additionally, standard neural network layers inefficiently capture the spatio-temporal correlations inherent in road network topologies, and training such models requires prohibitive amounts of time. To resolve these issues, the authors propose a Deep Graph Q-Network (DGQN) trained via an asynchronous update methodology. The DGQN is structured as a single-agent RL model with two key architectural innovations. First, it employs a novel graph convolution scheme with parameterized adjacency matrices to efficiently model spatio-temporal dependencies. Unlike fixed adjacency matrices, this approach allows the model to learn varying connection intensities between upstream and downstream lane groups, capturing how traffic states propagate through the network. Second, the output structure of the DQN is revised to handle the large action space without exponential growth; the number of output nodes scales linearly with the number of intersections, allowing the model to select joint signal phases for all intersections simultaneously. To accelerate convergence, the authors implement an asynchronous training algorithm using multiple parallel actor-learners, each interacting with a separate simulation environment with distinct traffic demands and maintaining individual replay memories. The proposed DGQN was validated on a real-world transportation network in Seoul, Korea, comprising 15 intersections and 77 lane groups, simulated using Vissim software. The state inputs included traffic delay and queue length, while the reward function was based on minimizing total cumulative network delay. The DGQN was compared against a standard DQN with fully connected layers, a graph convolutional network with a fixed adjacency matrix, and actual fixed-signal operations. The results demonstrate that the DGQN successfully finds an optimal policy for joint traffic signal control, significantly outperforming both the baseline RL algorithms and the existing fixed-signal operations. The asynchronous training scheme enabled the model to converge to an optimal policy within a practical computation time (approximately 40 hours for 2,000 episodes). This study confirms that a single-agent RL framework, when augmented with graph-based feature extraction and asynchronous training, can effectively manage large-scale traffic networks, overcoming the limitations of multi-agent coordination and computational inefficiency.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• traffic density