Simulation and optimization of traffic in a city

Wiering, Marco; Vreeken, Jilles; van Veenen, Jelle; Koopman, A. · 2004 · OpenAlex-citations

DOI: 10.1109/ivs.2004.1336426

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the problem of optimizing traffic light control in urban environments to minimize vehicle waiting times. The authors frame this as a multi-agent decision problem and propose using reinforcement learning (RL) algorithms to learn optimal traffic light configurations. Unlike fixed-cycle controllers, their adaptive approach learns the expected waiting times for cars at each intersection and adjusts lights to maximize individual car gains. To evaluate this method, the researchers developed the Green Light District (GLD) simulator, a microscopic cellular automaton model that allows for the construction of various infrastructures and the testing of different control algorithms. The proposed RL algorithm utilizes a car-based value function rather than a traffic-light-based one. It estimates the expected waiting time for each car by learning transition probabilities and the likelihood of lights being red or green through frequency counting. The traffic light controller selects the configuration that maximizes the cumulative gain, defined as the difference between waiting times for red and green lights for all cars in the queue. Additionally, the system supports "co-learning," where vehicles use the computed waiting times to select routes that minimize their total travel time. The authors compare their RL controllers (TC-1 variants) against several hand-designed controllers, including Best First, Relative Longest Q, and ACGJ-3, which uses a bucket mechanism to handle high traffic densities. Experiments were conducted on two distinct infrastructures: a grid-like network with 16 junctions and a city-center model surrounded by ring roads. In the grid experiment, the TC-1 algorithm with co-learning achieved the lowest average trip waiting time (ATWT) of 6.46, significantly outperforming the best non-adaptive controller (ACGJ-3), which had an ATWT of 8.46. In the city-center experiment, the TC-1 Destinationless variant performed best with an ATWT of 2.67, while co-learning resulted in slightly higher waiting times due to route saturation. Across both experiments, all RL-based controllers outperformed the hand-designed alternatives, with the best RL algorithms reducing waiting times by more than 25% compared to the best non-adaptive controller. The study concludes that reinforcement learning provides a superior method for adaptive traffic light control compared to traditional fixed or heuristic-based controllers. The car-based value function approach allows for more granular optimization than previous RL methods that used traffic-light-based value functions. The results demonstrate that adaptive controllers can significantly reduce average waiting times and handle complex traffic patterns. The authors suggest future work should include studying non-stationary traffic patterns, incorporating inter-controller communication, and refining the traffic model to account for varying vehicle speeds.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• mental model of traffic

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Theoretical Contribution: computational model