Adaptive Railway Traffic Control using Approximate Dynamic Programming

Ghasempour, Taha; Heydecker, Benjamin · 2019 · Crossref

DOI: 10.1016/j.trpro.2019.05.012

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of real-time railway traffic management (RTM) by proposing an adaptive controller based on Approximate Dynamic Programming (ADP). The motivation stems from the increasing congestion of railway networks and the limitations of current industry practices, which often rely on manual dispatcher experience or computationally intensive optimization methods like Mixed Integer Linear Programming (MILP) and exact Dynamic Programming (DP). These traditional approaches struggle with the stochastic nature of railway operations and the "curse of dimensionality," making them impractical for real-time application in complex, congested networks. The study aims to develop a framework that limits consecutive delays caused by trains entering a control area behind schedule by optimizing train sequencing at critical locations, such as junctions, while maintaining computational efficiency. The authors develop an ADP framework that approximates the value function of dynamic programming using a linear function approximation. This approach reduces computational burden by focusing on explicit evaluation of performance in the near future while using the approximation to estimate long-term costs. The parameters of the approximation function are updated dynamically using reinforcement learning techniques, specifically variants of Temporal Difference (TD) learning. The study explores different TD learning strategies, including one-step, M-step, and Least Squares TD learning, to adjust the approximation parameters based on operational experience. The control algorithm is implemented within a high-fidelity microscopic railway traffic simulator that accounts for stochastic variations, train characteristics, and signaling constraints (two, three, and four-aspect signals). The simulator evaluates the ADP controller’s ability to sequence trains and minimize total consecutive delays compared to the standard First-Come-First-Served (FCFS) heuristic. The results from the stochastic simulation environment demonstrate that the ADP-based controller achieves considerable improvements in reducing consecutive delays compared to the FCFS industry standard. The adaptive nature of the controller allows it to respond effectively to perturbations and minor deviations from scheduled operations. Furthermore, the study found that the estimates of the parameters for the approximate value function remained similar across a range of test scenarios with different mean train entry delays. This consistency suggests that the learned approximation is robust and generalizable across varying levels of initial disruption, validating the effectiveness of the reinforcement learning techniques in capturing the underlying dynamics of railway traffic flow. The significance of this work lies in its demonstration that ADP can provide a computationally tractable and effective solution for real-time railway traffic control. By avoiding the extensive explicit evaluation required by exact DP, the proposed method offers a viable alternative for managing complex railway networks where stochasticity and large state spaces render traditional optimization methods infeasible. The findings support the potential for deploying automatic re-scheduling tools that can adapt to changing operating conditions, thereby enhancing network capacity, reliability, and passenger satisfaction. The paper contributes to the field by bridging the gap between theoretical optimization methods and practical, real-time operational requirements, highlighting the promise of reinforcement learning in transportation management systems.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• rail grade crossings