A Systematic Comparison for Consistent Scenario Development Using Microscopic Simulation Software

Saroj, Abhilasha; Xu, Guanhao; Shao, Yunli; Wang, Chieh Ross · 2024 · OpenAlex-citations

DOI: 10.1109/wsc63780.2024.10838810

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the lack of systematic methodologies for benchmarking traffic microsimulation software, a critical gap as researchers increasingly use these platforms to develop and test emerging vehicle control technologies and machine learning algorithms. The authors propose a systematic approach to ensure consistent scenario development across different modeling platforms, defined as producing comparable performance measures when using identical key inputs such as traffic demand, network geometry, and control strategies. The study focuses on three widely used microscopic simulation platforms: PTV VISSIM, AIMSUN, and SUMO. The researchers implemented this approach through a case study using a real-world traffic network in Downtown Chattanooga. They developed identical scenarios in all three platforms using the same field-collected data, including 10-minute volume aggregates, turn ratios, signal timing plans, and speed limits. Network geometry was standardized by converting a pre-existing SUMO network file into an OpenDRIVE format, which was then imported into VISSIM and AIMSUN. The study evaluated both network-level metrics (average travel time, vehicle counts) and vehicle-level metrics (individual speed, travel time, acceleration, and fuel efficiency). To assess robustness, the authors conducted simulations under three demand scenarios: base, low (25% decrease), and high (25% increase), running ten replicate trials with different random seeds for each. The results indicate that network-level performance was consistent across all three platforms for base and low-demand scenarios. However, significant discrepancies emerged under high-demand conditions. VISSIM exhibited higher average travel times and greater stochasticity across random seeds compared to AIMSUN and SUMO. At the vehicle level, while speed and travel time distributions were similar for base cases, acceleration distributions varied significantly between platforms. These differences in acceleration profiles led to divergent fuel efficiency estimates when calculated using the VT-Micro energy consumption model. Specifically, AIMSUN produced unrealistic acceleration values and lower fuel efficiency outliers not present in the other platforms. The study concludes that while a standardized scenario development workflow can achieve consistency for general mobility metrics under normal traffic conditions, it is insufficient for energy-focused applications without further calibration. The authors emphasize that driver behavior and acceleration parameters must be explicitly calibrated to match across platforms to ensure valid comparisons for energy efficiency studies. This work establishes a foundational framework for cross-platform benchmarking, which is essential for validating algorithms trained on simulation data and for ensuring reproducibility in research involving connected and automated vehicles.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• simulator validity fidelity

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Methodological Resource: tool software