Providing Evidence for the Validity of the Virtual Verification of Automated Driving Systems

Neurohr, Birte; de Graaff, Thies; Eggers, Andreas; Bienmüller, Tom; Möhlmann, Eike · 2024 · Crossref

DOI: 10.1007/978-3-031-56776-6_1

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of validating virtual testing for Automated Driving Systems (ADS), where traditional statistical verification requires hundreds of millions of real-world kilometers, making it infeasible. The authors propose a "snippet-based trace validation" method to reduce the amount of real-world data needed to argue for the validity of simulation results. The core problem is establishing that a simulation environment is sufficiently similar to the real world to allow test results to transfer safely. The proposed method validates single simulation traces by comparing them against a database of traces that have been recorded in the real world and successfully replayed in simulation. To generalize validity from these known traces to new, unseen virtual traces, the authors introduce two mechanisms: slicing by validity aspects and slicing by time. First, traces are decomposed into independent "validity aspects" (e.g., front camera, rear camera, vehicle dynamics), reducing each to only its relevant "key variables." Second, these aspects are further sliced into short temporal chunks called "validity snippets" based on the causal history length of each aspect. A new test trace is validated if its corresponding snippets match those in the validated database, allowing the recombination of parts from different real-world recordings to validate a new scenario. The effectiveness of this approach was demonstrated in a proof of concept using the CARLA simulator. The authors designed parameterizable highway scenarios with varying complexity (easy, medium, complex) and evaluated the method against a "naive" validation approach that requires full trace matches. Using front and rear cameras and ego vehicle dynamics as validity aspects, the study showed that the snippet-based method could validate 1.7 to 2.2 times more test traces than the naive method. In the "easy" benchmark, nearly 60% of test traces were validated, though performance decreased in more complex scenarios due to combinatorial complexity. The significance of this work lies in providing a structured argument for the validity of virtual ADS testing, potentially making safety assessments more affordable and efficient. However, the authors note limitations, including the risk of emergent behavior being lost when splitting traces, potential timing discrepancies between simulation modules and real-world systems, and the need for robust similarity metrics. Future work aims to apply the method to actual real-recorded scenarios and investigate these coupling effects further.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• simulator validity fidelity
• dms validation

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Methodological Resource: validation psychometrics, tool software
• Theoretical Contribution: computational model