A Taxonomy for Quality in Simulation-Based Development and Testing of Automated Driving Systems
DOI: 10.1109/access.2022.3149542
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the critical challenge of ensuring the quality and safety of automated driving systems (ADS), particularly for SAE levels 4 and 5. Because validating these systems through real-world testing requires billions of kilometers to ensure safety, the authors argue that simulation-based testing is a necessary alternative. However, relying on simulation data requires that the simulation environment itself possesses a defined quality level. The paper aims to resolve ambiguity in this field by proposing a conceptual taxonomy that systematically classifies quality aspects throughout the development and testing process. This framework is designed to help researchers and engineers distinguish between different quality domains, thereby clarifying where further testing is needed for the ADS, the simulation tools, and the scenarios used. The authors’ methodology involves a comprehensive review of existing literature on scenario-based testing, simulation abstraction levels, and quality metrics. They define key terms to avoid ambiguity and structure their analysis around the V-model of development, which integrates different "X-in-the-loop" phases (e.g., Software-in-the-Loop, Hardware-in-the-Loop). The core of their contribution is the division of simulation quality into three distinct categories: (1) the quality of the automated vehicle or its components (the system under test); (2) the quality of the simulation, including the simulation tool and its models; and (3) the quality of the scenarios used for testing. The paper also examines traffic simulation abstraction levels (macroscopic, mesoscopic, microscopic, and nanoscopic) and various scenario abstraction levels (functional, logical, and concrete) to contextualize where these quality assessments occur. The primary finding is the proposed taxonomy, which provides a structured classification for quality metrics and aspects. The authors demonstrate that quality must be evaluated at various resolution levels and process steps. For instance, they highlight that while metrics for evaluating driver behavior (such as Time-to-Collision or Post-Encroachment Time) are well-established, there is a lack of widely recognized verification and validation methods for the simulation models and their coupling mechanisms themselves. The taxonomy serves as a tool to categorize these metrics, making it easier to communicate specific areas of interest within a simulation setup. By mapping these categories to the development lifecycle, the paper illustrates how quality assurance must be integrated early and continuously, from abstract functional scenarios in the concept phase to concrete scenarios in later testing phases. The significance of this work lies in its potential to standardize the understanding of quality in automotive simulation. By providing a clear separation of quality domains, the taxonomy helps researchers and industry professionals organize the differences between testing the vehicle, the simulation tool, and the test scenarios. This clarity is essential for the safe introduction of highly automated vehicles, as it ensures that simulation results are credible and that all components contributing to those results are adequately validated. The authors conclude that this structured approach facilitates better communication and more rigorous quality assurance in the field of automated driving simulation.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | Crossref | — | — | 1 | 2026-06-18 |
| archive | success | openalex | — | — | 5 | 2026-06-25 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-20 |
| chunk | success | chunk | — | — | 1 | 2026-06-20 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-20 |
| enrich | success | openalex | — | — | 1 | 2026-06-20 |
| promote | success | — | — | — | 1 | 2026-06-18 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-20 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: validation psychometrics, tool software
- Theoretical Contribution: conceptual framework