Developing a Standardized Performance Evaluation of Vehicles with Automated Driving Features
archive: archived pipeline: cataloged verified
Get this paper ↗ (full text — opens at the source; we link to it, we don't host it)
Summary
This study addresses the lack of standardized evaluation methods for Advanced Driver Assistance Systems (ADAS) in current production vehicles. While automated driving features like Adaptive Cruise Control (ACC), Lane Keep Assist (LKA), and Automatic Emergency Braking (AEB) are increasingly prevalent, existing research has largely focused on theoretical frameworks or extreme edge cases rather than routine daily driving scenarios. The primary objective was to develop and validate an initial set of standardized tests to assess the capabilities and limitations of these systems, providing a baseline for future industry consensus and conformance testing. The researchers developed test scenarios inspired by literature reviews, media reports of real-world failures, and vehicle operational design domains. Seven production vehicles from six manufacturers, equipped with Level 1 and Level 2 automation, were tested on closed tracks at the Virginia Tech Transportation Institute. The experimental design included seven specific scenarios: ACC Curve, ACC Cut-In, ACC Cut-Out & Reveal, ACC Stop & Go, LKA Inattentiveness, AEB Obstacle, and Lane Shift. Parameters such as speed, headway settings, and obstacle types were varied using a full factorial design. Three replications of each trial were conducted by four expert drivers to ensure consistency. Data collection relied primarily on qualitative observations coded into quantitative performance scores, supplemented by objective data from vehicle data acquisition systems. Statistical analysis, including ANOVA and t-tests, was used to identify significant performance differences. The results revealed considerable performance variability both across different manufacturers and within single vehicle models across repeated trials. Specific roadway characteristics significantly impacted system efficacy. For instance, ACC performance degraded on tight curves (108-foot radius) due to sensor field-of-view limitations, though medium headway settings improved tracking compared to long settings. In ACC Cut-Out scenarios, vehicles struggled significantly when the revealed vehicle was stationary, indicating current ACC systems are not designed for large speed deltas or stopped objects. Notably, none of the test vehicles detected barrel cones during Lane Obstruction or Lane Shift tests, highlighting a critical inability to recognize stationary work zone barriers. Additionally, response times for LKA inattentiveness warnings varied widely, and AEB systems frequently failed to react to soft-target obstacles, requiring driver intervention. The study concludes that standardized testing is essential for evaluating the current state of automated driving features and tracking improvements over time. The high variability and inconsistent performance observed, particularly regarding stationary objects and complex maneuvers, underscore the limitations of current ADAS implementations. These findings suggest that while standardized tests can effectively benchmark vehicle capabilities, significant refinement is needed to ensure consistency and predictability. The authors recommend expanding these testing frameworks to support future conformance processes, emphasizing that observational methods with minimal instrumentation provide valuable insights into system reliability and safety gaps.
Key finding
Considerable performance variability was observed between different vehicle manufacturers and within a single vehicle model across repeated trials, with specific roadway characteristics significantly impacting performance.
Methodology
on_road
Sample size: 7
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via bulk_ingest_rosap on 2026-05-23 (6 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | rosap | — | — | 2 | 2026-05-23 |
| archive | success | — | — | — | 1 | 2026-05-23 |
| extract | success | cached | — | — | 2 | 2026-06-10 |
| clean | success | — | — | — | 1 | 2026-06-01 |
| chunk | success | — | — | — | 1 | 2026-06-01 |
| embed | success | — | — | — | 1 | 2026-06-02 |
| enrich | success | — | — | — | 1 | 2026-05-23 |
| promote | success | — | — | — | 1 | 2026-05-23 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 3 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 19 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Applied Guidance: standards test procedures
- Methodological Resource: validation psychometrics, measurement protocol