A Literature Review of Performance Metrics of Automated Driving Systems for On-Road Vehicles

Sharath, Mysore Narasimhamurthy; Mehran, Babak · 2021 · DOAJ

DOI: 10.3389/ffutr.2021.759125

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This literature review addresses the critical need for standardized, objective performance metrics for Automated Driving Systems (ADS), specifically focusing on the complex environment perception and motion planning modules. The motivation stems from the absence of clear regulatory consensus on ADS safety standards, despite the technology’s potential to reduce human-error-related accidents. Current voluntary assessments by developers often rely on ill-conceived metrics that may provide a false sense of security. The authors argue that well-conceived metrics must be practicable, objective, and capable of quantifying the "human-likeness" of ADS to ensure safe coexistence with human drivers. The study employs a comprehensive review of existing literature to evaluate current performance indicators. It categorizes ADS components into hardware (sensors like cameras, LiDAR, RADAR), software (sensor fusion, algorithms), and vehicle states. The authors analyze traditional metrics such as Recall, Precision, F1 score, and Intersection over Union (IoU) for object detection, as well as CLEAR metrics like Multiple-Object-Tracking Accuracy (MOTA) and Precision (MOTP) for tracking. The review highlights the limitations of these binary or threshold-based metrics, which often fail to account for the severity of errors or multivariate interactions in dynamic environments. Key findings include the identification of significant gaps in current evaluation methods. The authors demonstrate that existing metrics do not adequately quantify the level of threat an obstacle poses, nor do they sufficiently capture the human-like nature of ADS perception and planning. They propose a novel methodology using a multivariate cumulative distribution approach to assess threat levels, arguing that erroneous perception of low-threat obstacles is less critical than missing high-threat ones. Furthermore, the review identifies that precrash events and states are currently underreported, limiting the ability of regulators to comprehensively assess ADS safety. The significance of this work lies in its proposed framework for safety regulation and data collection. The authors suggest that regulatory authorities establish a repository of precrash sequences to benchmark ADS performance objectively. This repository would enable the modeling of precrash scenarios, the anticipation of edge cases, and the evaluation of sensor configurations. By incorporating threat levels and human-likeness into performance metrics, the proposed approach aims to provide a more robust, scientific basis for ADS safety standards, facilitating better regulatory oversight and technological advancement without compromising road safety.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• naturalistic crash near crash
• adas effectiveness

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: crash risk outcomes
• Methodological Resource: dataset resource, tool software