Adaptive Failure Search Using Critical States from Domain Experts

Du, Peter; Driggs-Campbell, Katherine · 2021 · Unknown

DOI: 10.1109/icra48506.2021.9561477

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of efficiently validating safety-critical autonomous systems, specifically autonomous vehicles (AVs), by improving failure search techniques. Traditional validation methods, such as random simulation or physical testing, are inefficient due to the rarity of failure events. Adaptive Stress Testing (AST) offers a solution by formulating failure search as a Markov decision process solved via reinforcement learning. However, standard AST relies on probability models of environment actions, which are difficult to define for systems with discrete, high-level action spaces where agent dependencies exist. The authors propose incorporating "critical states"—scenarios identified as dangerous by domain experts—into the AST reward function to guide the search toward more meaningful failures without requiring explicit probability distributions. The authors develop a Human-based Critical State (HCS) classifier, a Bayesian neural network trained on human-labeled data to predict whether a given system state is dangerous. This classifier considers both the ego vehicle’s observations and the broader environment, addressing limitations of agent-centric metrics like Q-value variance. The HCS classifier is integrated into the AST framework, replacing or augmenting heuristic rewards. The reward function uses the classifier’s predicted probability and variance to penalize non-failure states, with variance acting as a confidence proxy to prevent misguided signals when the classifier is uncertain. Experiments were conducted using the Highway-Env simulator with a Deep Q-Network (DQN) controlled AV and a Monte Carlo Tree Search solver. The authors compared the HCS approach against two baselines: a heuristic reward based on longitudinal distance and a Q-value critical state reward. Results were evaluated using Responsibility-Sensitive Safety (RSS) metrics to quantify the proportion of unsafe states in generated failure trajectories. The Q-value approach showed only marginal improvement over heuristics. In contrast, the HCS approach significantly increased the proportion of RSS violations, with the median proportion of improper responses increasing by approximately 50% in tested scenarios. Qualitative analysis revealed that HCS-generated trajectories exhibited more aggressive and dangerous behaviors, such as unsafe following distances and cut-offs, compared to the other methods. The study concludes that incorporating human-defined critical states into AST significantly enhances the identification of dangerous failure scenarios in discrete action spaces. By leveraging a learned classifier, the method overcomes the limitations of hand-crafted heuristics and agent-centric metrics, providing a more robust framework for validating autonomous policies. This approach allows for efficient stress testing that captures a wider variety of unsafe behaviors, offering a practical solution for validating black-box systems where environment action distributions are unknown or complex.

Key finding

Incorporating a human-labeled critical state classifier into the Adaptive Stress Testing framework generates failure scenarios with a significantly higher proportion of unsafe states compared to heuristic or Q-value-based methods.

Methodology

simulation_modeling

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• naturalistic crash near crash