Disengagement Cause-and-Effect Relationships Extraction Using an NLP Pipeline

Zhang, Yangtao; Yang, X. Jessie; Zhou, Feng · 2022 · OpenAlex-citations

DOI: 10.1109/tits.2022.3186248

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the critical need to understand the causes of Autonomous Vehicle Disengagement (AVD) to improve system safety and guide future regulations. While previous research relied on driving simulations or labor-intensive manual analysis of naturalistic data, this study proposes a scalable, end-to-end Natural Language Processing (NLP) pipeline to automatically extract cause-and-effect relationships from the California Department of Motor Vehicles (CA DMV) disengagement reports released between 2014 and 2020. The goal is to create a consolidated database that allows for efficient analysis of AV testing trends and system failures without manual intervention. The methodology involves a four-stage pipeline. First, multi-format reports (PDFs and CSVs) were collected and standardized using Optical Character Recognition (OCR) and cleaning protocols, resulting in a database of 14,282 filtered reports. Second, human annotators labeled cause-and-effect relationships and categorized causes using the CrowdTruth framework to ensure high annotation quality. Third, the authors developed an NLP model based on ELECTRA, utilizing deep transfer learning. The model was pre-trained on large corpora, fine-tuned on the SemEval-2010 Task 8 dataset for semantic relations, and post-trained on the CA DMV data. This approach was compared against BERT, DistilBERT, and XLNet. Finally, the extracted data was analyzed using a taxonomy covering AV system, human, and environmental factors, alongside statistical tests and visualizations. The results demonstrate that the ELECTRA model, when fine-tuned and post-trained, achieved the highest weighted F-1 score of 0.90 for cause-and-effect extraction, outperforming other models in both accuracy and computational efficiency. Analysis of the consolidated database revealed that test drivers initiated over 80% of disengagements, while more than 75% of disengagements were caused by errors in the AV system’s perception, localization, mapping, planning, or control. The study also identified significant relationships between the initiator of the disengagement and the cause category, and found that manufacturers tested AVs most intensively during spring and winter months. The significance of this work lies in its provision of a robust, automated tool for analyzing AV safety data at scale. By achieving human-level performance with minimal manual labeling, the pipeline enables continuous monitoring of AV performance as new data becomes available. The findings offer actionable insights for manufacturers to address specific system failures and for regulators to understand the realities of AV deployment. Furthermore, the consolidated database serves as a valuable resource for future research into autonomous driving safety and human-machine interaction.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• pre crash contributing factors

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Theoretical Contribution: conceptual framework