Autonomous Vehicles that Alert Humans to Take-Over Controls: Modeling with Real-World Data
DOI: 10.1109/itsc48978.2021.9564434
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the critical challenge of safe control transitions in partially automated vehicles, specifically focusing on predicting the time required for a human driver to take over control from an autonomous system. The authors argue that effective transfer of control depends on understanding both the external driving scene and the internal state of the driver. To this end, they introduce a Take-Over Time (TOT) prediction model that utilizes real-world data to assess driver readiness, defined by three specific behaviors: eyes-on-road, hands-on-wheel, and foot-on-pedal. The study is motivated by the limitations of previous research, which often relied on simulations or post-hoc metrics rather than predictive, real-time analysis of driver behavior during actual driving scenarios. To develop this model, the researchers conducted a large-scale Controlled Data Study (CDS) involving 89 subjects driving a Tesla Model S equipped with three driver-facing cameras. Participants performed various distracting secondary activities, such as texting, reading, or phone calls, while the vehicle’s autopilot was engaged. Random auditory take-over requests were issued, resulting in a dataset of 1,375 control transition events. The data was annotated to mark the precise moments drivers resumed visual attention, hand placement, and foot placement. The proposed architecture employs independent Long Short-Term Memory (LSTM) networks to process mid-to-high-level features extracted from the camera feeds, including gaze direction, hand locations, and foot positions. This sequential approach allows the model to capture short- and long-term temporal patterns in driver behavior. The results demonstrate that the independent LSTM model effectively predicts continuous take-over times with low mean absolute errors. Ablation studies revealed that hand features, particularly those indicating hand-object interactions, were the most informative for prediction, followed by foot and gaze features. The model achieved an overall TOT mean absolute error of approximately 0.79 seconds on the validation set. Performance varied by secondary activity; tasks with higher cognitive loads, such as texting or reading, resulted in longer take-over times and higher prediction errors compared to less demanding activities like talking to a passenger. The study also found that using multiple camera views (face, hand, foot) was essential for optimal performance, with hand-object detection providing the strongest cues regarding driver distraction levels. The significance of this work lies in its contribution to the development of safer autonomous driving systems by enabling context-aware control transitions. By accurately predicting how long a driver needs to resume control, autonomous vehicles can make informed decisions about whether to hand over control immediately or execute a safe stop if the driver is not ready. The paper establishes a comprehensive framework for using non-intrusive in-cabin sensors to model driver state, providing a dataset and methodology that can benefit intermediate levels of automation. This approach moves beyond reactive metrics to proactive safety measures, ensuring smoother and safer interactions between humans and automated systems.
Key finding
A sequential neural network model utilizing independent LSTMs and visual driver features from multiple cameras can accurately predict take-over times in real-world driving scenarios, with hand-related features providing the most significant predictive value.
Methodology
field_study
Sample size: 89
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 2 | 2026-05-28 |
| archive | success | unpaywall | — | — | 2 | 2026-06-04 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | — | — | — | 1 | 2026-05-28 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: behavioral performance data
- Theoretical Contribution: conceptual framework, computational model