DeepTake: Prediction of Driver Takeover Behavior using Multimodal Data

Pakdamanian, Erfan; Sheng, Shili; Baee, Sonia; Heo, Seongkook; Kraus, Sarit; Feng, Lu · 2020 · arXiv

DOI: 10.1145/3411764.3445563

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper introduces DeepTake, a deep neural network (DNN) framework designed to predict driver takeover behavior in automated vehicles (AVs). The research addresses the safety challenge of handing control back to drivers who may be engaged in non-driving related tasks (NDRTs), such as reading or using mobile devices. Existing methods often fail to reliably predict whether a driver will respond to a takeover request (TOR), how long the response will take, or the quality of the subsequent driving maneuver. DeepTake aims to provide a unified prediction of these three aspects—intention, time, and quality—to enable AVs to adapt their alerting strategies and ensure safe transitions. The authors developed DeepTake using multimodal data collected from a driving simulator study with 20 participants. The input features included driver biometrics (eye movement, heart rate via PPG, and galvanic skin response), pre-driving survey responses (demographics, workload, and stress levels), details of the NDRTs being performed, and vehicle dynamics (lane position, steering angle, velocity). Data was segmented into 10-second windows preceding each TOR. The framework employs a feed-forward DNN with three hidden layers, trained using mini-batch stochastic gradient descent. To handle class imbalance, the authors used Synthetic Minority Oversampling Technique (SMOTE). The prediction tasks were defined as binary classification for takeover intention (whether the driver responds) and multi-class classification for takeover time (low, medium, high) and takeover quality (based on lateral deviation during obstacle avoidance). Experimental results demonstrate that DeepTake significantly outperforms six baseline machine learning models and previous state-of-the-art methods. The framework achieved an accuracy of 96% for predicting takeover intention, 93% for takeover time, and 83% for takeover quality. These high accuracy rates suggest that the integration of multimodal data, particularly biometrics and contextual task information, provides robust signals for predicting driver readiness. The study confirms that factors such as cognitive load and stress, captured through physiological sensors, are critical for accurate prediction. The significance of this work lies in its potential to improve the safety and user experience of automated driving systems. By accurately predicting driver behavior, AVs can optimize the timing and modality of takeover requests, potentially warning drivers to reduce distraction if their predicted takeover quality is low. This approach supports the development of adaptive human-automation interaction systems that balance safety with the productivity benefits of allowing drivers to engage in secondary tasks during automated driving.

Key finding

Multimodal fusion of survey, vehicle, NDRT, and biometric features in a deep neural network predicts takeover intention, time, and quality more accurately than single-source models, suggesting practical takeover-readiness estimation for conditionally automated vehicles.

Methodology

other

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via discover_arxiv on 2026-05-03 (3 acquisition events logged).

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• takeover transitions
• automation
• distraction detection algorithms
• automation surprise
• situational awareness
• temporal

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: behavioral performance data
• Methodological Resource: measurement protocol
• Theoretical Contribution: conceptual framework