ViT-DD: Multi-Task Vision Transformer for Semi-Supervised Driver Distraction Detection
DOI: 10.1109/iv55156.2024.10588802
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the critical safety issue of distracted driving, which contributes significantly to motor vehicle accidents, particularly in scenarios involving conditional automation where drivers must remain alert. The authors propose ViT-DD, a multi-modal Vision Transformer designed for semi-supervised driver distraction detection. The motivation stems from the limitation of existing methods that ignore driver emotion as a complementary signal and the scarcity of datasets containing both distraction and emotion labels. ViT-DD aims to leverage inductive information from facial expression recognition to enhance the generalization and accuracy of distraction detection. The methodology employs a pure Transformer-based architecture that processes two input modalities: the full driver image and a cropped face image. These inputs are patched, embedded, and fed into a shared Transformer encoder with task-specific classification heads for distraction and emotion. To address the lack of emotion labels in standard distraction datasets, the authors develop a pseudo-labeled multi-task self-training algorithm. First, a teacher Vision Transformer is trained on the AffectNet-7 dataset for facial expression recognition. This teacher model then generates pseudo-labels for the face images in distraction detection datasets. Finally, a student ViT-DD model is trained using both ground-truth distraction labels and these pseudo-emotion labels. Experiments were conducted on two benchmark datasets: State Farm Distracted Driver Detection (SFDDD) and the American University in Cairo Distracted Driver Dataset (AUCDD), utilizing both split-by-image and more rigorous split-by-driver evaluation protocols. Experimental results demonstrate that ViT-DD outperforms state-of-the-art methods. On the SFDDD dataset using the challenging split-by-driver setting, ViT-DD achieved a 6.5% accuracy improvement over the best existing method (DD-RCNN). On the AUCDD dataset, it surpassed the previous best method (C-SLSTM) by 0.9%. Ablation studies confirmed that incorporating emotion recognition via the proposed self-training strategy improved average accuracy by 2.2% on SFDDD and 2.7% on AUCDD compared to a standard supervised ViT. The model showed particular strength in distinguishing "safe driving" and "talking to passenger" behaviors, which correlate strongly with neutral and happy emotions, respectively. However, performance slightly decreased for behaviors like "phone left" and "reaching behind," where emotion cues are less informative or potentially misleading. Attention map visualizations revealed that the model effectively focuses on critical local cues, such as the steering wheel or phone, as the network depth increases. The significance of this work lies in demonstrating that a pure Transformer architecture, combined with semi-supervised multi-task learning, can significantly enhance driver distraction detection without requiring fully labeled multi-modal datasets. By leveraging emotion recognition as an auxiliary task, ViT-DD improves generalization, especially in realistic split-by-driver scenarios. This approach offers a scalable solution for intelligent vehicle systems, suggesting that future models could further integrate other in-cabin signals, such as gaze and head pose tracking, to bolster safety monitoring capabilities.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | Crossref | — | — | 1 | 2026-06-08 |
| archive | success | canonical_url | — | — | 7 | 2026-06-09 |
| extract | success | cached | — | — | 2 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-09 |
| chunk | success | chunk | — | — | 1 | 2026-06-09 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-09 |
| enrich | success | semantic_scholar | — | — | 1 | 2026-06-10 |
| promote | success | — | — | — | 1 | 2026-06-08 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 8 | 2026-06-11 |
| verify | success | — | — | — | 1 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: tool software
- Theoretical Contribution: conceptual framework