Detection of driver manual distraction via image-based hand and ear recognition
DOI: 10.1016/j.aap.2020.105432
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study addresses the critical safety issue of driver manual distraction, a leading cause of fatal traffic accidents. The authors propose a novel, image-based algorithm to detect and classify specific distracting behaviors by analyzing the spatial relationship between a driver’s right hand and right ear. The motivation stems from the limitations of previous methods, which often relied on facial features requiring intrusive camera placement or failed to distinguish between similar manual tasks, such as talking on a phone versus using a touchscreen. By incorporating ear detection alongside hand tracking, the system aims to provide a more robust and efficient solution for real-time driver monitoring. The methodology employs a two-module deep learning framework. The first module utilizes the You Only Look Once (YOLO) object detection network to predict bounding boxes for the driver’s right hand and right ear from RGB images. This module outputs an 8-dimensional vector containing spatial coordinates and dimensions for both body parts. The second module is a six-layer multi-layer perceptron that takes this vector as input to classify the driver’s activity. Data were collected from twenty participants in a driving simulator performing five specific tasks: normal driving, talking on a cell phone, texting, drinking, using a touchscreen, and placing an object in a cup holder. A total of 106,677 frames were extracted and annotated, with data from ten participants used for training and the remaining ten for testing. The YOLO model was pre-trained using a public hand-tracking dataset to enhance generalization. The results demonstrate that the proposed framework achieves an overall F1-score of 0.74 for distraction detection. Specifically, the system achieved F1-scores of 0.84 for normal driving, 0.69 for touchscreen use, and 0.82 for talking on a phone. The inclusion of ear detection significantly improved the ability to distinguish phone calls from other hand-based activities, addressing a weakness in prior kinematic studies. The entire framework operates at 28 frames per second, making it suitable for real-time applications. This performance is comparable in accuracy to similar research but significantly more efficient than previous methods, such as faster-RCNN, which operated at only 0.09 frames per second. The significance of this work lies in its potential to improve driver monitoring systems by providing a non-intrusive, low-cost, and computationally efficient method for detecting manual distractions. By accurately classifying specific distracting behaviors rather than just detecting general inattention, the system can better inform co-pilot systems and safety interventions. The study highlights the value of combining hand and ear spatial data to resolve ambiguities in manual task recognition, offering a scalable prototype for future driver safety technologies.
Key finding
The proposed image-based algorithm achieved an overall F1-score of 0.74 for detecting manual distraction and ran at 28 frames per second, demonstrating comparable accuracy to existing methods with greater computational efficiency.
Methodology
simulator
Sample size: 20
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 2 | 2026-05-28 |
| archive | success | canonical_url | — | — | 1 | 2026-06-06 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-07 |
| chunk | success | chunk | — | — | 1 | 2026-06-07 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-07 |
| enrich | success | semantic_scholar | — | — | 4 | 2026-06-15 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | partial | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified_with_issues.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
- distraction detection algorithms
- manual
- visual manual
- drowsiness detection algorithms
- visual
- gaze based attention detection
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: tool software
- Theoretical Contribution: conceptual framework, computational model