Automated Vehicle Driver Monitoring Dataset from Real-World Scenarios
DOI: 10.1109/itsc58415.2024.10920048
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the critical safety challenge of monitoring driver behavior in SAE Level 3 and higher automated vehicles, where drivers may engage in secondary tasks while the system drives. The authors identify a significant gap in existing research: most driver monitoring datasets rely on simulated environments or stationary vehicles, lacking the dynamic real-world conditions (such as varying illumination, weather, and moving backgrounds) necessary for Deep Learning (DL) models to generalize effectively. To bridge this gap, the paper introduces the Johannes Kepler University-Intelligent Transport Systems (JKU-ITS) Automated Vehicle Driver Monitoring (AVDM) dataset, the first of its kind collected in a real-world automated driving scenario. The dataset was generated using a research vehicle equipped with automated driving capabilities via a drive-by-wire system. Data was collected from 17 participants who performed eight specific activities: manual driving, sitting still, using a mobile phone, talking on the phone, reading a magazine, reading a book, reading a newspaper, and drinking from a bottle. These tasks were performed while the vehicle navigated a secure test route under diverse weather and lighting conditions. Video data was captured using a Logitech C920 webcam positioned on the passenger-side A-pillar. The authors developed a semi-automatic labeling tool to annotate the videos, providing labels in two formats: a 6-class version (grouping reading activities) and an 8-class version (distinguishing between book, magazine, and newspaper). The dataset comprises 335,000 images and 200 minutes of video, with classes evenly distributed to prevent training bias. To establish a performance benchmark, the authors trained and evaluated an Inflated 3D (I3D) convolutional neural network on the dataset. The model was trained on data from 15 participants and tested on the remaining two. Quantitative results demonstrated high accuracy for most categories: the model achieved 100% accuracy for "driving," "sitting still," and "reading" classes. However, performance varied for other activities, with 83% accuracy for "talking on the phone" and 50% for both "drinking" and "using the phone." Qualitative analysis revealed that misclassifications often stemmed from visual ambiguities, such as hand proximity to the steering wheel confusing "using phone" with "driving," or occlusion of objects like bottles leading to confusion between "drinking" and "talking on the phone." The significance of this work lies in providing a robust, real-world benchmark for developing driver monitoring systems that can handle the complexities of actual driving environments. By demonstrating that DL models can achieve high accuracy on real-world data despite environmental variations, the paper validates the necessity of moving beyond simulated datasets. The authors conclude that while the current model performs well, future work should focus on expanding the dataset with more classes, diversifying cockpit configurations, and incorporating higher dynamic range sensors to further improve model robustness and generalization for accident prevention in automated driving.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | Crossref | — | — | 1 | 2026-06-25 |
| archive | success | semantic_scholar | — | — | 6 | 2026-06-26 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-26 |
| chunk | success | chunk | — | — | 1 | 2026-06-26 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-26 |
| enrich | success | openalex | — | — | 1 | 2026-06-26 |
| promote | success | — | — | — | 1 | 2026-06-25 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-26 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
- distraction detection algorithms
- situational awareness
- dms validation
- drowsiness detection algorithms
- in vehicle coaching
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: tool software, dataset resource
- Theoretical Contribution: conceptual framework