Driver Monitoring System Using Computer Vision for Real-Time Detection of Fatigue, Distraction and Emotion via Facial Landmarks and Deep Learning.

Zambrano, Tamia; Arias, Luis; Haro, Edgar; Santos, Victor; Trujillo-Guerrero, María · 2026 · PubMed Central (PMC)

DOI: 10.3390/s26030889

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the critical safety issue of driver fatigue and distraction, which contribute to approximately 25% of fatal traffic accidents in Ecuador. The authors developed a real-time, non-invasive Driver Monitoring System (DMS) that integrates computer vision and deep learning to detect drowsiness, distraction, and emotional states. The motivation stems from the limitations of existing systems, which often rely on intrusive physiological sensors or fail to account for emotional context, leading to false alarms. By combining facial landmark analysis with emotion recognition, the system aims to provide a more robust and context-aware monitoring solution for real-world driving environments. The system was implemented using a Python 3.10 script that processes video streams from a standard USB camera. It utilizes MediaPipe’s 468 facial landmarks to calculate the Eye Aspect Ratio (EAR) for eye closure and the Mouth Aspect Ratio (MAR) for yawning. Distraction is detected by analyzing head pose (pitch and yaw) and gaze direction. For emotion recognition, a MobileNetV2-based Convolutional Neural Network (CNN) was fine-tuned on the RAF-DB dataset to classify seven emotional states. The experimental evaluation involved 27 adult participants in both simulated and real driving conditions, including low-light scenarios. The system employed fixed thresholds and temporal constraints to reduce false positives caused by natural movements or lighting changes. The results demonstrated high efficacy in detecting distraction and moderate success in fatigue detection. The system achieved 100% accuracy in identifying distraction behaviors, including gaze deviations to the left, right, up, and down. Fatigue detection yielded an accuracy of 88.89% for eye closure and 85.19% for yawning. Emotion recognition performed well for happiness (100%), anger/disgust (96.3%), and surprise (92.6%). However, the system struggled with sadness (66.7% accuracy) and failed to detect fear (0% accuracy), attributed to the subtlety of real-world expressions and a mismatch between the training dataset and natural driving behaviors. Misclassifications in fatigue detection were observed in participants with smaller eyes, highlighting the limitations of fixed global thresholds. The significance of this work lies in its demonstration that lightweight, non-invasive computer vision systems can effectively monitor driver states in real time. The integration of emotion recognition provides valuable context, helping to distinguish between fatigue-induced eye closure and momentary squinting from laughter, thereby reducing false alarms. The study concludes that while the current system is effective for distraction and prominent fatigue signs, future iterations should incorporate adaptive thresholding calibrated to individual baselines and domain-specific training data for emotion recognition. These improvements would enhance the system's robustness against inter-subject variability and environmental challenges, supporting the development of smarter, adaptive vehicle safety systems.

Key finding

Combining MediaPipe facial landmarks (EAR/MAR/head-pose) with a MobileNetV2 emotion CNN achieved 100% distraction and ~85-89% drowsiness detection in real-time driving tests, though detection of subtle emotions (sadness, fear) remained poor.

Methodology

lab_experiment

Sample size: 27 participants tested across real and simulated driving environments.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via discover_europe_pmc on 2026-05-04 (3 acquisition events logged).

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified_with_issues.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• drowsiness detection algorithms
• distraction detection algorithms
• dms validation
• gaze based attention detection

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data
• Methodological Resource: tool software, validation psychometrics