Systematic Review of Multimodal Fusion Strategies for Driver Fatigue Detection in Autonomous Vehicles

Huang, Xiang · 2025 · Crossref

DOI: 10.1051/itmconf/20257803014

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This systematic review addresses the critical safety challenge of driver fatigue in autonomous vehicles, which contributes to approximately 20% of global traffic accidents. The study is motivated by the limitations of traditional unimodal detection methods: vision-based approaches are susceptible to environmental interference such as lighting changes and occlusions, while electroencephalogram (EEG) signals suffer from noise, motion artifacts, and poor cross-subject generalization. To overcome these blind spots, the paper analyzes multimodal fusion strategies that integrate visual cues, physiological signals, and vehicle operation data to enhance detection accuracy and robustness. The review categorizes fusion strategies into early and late fusion frameworks. Early fusion integrates low-level features, such as visual facial key points and EEG spectral features, at the initial processing stage. The authors detail a specific architecture using EfficientDet for visual feature extraction and EEGNet for physiological data, aligning temporal resolutions via linear interpolation and downsampling. This approach employs joint representation learning, compressing features through global average pooling and fully connected layers, followed by classification using convolutional neural networks with batch normalization and dropout regularization. Late fusion, conversely, processes modalities independently and combines high-level predictions at the decision stage. The paper examines three late fusion strategies: weighted voting, which dynamically adjusts modality weights based on environmental sensors; D-S evidence theory, which quantifies conflict between modalities to suppress low-confidence inputs; and probabilistic averaging, which smooths random noise. Experimental results demonstrate that multimodal fusion significantly outperforms unimodal baselines. Early fusion achieves 97.8% accuracy in stable daytime conditions and 96.05% in occluded scenarios, with an inference delay of 18 milliseconds per frame. Late fusion proves more robust in dynamic environments, reducing false alarm rates by 18% in scenarios involving occlusions or low light by prioritizing reliable modalities. Comparative analysis shows that while early fusion excels in synchronized settings, late fusion offers superior adaptability to heterogeneous environments. Transformer-based architectures further bridge performance gaps, achieving 98.1% accuracy in ideal conditions through cross-modal attention mechanisms. However, computational costs remain a trade-off, with early fusion requiring higher GFLOPs due to feature alignment. The significance of this work lies in its guidance for deploying automotive-grade fatigue detection systems. The authors identify key challenges in real-world deployment, including real-time latency constraints, hardware limitations on edge devices, and the performance drop observed when transitioning from laboratory to field tests. Future directions emphasize privacy-preserving federated learning to handle sensitive biometric data, lightweight Transformer-based attention mechanisms for adaptive cross-modal interaction, and hardware-software co-design involving model compression and neural processing unit acceleration. By bridging laboratory prototypes with real-world needs, this review lays the groundwork for holistic fatigue intervention systems essential for L3/L4 autonomous driving safety.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• drowsiness detection algorithms
• distraction detection algorithms

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data
• Methodological Resource: tool software