DrowSFormer-XAI: A Cross-Attentive Neural Network-Transformer Fusion Framework for Drowsiness Detection with Explainable AI

Rajalingam, Kathirvel; Srinivasan, Saravanan; Govindaraju, Sakthi; Mathivanan, Sandeep Kumar; Ramaswamy, Sangeetha; Moorthy, Usha · 2026 · OpenAlex-citations

DOI: 10.1007/s11063-026-11841-6

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the critical need for reliable, non-invasive drowsiness detection systems to enhance safety in high-stakes environments such as automotive driving, aviation, and healthcare. While physiological methods offer high accuracy, they are invasive and computationally expensive, whereas existing behavioral deep learning models often rely on either convolutional neural networks (ConvNets) or Transformers alone, limiting their ability to capture both fine-grained spatial details and global contextual relationships. Furthermore, the lack of interpretability in these "black box" models hinders trust in safety-critical applications. To address these gaps, the authors propose DrowSFormer-XAI, a novel hybrid architecture that fuses ConvNet and Transformer branches via a cross-attention mechanism, integrated with Explainable AI (XAI) tools. The study utilizes an integrated dataset (DD-I) comprising 49,793 images from the Driver Drowsiness Dataset (DDD) and the MRL Eye Dataset, categorized into four classes: drowsy, non-drowsy, closed eyes, and open eyes. Images were resized to 224×224 pixels, normalized, and augmented using rotation, flipping, brightness variation, zoom, and translation to mitigate class imbalance and improve generalization. The dataset was split into 70% training, 15% validation, and 15% testing subsets, with tenfold cross-validation applied. The proposed DrowSFormer was evaluated against four baseline architectures: ConvNeXt-Base, ResNeSt101, CaiT-S36, and Twins-SVT-Base. All models were trained for 50 epochs with a batch size of 64. The DrowSFormer architecture employs a dual-branch design where the ConvNet branch extracts high-resolution spatial features and the Transformer branch captures long-range dependencies, fused dynamically through cross-attention. Additionally, Gradient-weighted Class Activation Mapping (Grad-CAM) was implemented to visualize the model’s decision-making process. Experimental results demonstrate that the proposed DrowSFormer model achieves a testing accuracy of 99.58%, significantly outperforming all baseline and state-of-the-art models, including both transformer-based and ConvNet-based architectures. The model exhibited outstanding performance across all evaluation metrics, confirming its superior capability in distinguishing drowsy from alert states under varying lighting and orientation conditions. The integration of Grad-CAM provided visual heatmaps highlighting the specific eye regions contributing to predictions, thereby offering transparency into the model’s logic. The significance of this work lies in its demonstration that hybrid neural network-transformer architectures can effectively combine local spatial feature extraction with global contextual reasoning for standalone drowsiness detection. By achieving near-perfect accuracy while providing interpretable outputs via XAI, DrowSFormer-XAI addresses the dual challenges of performance and trustworthiness. This framework offers a lightweight, robust solution suitable for real-time deployment in driver monitoring systems and other safety-critical domains, potentially reducing accidents caused by fatigue and enabling early intervention for sleep disorders.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• drowsiness detection algorithms