An Interpretable Audio Intelligence Framework for Driver Vigilance Using WavLM-Based Deep Representations

Kola Rishika; Dayakar Thalla; Kiran Gadapaka; Emmadi Swathi; Lellela Vishnu; Jannu Sushman · 2026 · Crossref

DOI: 10.64751/tkhpt493

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the limitations of current autonomous vehicle systems, which predominantly rely on visual sensors like cameras, LiDAR, and RADAR, thereby lacking the ability to detect critical auditory events. The authors propose an interpretable audio intelligence framework for driver vigilance and vehicle safety monitoring using Acoustic Event Detection (AED). The research is motivated by the reliability of acoustic signals in adverse weather and occlusion scenarios, aiming to complement existing sensor suites with polyphonic audio analysis capable of detecting multiple concurrent sound events within the in-vehicle environment. The methodology employs a deep learning pipeline centered on the WavLM transformer-based model for feature extraction. Raw in-vehicle audio recordings are preprocessed, resampled to 16 kHz, and converted into high-dimensional feature vectors that capture temporal and spectral dependencies. These embeddings serve as inputs for several classification models: CatBoost Classifier (CBC), Histogram Gradient Boosting Classifier (HGBC), Extra Trees Classifier (ETC), and a proposed Tree-Based Generalized Additive Model (TGAM). The system supports both main-class and sub-class classification to identify specific driver-related events and vehicle conditions. A graphical user interface facilitates dataset management, model training, and real-time prediction visualization. Experimental results demonstrate that the proposed TGAM significantly outperforms baseline models. For main-class classification, TGAM achieved 99.92% accuracy, 99.91% precision, 99.92% recall, and a Micro-AUC of 1.0000. In contrast, CBC achieved 91.67% accuracy, while HGBC and ETC performed substantially lower at 69.17% and 42.25%, respectively. For sub-class classification, TGAM maintained superior performance with 99.58% accuracy and 99.41% F1-score, whereas HGBC and ETC dropped to 23.25% and 24.42% accuracy. Confusion matrices and ROC curves confirmed near-perfect discrimination between audio categories, with minimal misclassification. The study concludes that integrating WavLM-based deep feature extraction with the TGAM classifier provides a robust and highly accurate solution for in-vehicle audio event detection. This approach enhances the reliability of intelligent transportation systems by enabling precise monitoring of driver alertness and vehicle states through acoustic cues. The framework’s high interpretability and performance suggest significant potential for improving driver safety and situational awareness in autonomous and semi-autonomous driving contexts.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• drowsiness detection algorithms
• auditory

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Theoretical Contribution: computational model