Functional near-infrared spectroscopy (fNIRS) and Eye tracking for Cognitive Load classification in a Driving Simulator Using Deep Learning
archive: archived pipeline: cataloged verified
Get this paper ↗ (full text — opens at the source; we link to it, we don't host it)
Summary
This study addresses the gap in cognitive load assessment during driving, particularly under challenging low-visibility conditions such as nighttime rain. While driving simulators allow for safe investigation of driver-vehicle interactions, previous research often relied on subjective measures or limited cognitive load levels (typically binary) in bright daylight. This work aims to objectively classify three levels of cognitive load (baseline, 0-back, 1-back, and 2-back tasks) using a hybrid deep learning approach that integrates physiological data with vehicle dynamics. The experiment involved 10 healthy adult drivers performing simulated nighttime driving in rainy conditions using a motion platform and *Euro Truck Simulator 2*. Cognitive load was induced via an auditory n-back task. Data collection utilized functional near-infrared spectroscopy (fNIRS) to measure prefrontal cortex hemodynamic responses (oxyhaemoglobin and deoxyhaemoglobin), Pupil Core glasses for eye-tracking (fixation duration and gaze direction), and the simulator’s SDK for vehicle dynamics (speed, acceleration, steering, etc.). The methodology employed a Convolutional Neural Network (CNN) for feature extraction from time-series data, followed by Long Short-Term Memory (LSTM) layers to capture temporal dependencies. Feature selection was performed using ANOVA, identifying car speed as the most significant driving parameter and oxyhaemoglobin features as the most influential physiological indicators. The results demonstrated that the proposed CNN-LSTM model significantly outperformed traditional machine learning classifiers (such as Naïve Bayes, k-NN, and Decision Trees). When using only vehicle dynamics data, the model achieved 89.6% accuracy in classifying cognitive load levels. However, fusing vehicle dynamics with fNIRS and eye-tracking data substantially improved performance, achieving 99.82% accuracy, 99.67% precision, and 99.67% recall. The integration of multimodal data reduced misclassifications and highlighted the superior capability of deep learning models to handle complex, sequential physiological and behavioral data compared to classical algorithms. The significance of this research lies in its demonstration that high-accuracy, real-time cognitive load assessment is feasible using multimodal data fusion, even in adverse weather conditions. The findings suggest that adaptive vehicle systems could leverage such models to monitor driver mental states and intervene during periods of high cognitive demand, thereby enhancing road safety. While the study acknowledges limitations regarding sample size and the practical complexity of deploying fNIRS and eye-tracking hardware, it establishes a robust framework for future development of intelligent driver monitoring systems.
Key finding
A CNN-LSTM model fusing fNIRS and eye-tracking achieved 99% cognitive-load classification accuracy in nighttime/rainy simulated driving, outperforming vehicle-dynamics-only features (89%).
Methodology
simulator
Sample size: N=10 (9 male, 1 female)
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via discover_arxiv on 2026-05-04 (3 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | arxiv | — | — | 3 | 2026-05-04 |
| archive | success | — | — | — | 1 | 2026-05-04 |
| extract | success | cached | — | — | 2 | 2026-06-10 |
| clean | success | — | — | — | 1 | 2026-06-01 |
| chunk | success | — | — | — | 1 | 2026-06-01 |
| embed | success | — | — | — | 1 | 2026-06-02 |
| enrich | success | — | — | — | 1 | 2026-05-04 |
| promote | success | — | — | — | 1 | 2026-05-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 16 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: physiological data
- Methodological Resource: tool software
- Theoretical Contribution: computational model