On Time Series Cross-Validation for Deep Learning Classification Model of Mental Workload Levels Based on EEG Signals

Kingphai, Kunjira; Moshfeghi, Yashar · 2023 · Crossref

DOI: 10.1007/978-3-031-25891-6_30

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the methodological flaw of applying traditional cross-validation (CV) techniques to electroencephalogram (EEG) time series data for mental workload (MWL) classification. Traditional CV methods, which randomly shuffle and split data, violate the independent and identically distributed (i.i.d.) assumption inherent in time series, potentially causing data leakage and overfitting. The authors propose and evaluate blocked time series cross-validation (TSCV) strategies—specifically expanding window and rolling window approaches—to better respect the temporal nature of EEG signals. The study aims to determine which TSCV strategy yields more robust models and how training block sizes affect performance. The researchers utilized the Simultaneous Task EEG Workload (STEW) dataset, comprising EEG signals from 48 subjects recorded across 14 electrodes. They performed two classification tasks: distinguishing between resting and testing states (Task 1) and classifying MWL into low, moderate, and high levels (Task 2). Preprocessing involved artifact removal using ICA-ADJUST and feature extraction from time, frequency, linear, and non-linear domains, resulting in 84 features per subject. These features were standardized to mitigate inter-subject variability. The study evaluated seven deep learning architectures, including stacked LSTM, BLSTM, stacked GRU, BGRU, BGRU-GRU, BLSTM-LSTM, and CNN models. The models were trained using 5-fold TSCV with varying training block sizes (20% to 90% of the data) under both expanding and rolling window strategies. Performance was measured using accuracy, sensitivity, specificity, precision, false acceptance rate, and false rejection rate. The results demonstrated that the expanding window strategy consistently outperformed the rolling window strategy in terms of accuracy and stability. For Task 1, the expanding window strategy achieved a maximum accuracy of 93.53% when trained on 90% of the data, compared to 91.78% for the rolling window strategy. Similarly, for Task 2, the expanding window strategy yielded higher accuracy (81.38% vs. 80.44% at 90% training size). Furthermore, models trained with the expanding window strategy exhibited lower standard deviations in accuracy, indicating greater robustness against temporal variance in the data. Among the deep learning architectures, the BGRU-GRU model achieved the highest performance for both tasks, recording 95.90% accuracy for Task 1 and 82.45% for Task 2 when evaluated with the expanding window strategy and 90% training data. The study concludes that blocked TSCV with an expanding window strategy is superior for evaluating deep learning models on EEG-based MWL classification. This approach prevents data leakage by ensuring training data precedes test data chronologically and leverages all available historical data, leading to more reliable and generalizable models. The findings highlight the importance of respecting temporal dependencies in physiological signal processing and provide a validated framework for future research in brain-computer interfaces and mental state monitoring.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• drowsiness detection algorithms