Multimodal Assessment of Pilot Cognitive Workload Using ECG and Eye-Tracking Features in Simulated Flight Tasks

Guan, Yixuan; Han, Longzhu; Zhou, Pengyan; Bao, Jiayi · 2026 · Crossref

DOI: 10.54941/ahfe1007396

archive: archived pipeline: cataloged

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the challenge of objectively assessing pilot cognitive workload in realistic flight environments, where traditional electroencephalography (EEG) is often impractical due to susceptibility to motion artifacts and environmental interference. The research proposes a multimodal assessment method utilizing electrocardiography (ECG) and eye-tracking signals, which offer higher reliability for real-world application. The primary objective was to develop a robust classification model capable of distinguishing between low, medium, and high cognitive workload levels during simulated flight tasks. The experimental design involved 30 male pilot trainees (mean age 23.0 years) performing three distinct flight tasks in the DCS World simulator using a Su-27 model. The tasks were designed to induce graduated workload levels: a low-difficulty visual cruise, a medium-difficulty standard traffic pattern, and a high-difficulty instrument flight in adverse weather. Participants completed the Overall Workload Scale (OWL) and NASA Task Load Index (NASA-TLX) after each task. Physiological data were collected continuously using Tobii Pro Glasses 3 (100 Hz) for eye-tracking and a BIOPAC MP160 system (500 Hz) for ECG. Feature extraction employed 1-minute sliding windows, with time-domain, frequency-domain, and nonlinear metrics derived from ECG, and fixation, saccade, blink, and pupil metrics from eye-tracking. Results confirmed that subjective workload ratings increased significantly with task difficulty. Physiological analysis revealed distinct trends: ECG metrics showed a significant increase in heart rate and decreases in heart rate variability indices (Mean NN, RMSSD, LF, HF, TP, and Lempel-Ziv complexity), indicating sympathetic dominance. Eye-tracking metrics exhibited increased fixation duration, blink interval, and pupil diameter, alongside decreased saccade duration and blink frequency. After removing redundant features via Spearman correlation analysis (threshold > 0.8), eleven key features were retained: six from ECG and five from eye-tracking. A stacking ensemble learning framework was constructed using decision-level fusion, employing Random Forest for the ECG modality and K-Nearest Neighbors for the eye-tracking modality, with Logistic Regression as the meta-learner. This multimodal model achieved a classification accuracy of 0.959, outperforming single-modality models where Random Forest alone yielded accuracies of 0.889 for ECG and 0.707 for eye-tracking. The findings demonstrate that fusing ECG and ocular metrics provides a sensitive and valid method for monitoring pilot cognitive workload without the limitations of EEG. The superior performance of the heterogeneous ensemble model highlights the complementary nature of autonomic and visual attention signals. This approach offers a viable technical foundation for developing real-time physiological monitoring and early-warning systems in aviation, enhancing flight safety by objectively tracking pilot mental states during dynamic operations.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.8-27b-gittensor on 2026-08-23; verification: pending re-verification.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• workload measurement
• neuro workload indices

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data, self report data