Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators

Walocha, Fabian; Schrank, Andreas; Nguyen, Hoai Phuong; Ihme, Klas · 2025 · Crossref

DOI: 10.3390/info16010064

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the critical human-factor challenge of monitoring mental workload in remote operators assisting highly automated vehicles (HAVs). As SAE Level 4 autonomous vehicles increasingly rely on remote human intervention for complex scenarios, operators face high cognitive demands that can lead to performance degradation if workload is not managed. The research aims to identify non-invasive, multimodal physiological and behavioral indicators—specifically eye-tracking, skin conductance, and cardiovascular metrics—that can differentiate between low, medium, and high workload levels in real-time. This capability is essential for developing workload-adaptive interfaces that optimize operator safety and efficiency. The researchers conducted a controlled user study with 37 participants using a dual-task experimental design. The primary task involved solving three prototypical remote assistance scenarios (e.g., obstacle clearance, rerouting) using a simulated operation center with multiple screens. Mental workload was manipulated via a secondary auditory n-back task with varying difficulty levels: no secondary task (low workload), 1-back (medium workload), and 2-back (high workload). Data were collected using an infrared eye-tracking system, electrodermal activity (EDA) sensors, and electrocardiogram (ECG) monitors. The study extracted specific indicators including pupil diameter, fixation duration, fixation dispersion, eyelid opening, blink rate, tonic skin conductance level, heart rate, and heart rate variability. Statistical analysis involved MANOVA and post-hoc ANOVAs to identify significant indicators, followed by machine learning modeling using XGBoost for multi-class classification of workload states. The results confirmed that the secondary task successfully induced varying levels of mental workload, evidenced by increased subjective NASA-TLX ratings and degraded primary task performance under high workload conditions. Physiologically, the study found significant differences across workload levels in tonic skin conductance ($F(2,72) = 24.538, p < 0.001$) and pupil dilation ($F(2,72) = 13.872, p < 0.001$). These two indicators were the most robust predictors of workload state. Using these features, the XGBoost model achieved a classification accuracy of 58% in distinguishing between low, medium, and high workload conditions. Other indicators, such as heart rate and blink rate, did not show the same level of discriminative power in this specific context. The findings demonstrate that mental workload during HAV remote assistance can be objectively assessed using non-invasive physiological sensors, particularly through changes in pupil dilation and skin conductance. While the classification accuracy of 58% indicates room for improvement, the study provides foundational evidence that multimodal assessment is feasible for differentiating workload levels in a time-resolved manner. This supports the development of adaptive human-machine interfaces that can dynamically adjust task allocation and information presentation based on the operator’s real-time cognitive state, thereby enhancing the safety and reliability of remote assistance systems for automated vehicles.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• workload measurement
• mental demand
• teleoperation remote driving
• neuro workload indices
• drowsiness detection algorithms
• stress driving

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data, self report data
• Theoretical Contribution: theory or model