Evaluation of Subjective and EEG-Based Measures of Mental Workload

Funke, Gregory; Knott, Benjamin; Mancuso, Vincent F.; Strang, Adam; Estepp, Justin; Menke, Lauren; Brown, Rebecca; Dukes, Allen; Miller, Brent · 2013 · Crossref

DOI: 10.1007/978-3-642-39473-7_82

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the need for a comprehensive comparison of mental workload assessment methodologies, specifically examining the sensitivity, convergent validity, and concurrent validity of subjective self-report measures versus EEG-based physiological metrics. While various workload measures exist, prior research had not simultaneously evaluated this specific combination of instruments within a single experimental framework. The authors aimed to determine how these measures correlate with each other and with task performance, thereby aiding human factors practitioners in selecting appropriate tools for varied applications. The experiment involved 25 participants who performed the Air Force Space Shooter (AFSS) simulation, a variant of the game Asteroids. Task difficulty was manipulated by varying the number of asteroids on screen across easy (10), medium (15), and hard (20) conditions. Four subjective workload instruments were employed: the NASA Task Load Index (TLX), Multiple Resource Questionnaire (MRQ), Workload Profile (WP), and Subjective Workload Dominance Technique (SWORD). Simultaneously, EEG data were collected using a wireless headset, analyzing theta, alpha, and gamma band powers at specific electrode sites (Fz, POz, P3) and calculating an EEG-based task engagement metric. Performance was quantified by the ratio of asteroids destroyed to total asteroids presented. Results indicated that the task difficulty manipulation was effective, with performance scores decreasing linearly as difficulty increased. Regarding sensitivity, most measures responded to changes in task difficulty; however, subjective measures generally demonstrated greater sensitivity than EEG metrics, with the exceptions of P3 gamma power and EEG task engagement, which showed comparable responsiveness. In terms of convergent validity, correlations between measures were generally modest. Subjective multidimensional measures (TLX, WP, MRQ) showed moderate positive correlations with each other, while EEG measures also displayed some internal convergence. Notably, there was poor convergence between subjective and EEG measures, suggesting they may index different aspects of workload, such as aggregate versus moment-to-moment changes. Concurrent validity, measured by correlations with performance scores, was generally low across all instruments. The WP and SWORD subjective measures, along with POz alpha power, showed the strongest, albeit modest, relationships with performance. The study concludes that while most workload measures are sensitive to task difficulty, their validity in predicting performance or correlating with one another is limited. The findings support the utility of the Workload Profile and, to a lesser extent, the SWORD for subjective assessment, and highlight P3 gamma power and EEG task engagement as favorable physiological indicators. The lack of strong correlation between subjective and physiological measures suggests they capture distinct dimensions of mental workload, implying that practitioners should select measures based on specific application needs rather than expecting high interchangeability.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• neuro workload indices
• workload measurement
• mental demand

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data, self report data
• Methodological Resource: validation psychometrics