Combining Electroencephalograph and Functional Near Infrared Spectroscopy to Explore Users’ Mental Workload

Hirshfield, Leanne M.; Chauncey, Krysta; Gulotta, Rebecca; Girouard, Audrey; Solovey, Erin T.; Jacob, Robert J. K.; Sassaroli, Angelo; Fantini, Sergio · 2009 · Crossref

DOI: 10.1007/978-3-642-02812-0_28

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of objectively measuring users' mental workload (WL) in Human-Computer Interaction (HCI) to improve usability testing and enable adaptive user interfaces. The authors propose combining Electroencephalography (EEG) and functional Near-Infrared Spectroscopy (fNIRS) to leverage their complementary strengths. EEG offers high temporal resolution but suffers from low spatial resolution and susceptibility to motion artifacts, while fNIRS provides better spatial localization of frontal lobe activity related to cognitive load but has lower temporal resolution. The study aims to guide researchers on the technical integration of these devices and evaluate their efficacy in classifying WL states using machine learning. The experimental design involved four undergraduate participants performing a working memory task with three conditions: low WL (tracking two planes per row), high WL (tracking six planes per row), and random WL (varying between two and six planes). Data were collected concurrently using a 32-channel EEG cap and an fNIRS device with probes placed on the forehead. Due to hardware limitations, synchronization was approximated via simultaneous button presses. The authors applied distinct machine learning techniques to each data stream: fNIRS data were processed using a weighted k-nearest-neighbor classifier with Symbolic Aggregate Approximation for dimensionality reduction, while EEG data were analyzed using feature extraction (spectral power, coherence) followed by a Naïve Bayes classifier. The results indicated that fNIRS was effective in distinguishing between WL states, achieving classification accuracies up to 82% for two-class distinctions and up to 50% for three-class distinctions. In contrast, EEG classification performance was poor, yielding nearly random accuracy for three of the four subjects, with only one subject showing moderate success. The authors attribute the low EEG performance to potential noise introduced by fNIRS light sources or the task's specific activation of the prefrontal cortex, which was primarily monitored by the fNIRS sensors. The study concludes that while fNIRS shows promise for measuring mental workload in HCI contexts, the practical combination of EEG and fNIRS requires further refinement. The authors suggest that future research should investigate tasks that activate broader brain regions beyond the prefrontal cortex to better utilize the complementary nature of both devices. The paper serves as a foundational guide for the technical and analytical challenges of concurrent multi-modal brain imaging in HCI.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• neuro workload indices

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data