Multimodal Learnability Assessment of a Touch-based Large Area Display with Eye Tracking and Optical Brain Imaging

Çigdem Şahin, Ezgi; Diğba Özmenoğlu, Dilan; Paker, Ahmet; Kaygusuz, Yasin; Aydemir, Hakan; Acartürk, Cengiz; Perit Çakır, Murat · 2023 · Crossref

DOI: 10.54941/ahfe1003007

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study investigates the learnability of touch-based Large-Area Displays (LADs) in modern aircraft cockpits, addressing the challenge that while LADs reduce physical clutter, their novel interaction paradigms may hinder effective pilot performance. The research aims to assess how pilots adapt to these interfaces by measuring behavioral performance, eye movements, and brain activity, thereby providing neuroergonomic evidence for interface usability. The experimental design involved two test pilots with experience in traditional cockpit layouts but no prior exposure to the specific LAD interface. Participants performed flight tasks in a HURJET flight simulator, including functional layout assignment, communication parameter settings, and monitoring of barometric and fuel levels across various flight phases. Data collection occurred during an initial session and a replication session two weeks later. Multimodal data were captured using a Pupil Labs Invisible mobile eye tracker (120 Hz) to record binocular eye movements and an fNIR Imager 1100 system to monitor prefrontal oxygenation changes via functional near-infrared spectroscopy (fNIRS). The fNIRS system measured oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) concentrations at 16 locations over the prefrontal cortex, specifically targeting Brodmann areas 9, 10, 44, and 45. Results demonstrated significant improvements in learnability between the two sessions. Behaviorally, task completion times decreased significantly for the majority of subtasks; for example, the portal setting task average time dropped from 124.8 seconds to 97.5 seconds. Eye-tracking analysis revealed that pilots exhibited longer mean fixation durations and fewer total fixations on target areas of interest in the second session, indicating more efficient visual attention allocation. Gaze accuracy remained high (>90%) in both sessions. Neurophysiologically, fNIRS data showed a significant decrease in HbO responses in the right dorsolateral and right superior prefrontal cortex (optodes 11–16) during the second flight. Statistical tests confirmed these decreases with high effect sizes (e.g., Cohen’s d ranging from 1.15 to 1.68 for specific optodes), suggesting reduced cognitive effort and neural activation associated with deliberate visual attention guidance. The findings provide supporting evidence that pilots can rapidly develop sensorimotor adaptations to LAD-mediated cockpit environments. The convergence of reduced task times, optimized eye movement patterns, and decreased prefrontal activation indicates that the interface is learnable and that pilots successfully internalized the functional layout and touch gestures. This study highlights the utility of multimodal neuroergonomic assessments—combining behavioral, ocular, and neural metrics—for evaluating the learnability of mission-critical human-machine interfaces, offering a robust framework for future cockpit design and usability testing.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• neuro workload indices