Prefrontal Correlates of Passengers’ Mental Activity Based on fNIRS for High-Level Automated Vehicles

Zhang, Xiaofei; Li, Chuzhao; Li, Jun; Cao, Bin; Fu, Junwen; Wang, Qiaoya; Wang, Hong · 2024 · Crossref

DOI: 10.1007/s42154-023-00252-1

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the safety of the intended functionality (SOTIF) in high-level automated vehicles, specifically focusing on overcoming perception algorithm deficiencies by leveraging passengers as human sensors. While previous research has extensively analyzed driver mental activity, this paper investigates the prefrontal correlates of passenger mental activity during risk scenarios. The authors posit that monitoring passenger physiological states via functional near-infrared spectroscopy (fNIRS) can provide critical data for improving vehicle safety systems, particularly in cut-in scenarios where perception failures may lead to accidents. The methodology involved two distinct experiments: one using a driving simulator and another using a real-world vehicle on a dedicated test track. Participants wore an OctaMon+ fNIRS device to monitor cerebral oxygenation in the prefrontal cortex. The researchers defined risk levels using a kinetic energy field metric, dividing scenarios into low-risk and high-risk segments. In the simulation experiment, 20 participants experienced a cut-in scenario where a target vehicle merged into the ego vehicle’s path. In the real-world experiment, a single participant underwent nine tests involving a Global Vehicle Target (GVT) performing cut-in maneuvers. Data processing involved aligning kinetic energy field metrics with changes in total hemoglobin concentration ($\Delta TH$), a proxy for mental activity. Statistical analyses, including t-tests and Wilcoxon Signed Rank Tests, were employed to compare $\Delta TH$ values between low-risk and high-risk segments across five time windows. The results demonstrated a statistically significant difference in prefrontal cortex mental activity between low-risk and high-risk segments in both experimental settings. In the simulation experiment, t-test p-values for $\Delta TH$ differences were consistently below 0.1 across all time windows, indicating that perceived risk significantly altered passenger mental states. Similarly, the real-world vehicle experiment yielded p-values below 0.1 using the Wilcoxon Signed Rank Test, confirming that the kinetic energy field effectively delineated risk segments that corresponded to measurable changes in cerebral oxygenation. The mean values of the kinetic energy field were consistently higher in high-risk segments, validating the risk classification method. The study concludes that risk exposure significantly alters passengers’ mental activity in the prefrontal cortex, detectable via fNIRS. This finding suggests that passengers can serve as effective sensors for automated vehicles. By integrating passenger physiological states into decision-making loops, potentially through brain-computer interfaces or reinforcement learning techniques, manufacturers can mitigate functional deficiencies in perception algorithms. This approach offers a viable pathway to enhance SOTIF, thereby improving the safety and reliability of high-level automated vehicles.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness
• passenger motion sickness comfort
• neuro workload indices

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data
• Methodological Resource: tool software
• Theoretical Contribution: computational model