Driver State Modeling through Latent Variable State Space Framework in the Wild

Tavakoli, Arash; Boker, Steven; Heydarian, Arsalan · 2022 · arXiv

archive: archived pipeline: cataloged verified

Get this paper ↗ (full text — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of modeling driver states, specifically stress levels and workload, which are psychological constructs that cannot be directly measured. The authors argue that existing methods often fail to account for the simultaneous interaction of these constructs, their time-dependency, and the influence of external environmental contexts. To improve driver-vehicle interaction systems and safety, particularly for automated driving scenarios requiring timely takeovers, the study proposes a holistic framework that estimates these latent variables through multimodal sensor data while explicitly modeling their temporal dynamics and contextual perturbations. The researchers employed a latent-variable state-space modeling (SSM) framework to analyze naturalistic driving data collected from 11 participants using the HARMONY multimodal sensing platform. The data included physiological measures (heart rate via PPG), behavioral metrics (gaze transition entropy and facial action units via cameras), and environmental context (traffic density estimated via object detection and secondary task demands inferred from hand movement IMUs). The methodology involved extracting features such as heart rate variability, gaze entropy, and specific facial action units correlated with stress and workload. The authors compared two SSM configurations: a base model with a single latent variable representing psychophysiological state, and a two-latent-variable model distinguishing between stress and workload with an estimated covariance between them. Model fitness was assessed using log-likelihood metrics, and the analysis accounted for time dependency by comparing states at 10-second intervals. The results demonstrated that the two-latent-variable model consistently provided a better fit to the data than the single-latent-variable model across all participants, as evidenced by significantly lower -2 log-likelihood values. This indicates that stress and workload are distinct but interacting constructs rather than a single unified state. The study found that external contextual factors, such as traffic density and secondary task demands, were associated with changes in these latent states. Furthermore, the analysis revealed significant individual differences; for instance, in one participant, higher workload was associated with lower stress levels, while the nature of this association varied across other drivers. The models also confirmed that a driver’s latent states at previous timesteps were highly predictive of their current states, validating the utility of state-space models for capturing temporal dynamics. The significance of this work lies in its demonstration that state-space latent variable models can effectively unify internal psychophysiological states and external contextual inputs in a temporal framework. By distinguishing between stress and workload and modeling their interactions, the approach offers a more realistic representation of driver states in naturalistic settings. This method supports the development of human-centered driver-vehicle interaction systems by enabling more accurate detection of driver states, which is critical for safety in both manual and automated driving contexts. The findings also highlight the importance of accounting for individual differences and the lagged effects of environmental perturbations on driver psychophysiology.

Key finding

Latent variable state-space models successfully estimate driver stress and workload from multimodal sensor data. Different drivers are impacted differently by environmental perturbations, and previous latent states are highly associated with current states.

Methodology

naturalistic

Sample size: 11

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via discover_arxiv_cs.HC on 2026-05-04 (5 acquisition events logged).

Summary generated by qwen3.6-27b-prismaquant on 2026-06-07; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• stress driving
• situational awareness
• drowsiness detection algorithms
• workload measurement
• drowsiness
• mental demand

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: physiological data
• Theoretical Contribution: theory or model, computational model