Multi-camera Face Tracking for Estimating User’s Discomfort with Automated Vehicle Operations
DOI: 10.54941/ahfe1001104
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study investigates the potential of automated facial expression analysis as an unobtrusive sensor technology for monitoring user discomfort during automated vehicle operations. The research is motivated by the need to improve human-machine interaction in automated driving, where continuous comfort monitoring can prevent safety-critical driver interventions and enhance acceptance. Specifically, the authors aimed to determine if changes in Facial Action Units (AUs) could reliably indicate subjective discomfort in response to specific automated driving maneuvers, such as close approach situations. The methodology involved analyzing video data from two driving simulator studies comprising a gender-balanced sample of 81 participants aged 24 to 84. Participants experienced a standardized, highly automated drive featuring three consecutive close-approach scenarios involving a slower truck ahead. To ground-truth subjective discomfort, participants used a handset control to continuously report their perceived discomfort levels. Facial data was captured using multiple video cameras (two in Study 1, four in Study 2) and processed using Visage facial feature detection software (version 8.7). The researchers extracted tracking values for 23 AUs, filtering for quality and applying z-transformation to normalize data across cameras and participants. Discomfort sequences were defined as the period from 10 seconds before to 10 seconds after the participant pressed the discomfort control, allowing for the analysis of temporal changes in facial expressions relative to the event. The results identified specific, situation-related facial patterns associated with discomfort. Analysis of aggregated data revealed a decrease in face scale, indicating a pushback movement of the head and upper body. Eye-related AUs showed reduced eye closure (fewer blinks) during the approach, followed by a rise in the inner brow raiser (AU1) and upper lid raiser (AU5) immediately after the event, patterns interpreted as visual attention and surprise. Mouth-related AUs demonstrated increased lip pressing (AU24, AU15) and stretching (AU20), which the authors interpreted as signs of tension. These findings were consistent across the combined dataset of 428 valid sequences, confirming that facial expressions change systematically in response to uncomfortable automated driving scenarios. The study concludes that automated facial expression analysis holds significant potential for estimating user comfort in automated vehicles, offering a non-intrusive method to detect states like tension, surprise, and attention. However, the authors emphasize that while effects are clear at the aggregated data level, obtaining stable and reliable results at the individual level remains challenging due to variability in facial expressivity and latency jitter. The findings suggest that while face tracking can contribute valuable information for adaptive vehicle systems, further validation across different scenarios and the development of machine learning algorithms are necessary to address individual variability and improve real-time detection accuracy.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | Crossref | — | — | 1 | 2026-06-06 |
| archive | success | canonical_url | — | — | 1 | 2026-06-09 |
| extract | success | cached | — | — | 2 | 2026-06-09 |
| clean | success | clean | — | — | 1 | 2026-06-07 |
| chunk | success | chunk | — | — | 1 | 2026-06-07 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-07 |
| promote | success | — | — | — | 1 | 2026-06-06 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-09 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 1 | 2026-06-09 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-09; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: physiological data
- Methodological Resource: tool software