Visual and Cognitive Demands of a Large Language Model-Powered In-vehicle Conversational Agent
DOI: 10.48550/arXiv.2601.15034
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study addresses the safety implications of deploying Large Language Model (LLM)-powered conversational agents in vehicles, specifically examining the visual and cognitive demands of Google’s Gemini Live. As driver distraction remains a leading cause of crashes, and existing guidelines lack specific standards for voice-based interfaces, the research aimed to determine if advanced AI interactions impose risks comparable to established low-risk benchmarks or exceed them due to increased cognitive complexity. The researchers conducted an on-road experiment with 32 licensed drivers in a 2025 Chevrolet Trailblazer. Participants performed five secondary tasks: visual turn-by-turn guidance (low-load baseline), hands-free phone calls, Gemini Live single-turn interactions, Gemini Live multi-turn interactions, and the Operation Span (OSPAN) task (high-load anchor). Visual attention was measured using eye-tracking to record mean glance duration (MGD) and total eyes-off-road time (TEORT). Cognitive load was assessed via a tactile Detection Response Task (DRT), which measured reaction times and miss rates to vibrotactile stimuli. Subjective workload and distraction were also recorded using Likert scales. Statistical analysis employed linear mixed models to compare performance across conditions. Results indicated that Gemini Live interactions, both single-turn and multi-turn, imposed cognitive loads similar to hands-free phone calls, falling between the low demands of visual navigation and the high demands of the OSPAN task. Crucially, exploratory analysis revealed that cognitive load remained stable throughout extended multi-turn conversations, showing no cumulative increase over time. Visually, all tasks maintained mean glance durations well below the NHTSA’s 2-second safety threshold. Although total eyes-off-road time varied, drivers consistently dedicated longer glances to the roadway between brief off-road glances, particularly during voice interactions. Subjective ratings aligned with objective data, with participants reporting low effort and perceived distraction for Gemini Live. The findings suggest that advanced LLM conversational agents, when implemented via voice interfaces, impose cognitive and visual demands comparable to established, low-risk hands-free benchmarks. The study concludes that these systems do not inherently elevate crash risk beyond that of traditional voice calls, supporting their safe deployment in driving environments. This provides empirical evidence that despite the increased naturalism and complexity of AI-driven dialogue, the cognitive burden does not escalate dangerously, offering a basis for future regulatory guidelines for voice-based in-vehicle technologies.
Key finding
LLM-powered in-vehicle voice agents (Gemini Live), in both single- and multi-turn modes, impose cognitive load similar to hands-free phone calls and substantially below OSPAN, with mean glance durations under 2 seconds, supporting the safety of voice-only LLM agents in driving.
Methodology
on_road
Sample size: 32
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via arxiv_oa_fetch on 2026-05-08 (2 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | — | — | — | 1 | 2026-05-07 |
| archive | success | canonical_url | — | — | 2 | 2026-06-03 |
| extract | success | cached | — | — | 2 | 2026-06-10 |
| clean | success | — | — | — | 1 | 2026-06-01 |
| chunk | success | — | — | — | 1 | 2026-06-01 |
| embed | success | — | — | — | 1 | 2026-06-02 |
| enrich | success | — | — | — | 1 | 2026-05-07 |
| promote | success | — | — | — | 3 | 2026-06-06 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 16 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Applied Guidance: design guidelines
- Empirical Findings: behavioral performance data
- Theoretical Contribution: theory or model