I See What You See: Inferring Sensor and Policy Models of Human Real-World Motor Behavior
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the challenge of inferring sensor and policy models of human motor behavior from observational data, specifically in scenarios where direct sensory measurements are unavailable. The authors argue that predicting human behavior requires modeling both the actions chosen and the sensory information guiding those choices. Existing methods often rely on artificial tasks or assume known sensory data, limiting their applicability to complex, real-world tasks like driving. To overcome this, the authors propose an abstract structural estimation approach based on Simultaneous Estimation of Rewards and Dynamics (SERD), which allows for the joint inference of sensor models and stochastic policies under the assumption of rational behavior and optimal sensory fusion. The method is concretely implemented for Sensor Scheduling Linear Quadratic Gaussian (SLQG) problems, a class of Partially Observable Markov Decision Processes (POMDPs) suitable for modeling active information gathering. The approach utilizes probabilistic Inverse Optimal Control (IOC) within the Maximum Causal Entropy (MCE) framework. By leveraging the differentiability of the soft Bellman equations with respect to sensor model parameters, the method estimates sensor noise covariances by minimizing a Lagrangian dual function. This allows the inference of how humans perceive their environment (e.g., visual gaze direction) solely from observed actions and states, without needing recorded sensory inputs. The effectiveness of the approach is demonstrated through a numerical evaluation using data from a real-world driving experiment. Seventeen experienced drivers performed lane-keeping tasks at varying speeds while engaging in visually demanding secondary tasks requiring glances at different vehicle displays (Head-up Display, Combi Instrument, Navigation System). The authors modeled the driver’s sensor states based on gaze direction and inferred the associated sensor noise parameters. The inferred models were used to predict future glance and steering behavior. Prediction quality was assessed using Kullback-Leibler divergence for glance duration distributions and negative log-likelihood for state predictions. The results indicate that inferring sensor models significantly improves prediction accuracy compared to baselines that use fixed "best guess" sensor parameters or direct policy estimation. Specifically, the proposed method outperformed approaches that assumed perfect sensing or static sensor models, demonstrating that accounting for the uncertainty and switching nature of human sensory perception is crucial for accurate behavior prediction. This work provides the first general approach for joint inference of sensor and policy models, offering a robust tool for applications such as driver assistance systems that require accurate models of human attention and control strategies in dynamic environments.
Key finding
The proposed method for jointly inferring sensor and policy models from behavioral data significantly improves the prediction of automobile driver glance and steering behavior compared to baseline approaches.
Methodology
on_road
Sample size: 17
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 2 | 2026-05-28 |
| archive | success | canonical_url | — | — | 1 | 2026-06-04 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | semantic_scholar | — | — | 2 | 2026-06-04 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Theoretical Contribution: computational model