Exploratory Advanced Research Program : Video Analytics Research Projects
archive: archived pipeline: cataloged verified
Get this paper ↗ (full text — opens at the source; we link to it, we don't host it)
Summary
This report outlines the Federal Highway Administration’s (FHWA) Exploratory Advanced Research (EAR) Program initiatives aimed at automating the analysis of massive video datasets, specifically the Strategic Highway Research Program 2 (SHRP 2) Naturalistic Driving Study (NDS). The NDS comprises over 1.2 million hours of video and 2 petabytes of data collected from approximately 3,000 volunteers. The sheer volume creates a significant bottleneck, as manual feature extraction would require nearly 600 technicians working for a year. To address this, the EAR Program funded six research projects to develop automated tools for feature extraction, driver behavior analysis, and privacy protection, thereby making the data accessible for safety research. The initial project, conducted by Carnegie Mellon University (CMU), focused on machine learning for automated analysis of forward-facing highway video. Researchers developed a prototype tool integrating computer vision and machine-learning algorithms to detect roadway features such as vehicles, traffic signs, and traffic light states. The system utilizes segmentation algorithms to understand scene context and handles challenges like overlapping objects and low-resolution data. Subsequent projects expanded the focus to interior vehicle cameras. SRI International developed "DCode," a comprehensive automatic coding system that extracts driver behavior features (e.g., head pose, gaze, hand positions) and contextual environmental factors (e.g., weather, pedestrians). CMU’s "DB-SAM" project created a real-time system to assess driver emotional states, detecting fatigue and distraction by monitoring eye openness, mouth movement, and hand placement on the steering wheel. The University of Wisconsin–Madison developed an open-source platform for quantifying driver distraction, using image enhancement and region-of-interest segmentation to track facial landmarks and hand activity, allowing for automated behavior characterization. To address privacy concerns regarding personally identifiable information in the NDS, two projects focused on automated identity masking. CMU and the University of Pittsburgh developed "Facial Action Transfer" (FAT), a non-reversible technique that clones facial actions from the driver to a target face, preserving behavioral cues while obscuring identity. SRI International developed "DMask," which replaces the driver’s head with a computer-generated avatar. This system tracks facial motions and gaze direction, ensuring that behaviorally relevant data remains accessible while protecting volunteer privacy. Both systems include graphical user interfaces to allow for human verification and correction of masking errors. The significance of these projects lies in their ability to transform the utility of large-scale naturalistic driving data. By automating feature extraction and identity masking, the EAR Program reduces the cost and time required to process raw data, enabling a broader range of researchers to analyze driver behavior and safety factors. These tools facilitate the development of better safety countermeasures by providing accurate, scalable methods for identifying high-risk behaviors and contextual driving conditions. The program also established benchmarking techniques with Oak Ridge National Laboratory to ensure the technical assessment and calibration of these automated systems, laying the groundwork for future advancements in transportation safety research.
Key finding
Automated video analytics and machine learning algorithms can effectively replace manual coding to extract roadway features, driver behaviors, and identity-masked data from large-scale naturalistic driving studies.
Methodology
mixed_methods
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via bulk_ingest_rosap on 2026-05-23 (6 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | rosap | — | — | 2 | 2026-05-23 |
| archive | success | — | — | — | 1 | 2026-05-23 |
| extract | success | cached | — | — | 2 | 2026-06-10 |
| clean | success | — | — | — | 1 | 2026-06-01 |
| chunk | success | — | — | — | 1 | 2026-06-01 |
| embed | success | — | — | — | 1 | 2026-06-02 |
| enrich | success | — | — | — | 1 | 2026-05-23 |
| promote | success | — | — | — | 1 | 2026-05-23 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 3 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 19 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: tool software, dataset resource, measurement protocol