Driver Behavior Extraction from Videos in Naturalistic Driving Datasets with 3D ConvNets

Miao, Hanwen; Zhang, Shengan; Flannagan, Carol · 2022 · Crossref

DOI: 10.1007/s42421-022-00053-8

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the underutilization of video data in Naturalistic Driving Data (NDD) datasets, such as the SHRP2 Naturalistic Driving Study. While NDD is critical for understanding crash causation and developing safety countermeasures, the vast majority of recorded video remains unannotated because manual coding by humans is labor-intensive and slow. The authors propose a computer vision method using 3D Convolutional Neural Networks (3D ConvNets) to automatically extract driver cell-phone-related behaviors from video footage, thereby increasing the efficiency of data analysis. The study utilizes video data from the Integrated Vehicle-Based Safety Systems (IVBSS) Field Operational Test, which includes interior cabin and face-camera views of 96 drivers. The dataset was previously annotated for three cell-phone behaviors: dialing, interacting, and talking. The authors developed a three-module algorithm: clip generation, clip classification, and clip aggregation. Videos were segmented into 8-second clips. The classification module employed a Two-stream Inflated 3D ConvNet (I3D) to predict behavior categories and whether a clip contained the start or end point of a behavior. To handle class imbalance and confusion between "dialing" and "interacting," the authors merged these categories and balanced the training dataset. The aggregation module combined consecutive clips to identify continuous behavior events, using both rough merging and refined aggregation based on start/end point predictions. Experimental results demonstrated that the model achieved a test accuracy of 79.4% for classifying clips when using both camera views and a balanced three-class dataset. The model struggled to distinguish between dialing and interacting, necessitating their merger. Temporal prediction accuracy showed that 71% of predicted "interacting" chunks and 86% of predicted "talking" chunks contained the correct behavior. Although the start/end point prediction was weak, the overall approach successfully identified relevant video segments. The significance of this work lies in its potential to dramatically improve the efficiency of reviewing NDD video. Given that cell-phone use occurs in approximately 6% of driving time, the algorithm’s 79% precision means it can identify relevant clips roughly 13 times more efficiently than random viewing. This method allows researchers to quickly locate specific driver behaviors for further analysis, making large-scale video datasets more usable for safety research without requiring full manual annotation.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-25; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• distraction detection algorithms
• naturalistic crash near crash

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: observational prevalence
• Methodological Resource: dataset resource, tool software