Data Reduction Solution for Driving Simulator

Pawar, Piyush; Anthony, Thomas; McManus, Benjamin; Stavrinos, Despina · 2020 · Crossref

DOI: 10.1109/southeastcon44009.2020.9249691

archive: archived pipeline: cataloged

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This paper addresses the challenge of processing and reducing large volumes of raw data generated by high-fidelity driving simulators integrated with eye-tracking equipment. The motivation stems from the increasing complexity of data acquisition in translational research aimed at understanding at-risk driver behavior to reduce motor vehicle collision morbidity and mortality. While driving simulators offer high experimental control, their raw output is formatted for software interfaces, making it too complex and large for direct statistical analysis. The authors developed a programmatic data reduction solution to transform this raw data into an analysis-ready format, specifically tailored for studies using the Realtime Technologies Inc. simulator. The methodology involves a multi-stage pipeline implemented in Python using the Pandas library within a Jupyter Notebook environment on a High Performance Computing (HPC) cluster. The process begins with data auditing to ensure integrity, verifying that drive data (`.plt` and `.hdr` files) and eye data (`.log` files from the Smart Eye Pro system) are correctly labeled and located. Data acquisition involves importing these files into DataFrames; drive data consists of 81 columns, while eye data consists of 62 columns. The pipeline converts UNIX timestamps to UTC, standardizes units (e.g., meters to miles/feet), and prefixes column names with 'D' for drive and 'E' for eye data to distinguish sources. A critical step is the merging of these two datasets using an outer merge on UTC and group counter columns, ensuring a 1:1 temporal synchronization of driving and gaze metrics. The core of the reduction algorithm calculates 76 pre-defined statistical parameters (mean and standard deviation) for both drive and eye data, resulting in 152 values per subset. These statistics are computed across four levels of granularity: overall dataset, specific "Map Sections" (regions of interest like freeways), combined "Map Sections and Map Halves" (specific roadway elements like curves or intersections), and hazard-specific regions. The algorithm handles missing data by filling absent parameters with a constant, ensuring consistent output structure regardless of which scenarios a participant encountered. The solution was tested across numerous participants and varying experimental scenarios, demonstrating its ability to curate data according to specific study requirements. The significance of this work lies in its provision of a standardized, reproducible framework for handling complex simulator data. By automating the transformation of raw, multi-source data into a summarized tabular format, the solution lowers the barrier for researchers to perform statistical analyses on driving behavior. The approach is generalizable to other studies utilizing driving simulators, facilitating more efficient translational research in transportation safety and injury prevention.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.8-27b-gittensor on 2026-08-23; verification: pending re-verification.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• simulator validity fidelity

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Methodological Resource: tool software, measurement protocol, dataset resource