Realistic Artificial Datasets: Objective Evaluation of Data-Driven Safety Analysis Models
archive: archived pipeline: cataloged verified
Get this paper ↗ (full text — opens at the source; we link to it, we don't host it)
Summary
This document outlines two research initiatives funded by the Federal Highway Administration’s (FHWA) Exploratory Advanced Research (EAR) Program, aimed at improving the evaluation of data-driven safety analysis (DDSA) models. DDSA models assist transportation agencies in quantifying safety data and predicting the effects of safety measures. However, a critical limitation exists: because the true underlying relationships between crash variables (such as driver behavior, environment, and roadway design) are unknown in real-world data, it is difficult to verify if a model accurately captures these cause-and-effect interactions or merely predicts overall crash frequencies. To address this, the projects focus on generating Realistic Artificial Datasets (RAD) with predetermined safety relationships, serving as objective testbeds to evaluate model performance. The first project, led by the University of Missouri in collaboration with Iowa State University and Texas Tech University, utilizes statistical and machine learning methods to create open-source RADs for urban interchanges. Urban interchanges were selected because they are overrepresented in fatal and injury crashes yet lack accurate crash data. The team uses de-identified data from the second Strategic Highway Research Program naturalistic driving study, which includes over one million hours of video data, to build at least three simulator testbeds representing varying conditions, such as daytime versus nighttime or specific crash scenarios. The goal is to produce datasets that allow practitioners to determine if a method effectively identifies underlying cause-and-effect relationships. The team is also documenting their methodology to enable future researchers to generate RADs for other facility types, such as work zones or pedestrian facilities. The second project, a collaboration between the University of Connecticut and the University of Central Florida, is developing a customizable RAD-generating framework applicable to all facility types. This tool generates datasets at both macroscopic and microscopic levels. The macroscopic level supports single-step data models currently used in safety analysis, while the microscopic level accommodates complex factors like human behavior and emerging technologies, such as autonomous vehicles. Principal Investigator John Ivan notes that this approach moves beyond road characteristics to address the significant role of driver behavior in crash causality. To demonstrate proof of concept, the team will generate case studies focusing on vehicle crashes on segments and vehicle-pedestrian crashes at intersections. The significance of these initiatives lies in their potential to standardize the objective evaluation of DDSA models. By providing RADs with known underlying relationships, researchers and practitioners can rigorously test new models to determine their efficacy without relying on ambiguous real-world data. This capability supports the FHWA’s goal of expanding DDSA adoption by enabling users to select the best analytical methods for their specific data needs. The RAD tools leverage leading-edge machine learning algorithms to support both current safety analysis approaches and future exploratory models, ensuring robust evaluation capabilities as transportation systems evolve.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via bulk_ingest_rosap on 2026-05-23 (8 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | rosap | — | — | 2 | 2026-05-23 |
| archive | success | — | — | — | 1 | 2026-05-23 |
| extract | success | cached | — | — | 6 | 2026-06-15 |
| clean | success | — | — | — | 1 | 2026-06-01 |
| chunk | success | — | — | — | 1 | 2026-06-01 |
| embed | success | — | — | — | 1 | 2026-06-02 |
| enrich | success | — | — | — | 1 | 2026-05-23 |
| promote | success | — | — | — | 1 | 2026-05-23 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 8 | 2026-06-15 |
| tag | success | vector_similarity | — | — | 19 | 2026-06-11 |
| verify | success | — | — | — | 1 | 2026-06-15 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-15; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: crash risk outcomes
- Methodological Resource: dataset resource, tool software