Technical Report for Argoverse2 Scenario Mining Challenges on Iterative Error Correction and Spatially-Aware Prompting

Chen, Yifei; Greer, Ross · 2025 · ArXiv.org

DOI: 10.48550/arxiv.2506.11124

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This technical report addresses the challenges of mining specific driving scenarios from large-scale autonomous vehicle datasets, such as Argoverse 2, using Large Language Models (LLMs). While the RefAV framework allows natural language queries to be translated into executable code for scenario retrieval, it suffers from two primary limitations: runtime errors in LLM-generated code and semantic inaccuracies in interpreting complex spatial relationships. The authors propose two enhancements to mitigate these issues: a Fault-Tolerant Iterative Code Generation (FT-ICG) mechanism and Enhanced Prompting for Spatial Relational Functions (EP-SRF). The FT-ICG method treats the LLM as an iterative debugger rather than a single-shot generator. When generated code fails during execution, the system captures the runtime error message and re-prompts the LLM with the previous code and error context, allowing it to refine the script up to five iterations. This approach increases the success rate of code generation and improves pipeline robustness. The EP-SRF method addresses semantic errors where LLMs misassign parameters in functions describing relative positions (e.g., confusing "a car in front of a pedestrian" with "a pedestrian in front of a car"). By augmenting prompts with explicit instructions clarifying the roles of "track candidates" and "related candidates" for specific directional and orientational functions, the system ensures higher fidelity in translating natural language into correct functional representations. Experiments were conducted on the Argoverse 2 validation and test sets using three LLMs: Qwen2.5-VL-7B, Gemini 2.5 Flash, and Gemini 2.5 Pro. Performance was evaluated using HOTA-Temporal, HOTA, Timestamp F1, and Log F1 metrics. Results demonstrate consistent improvements across all models. For instance, using Gemini 2.5 Pro, the baseline RefAV achieved a HOTA-Temporal score of 43.34, which increased to 45.53 with FT-ICG alone and further to 46.71 with the combined FT-ICG and EP-SRF methods. On the official test set, the combined approach achieved a HOTA-Temporal score of 52.37, significantly outperforming the baseline. Ablation studies confirm that FT-ICG primarily boosts recall by recovering from syntax errors, while EP-SRF enhances semantic precision, improving Timestamp and Log-level metrics. The findings indicate that domain-specific scaffolding, including iterative error correction and explicit semantic prompting, significantly enhances the reliability and accuracy of LLM-driven scenario mining. The proposed methods achieve state-of-the-art performance without manual intervention, addressing fundamental challenges in converting complex human language into precise machine-executable instructions. This work underscores the importance of robust code generation and precise parameter interpretation for effective automated analysis of autonomous driving data.

Key finding

The proposed fault-tolerant iterative code generation and enhanced spatial prompting techniques consistently improve scenario mining performance across multiple LLMs, achieving a state-of-the-art HOTA-Temporal score of 52.37 on the Argoverse 2 test set.

Methodology

simulation_modeling

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• generative ai voice assistants