Automated Data Curation Using GPS & NLP to Generate Instruction-Action Pairs for Autonomous Vehicle Vision-Language Navigation Datasets
DOI: 10.48550/arxiv.2505.03174
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the high cost and inefficiency of manually annotating Instruction-Action (IA) datasets required for training autonomous vehicle (AV) vision-language navigation (VLN) models. The authors propose leveraging Global Positioning System (GPS) navigation applications as an untapped source of natural language instructions. By combining GPS voice commands with video data and vehicle trajectories, the study aims to automatically generate large-scale Vision-Language-Action (VLA) triads without human annotation, thereby accelerating dataset creation for robust AV training. The methodology involves a pilot data collection phase using three common navigation apps: Apple Maps, Google Maps, and Waze. The researchers drove five routes across residential, rural, commercial, and freeway environments in California, collecting 71, 82, and 80 verbal commands, respectively. These commands were manually categorized into eight referentiality classes: Road Names, Distance, Static Objects, Turn, Cardinal direction, Location Name, Lane Information, and Light Information. To demonstrate full automation, the authors developed the ADVLAT-Engine, a prototype system that synchronizes visual perception (video camera), language commands (GPS audio transcribed via OpenAI’s Whisper model), and action streams (GPS logs). This system links specific video frames to verbalized instructions, creating synchronized VLA data. The results characterize the linguistic diversity of GPS instructions, revealing that commands vary significantly in referential cues, such as using distance versus static landmarks. Statistical analysis of the pilot data showed that multi-attribute commands are common, with combinations like "Destination, Road, Turn" appearing most frequently. The study confirmed that the ADVLAT-Engine successfully generates annotated VLA triads by synchronizing video, spatial positioning, and transcribed audio. The automated pipeline effectively replaces manual labor, capturing active decision-making segments while filtering out stationary driving periods. The significance of this work lies in its potential to solve the scarcity and cost issues associated with domain-specific VLA datasets. By automating data curation, the approach enables the rapid generation of high-quality, diverse training data for end-to-end learning models. This method supports contrastive learning approaches and improves the robustness of AV decision-making across different geographic regions. The authors conclude that this automated framework can significantly reduce the performance gap between familiar and new environments, facilitating the development of safer, more interactive autonomous systems.
Key finding
An automated system using GPS app outputs and speech transcription can successfully generate synchronized vision-language-action datasets by categorizing navigation instructions into eight distinct referential classes without manual human annotation.
Methodology
field_study
Sample size: 233
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 2 | 2026-05-28 |
| archive | success | canonical_url | — | — | 1 | 2026-06-04 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | — | — | — | 1 | 2026-05-28 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: dataset resource, tool software