Generative AI for Autonomous Driving: Frontiers and Opportunities
DOI: 10.48550/arxiv.2505.08854
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper presents a comprehensive survey on the integration of Generative Artificial Intelligence (GenAI) into autonomous driving systems, addressing the critical technical barriers preventing the achievement of Level 5 autonomy. The authors identify three primary challenges hindering current progress: the inability to generalize to rare "long tail" events, difficulties in managing uncertainty and reliability, and the high computational and economic costs of complex sensor suites. Motivated by the transformative potential of GenAI to synthesize realistic data and unify perception with reasoning, the paper explores how generative models can overcome these limitations by creating diverse training scenarios, enhancing situational awareness, and enabling scalable, data-driven architectures. The survey systematically reviews the foundational principles of generative modeling, including Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), Diffusion Models, Neural Radiance Fields (NeRF), 3D Gaussian Splatting, and Large Language Models (LLMs). It categorizes existing datasets into real-world single-vehicle perception, multi-agent prediction, synthetic simulation, and language-annotated benchmarks. The authors then map frontier applications of these models across various modalities, such as image, LiDAR, trajectory, occupancy, and video generation. Furthermore, the paper details practical implementations, including synthetic data workflows for sensor-space and traffic state generation, end-to-end driving strategies, personalized driving systems, high-fidelity digital twins, and intelligent transportation networks. It also extends the discussion to embodied robotics, examining simulation-to-reality transfer and multimodal perception-to-action translation. Key findings highlight that GenAI offers a paradigm shift from brittle modular pipelines to unified systems capable of deeper understanding and adaptability. Specifically, generative models address the "long tail" problem by synthesizing high-fidelity data for rare events, such as accidents or extreme weather, which are difficult to capture in real-world datasets. The integration of multimodal foundation models allows for the unification of perception, prediction, and planning within a single architecture, leveraging pre-trained world knowledge to improve decision-making under uncertainty. The survey also identifies significant opportunities in cross-domain transfer and the creation of comprehensive digital twins that facilitate robust generalization. The significance of this work lies in its provision of a forward-looking reference for researchers, engineers, and policymakers. The authors conclude by outlining critical future directions, including the need for theoretical assurances, trust metrics, and standardized evaluation methodologies. They also address broader implications, such as regulatory compliance, ethical concerns, environmental impacts, and the socio-technical influence of autonomous systems. By unifying technical advancements with practical challenges, the survey aims to accelerate the development of safe, reliable, and equitable autonomous mobility systems, positioning GenAI as a central enabler for the next generation of intelligent transportation.
Key finding
Generative AI offers a paradigm shift for autonomous driving by enabling high-fidelity data synthesis, unified perception-to-action architectures, and improved generalization for rare scenarios, thereby addressing key technical barriers to Level 5 autonomy.
Methodology
review
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 2 | 2026-05-28 |
| archive | success | canonical_url | — | — | 1 | 2026-06-04 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | — | — | — | 1 | 2026-05-28 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Methodological Resource: tool software
- Theoretical Contribution: computational model