Advancements in Mixed Reality for Autonomous Vehicle Testing and Advanced Driver Assistance Systems: A Survey

Argui, Imane; Gueriau, Maxime; Ainouz, Samia · 2024 · Crossref

DOI: 10.1109/tits.2024.3473740

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This survey addresses the critical challenge of validating autonomous vehicles (AVs) and Advanced Driver Assistance Systems (ADAS) prior to widespread deployment. While fully autonomous Level 5 vehicles remain elusive, current systems require rigorous testing to ensure safety and robustness against unpredictable scenarios. Traditional testing methods, including public road testing, proving grounds, and pure simulation, face significant limitations. Specifically, pure simulations suffer from the "reality gap," where discrepancies between simulated dynamics and real-world physics lead to inaccurate model performance. Conversely, real-world testing is prohibitively expensive, time-consuming, and dangerous for replicating rare or critical edge cases. The paper investigates Mixed Reality (MR) as a hybrid solution that bridges these gaps by allowing vehicles to interact simultaneously with physical and virtual objects, thereby offering a safer, more realistic, and cost-effective testing environment. The authors conducted a comprehensive literature review to analyze the state-of-the-art in MR applications for self-driving systems. The study defines MR within the context of the Milgram continuum, distinguishing it from Augmented Reality (AR) and Virtual Reality (VR) by its capacity for simultaneous interaction between real and virtual entities. The review examines various "X-in-the-Loop" frameworks, particularly Scenario-in-the-Loop, which aligns closely with MR methodologies. The paper details the technical infrastructure required for MR testing, including popular simulators such as CARLA, Unity3D, Gazebo, and MATLAB. It further explores sensor integration techniques, focusing on the fusion of real-world data from LiDAR, cameras, and odometry sensors with virtual elements. Key technical challenges addressed include data synchronization via the Robot Operating System (ROS), temporal delay management using 5G technology, and sensor fusion algorithms like the Extended Kalman Filter to ensure accurate localization and perception. The findings highlight that MR testing offers substantial advantages over traditional methods by closing the reality gap and reducing resource requirements. MR enables the duplication of critical scenarios, such as pedestrian interactions or complex traffic conditions, in a controlled yet physically grounded environment. This approach allows for the validation of algorithms and machine learning models with higher fidelity than pure simulation while maintaining the safety constraints lacking in public road tests. The survey identifies that MR frameworks rely on digital twins to maintain continuous feedback between the physical vehicle and the virtual environment, ensuring that the vehicle’s sensors perceive virtual objects as if they were real. The analysis reveals that while MR significantly enhances testing efficiency and realism, it requires sophisticated handling of latency and data synchronization to maintain system integrity. The significance of this work lies in its contribution to the development of reliable AV safety certification protocols. By synthesizing current research, the paper establishes MR as a vital component in the testing pipeline, complementing existing pillars of AV validation. It underscores the necessity of ongoing technical progress in sensor fusion and low-latency communication to fully realize the potential of MR. The survey concludes by identifying unexplored research directions, emphasizing that continued advancements in MR technologies are essential for overcoming the remaining obstacles to deploying safe and robust autonomous driving systems. This review provides a structured foundation for future research, guiding developers toward more integrated and realistic testing methodologies.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• hud ar windshield