Beyond the Wizard of Oz: Using Imperfect Machine Learning to Examine the Impact of Reliability of Augmented Reality Cues on Visual Search Performance

Raikwar, Aditya; Mifsud, Domenick; Wickens, Christopher D.; Kelly, Brendan J.; Warden, Amelia C.; Clegg, Benjamin A.; Ortega, Francisco R. · 2023 · Unknown

DOI: 10.1109/ismar-adjunct60411.2023.00093

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study investigates the impact of imperfect machine learning (ML) cues on visual search performance in Augmented Reality (AR) environments, specifically addressing the phenomenon of automation bias. While AR systems can enhance search efficiency by providing visual cues, real-world ML algorithms are not perfectly accurate. Previous research relied on Wizard-of-Oz (WoZ) methods to simulate errors, which may not accurately reflect how users respond to genuine algorithmic failures. This research aims to determine if real-time ML-driven cues produce different behavioral outcomes than simulated ones and to quantify the negative effects of imperfect automation on user performance. The researchers conducted an experiment with 53 participants using a Varjo XR-3 AR head-mounted display. Participants performed a visual search task to locate specific objects in a 3D environment. The study employed a mixed-subject design with two between-subject groups: one receiving 100% accurate cues (perfect automation) and the other receiving cues generated by a YOLOv5-Nano ML model with 88.9% accuracy (imperfect automation). Both groups also completed a control condition with no cues. The ML system used a synthetic dataset for training and provided real-time arrow cues pointing to target locations. Participants were instructed that the system was not fully reliable and should verify suggestions. The results demonstrated that while cueing generally improved accuracy compared to the no-cue condition, the reliability of the cue significantly impacted performance. Participants in the perfect cueing condition achieved 98.7% accuracy, whereas those in the imperfect ML condition achieved only 90.7% accuracy. Crucially, performance with imperfect automation was significantly worse than with perfect automation. The study found that participants exhibited strong automation bias, frequently following incorrect ML cues rather than relying on their own visual search. This reliance on flawed algorithmic guidance led to higher error rates and longer search times when the ML system failed, confirming that imperfect cues can degrade performance below baseline levels in certain trials. The significance of this work lies in its validation of automation bias in realistic, real-time ML-driven AR systems, moving beyond simulated WoZ scenarios. The findings indicate that ML-generated cues can induce a higher magnitude of automation bias than previously observed in WoZ studies, likely due to the perceived reliability of the technology. This has critical implications for safety-critical applications such as aviation, medical diagnosis, and autonomous vehicles, where over-reliance on imperfect automation can lead to severe errors. The study highlights the need for system designs that mitigate automation bias and suggests that the benefits of AR cueing are contingent upon high algorithmic reliability.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-08.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-09; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• visual search