Supporting detection of hostile intentions: automated assistance in a dynamic decision-making context

Patton, Colleen E.; Wickens, Christopher D.; Smith, C. A. P.; Noble, Kayla M.; Clegg, Benjamin A. · 2023 · Cognitive Research Principles and Implications

DOI: 10.1186/s41235-023-00519-5

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study investigates the efficacy of automated assistance and automation transparency in a dynamic decision-making (DDM) context, specifically focusing on the detection of hostile intentions. The research addresses the challenge of human–automation teaming in environments requiring evidence accumulation over time, such as naval surveillance or medical diagnosis. While automation can reduce cognitive load, it risks inducing automation bias or deskilling. The authors sought to determine if an imperfect automated aid improves detection accuracy and response time, and whether providing "medium" transparency—explaining the reasoning behind the aid’s recommendations—further enhances performance or alters decision-making processes. The experiment utilized a simulation where participants controlled a ship to identify which of six other ships exhibited hostile behavior (either "hunting" or "shadowing") based on movement patterns. The task required participants to make a series of moves to accumulate evidence before terminating the trial and submitting a diagnosis. The study involved 128 participants divided into two conditions: one receiving only an automated attention aid that highlighted the most likely hostile ship, and another receiving the aid plus transparency statements explaining the algorithm’s reasoning (e.g., "Ship 2 has been shadowing your ship for 13 out of the last 15 moves"). The design included three blocks: an unaided baseline, an aided phase, and a final unaided phase to assess learning or deskilling. The automated aid was imperfect, with accuracy improving as more moves were made, reaching an asymptotic accuracy of approximately 94%. Results indicated that the automated aid significantly improved performance compared to the unaided baseline, leading to higher accuracy and shorter response times. Participants frequently complied with the aid’s suggestions. However, detection remained suboptimal, suggesting that even with assistance, humans did not fully reach the automation’s potential performance level. Crucially, the addition of transparency had limited impact on all aspects of performance. There were no significant improvements in accuracy, response time, or compliance rates in the transparency condition compared to the non-transparent aided condition. Furthermore, transparency did not significantly alter the number of steps taken or the strategic approach to decision-making. The study also found that while the aid reduced cognitive processing demands, it did not necessarily facilitate better independent performance when the aid was withdrawn, challenging the hypothesis that reduced workload would automatically translate to improved skill retention. The findings suggest that while automated attention aids are beneficial in dynamic decision-making contexts, the benefits of automation transparency are not guaranteed and may be negligible in high-load environments. The lack of performance improvement from transparency implies that users may not attend to or effectively utilize explanatory information, or that the cognitive cost of processing transparency offsets its benefits. These results highlight the complexities of integrating automation into DDM tasks and suggest that simply providing reasoning for automated decisions does not inherently improve human–automation teaming outcomes. The study underscores the need for further research into how transparency can be designed to effectively support learning and trust calibration in dynamic, evidence-accumulation scenarios.

Key finding

The automated attention aid improved detection accuracy and reduced response times, but automation transparency provided limited additional benefits to performance.

Methodology

lab_experiment

Sample size: 126

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via verifier_blocked on 2026-05-10.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• automation surprise