The impacts of multi-agent quantity, type and transparency on mental workload, situation awareness and human out-of-the-loop

Xu, Xinran; Yu, Ruifeng; Ji, Sihan; Yuan, Minhui · 2023 · Crossref

DOI: 10.54941/ahfe1003964

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study investigates optimal cooperation modes between humans and multi-agent systems, specifically examining how the quantity, type, and transparency of autonomous agents affect human mental workload, situation awareness (SA), and out-of-the-loop (OOTL) performance. As autonomous systems become more prevalent, humans must monitor and collaborate with them, yet increasing agent numbers or complexity can overload cognitive resources, degrade SA, and increase the risk of errors. The research aims to determine how these factors influence operator performance and whether enhancing agent transparency can mitigate negative effects. The researchers conducted two within-subjects experiments using a simulated environment involving two distinct agents: an intelligent assistant for assigning unmanned vehicles and a semi-autonomous dynamic positioning system for vessels. Experiment 1 utilized a 2 (multi-agent quantity: 1 vs. 2) × 2 (multi-agent type: homogeneous vs. heterogeneous) design with 12 participants. Homogeneous agents performed similar functions, while heterogeneous agents had different functions. Experiment 2 involved 22 participants and examined a 2 (multi-agent type) × 2 (transparency: SAT Level 1 vs. SAT Level 2) design. Transparency levels were based on the Situation Awareness-based Transparency model, where Level 1 provided basic status information and Level 2 included reasoning processes and diagnostic analysis. Dependent variables were measured using the NASA-TLX for mental workload, the Situational Awareness Rating Technique (SART) for SA, a self-rating scale for OOTL degree, and task accuracy for performance. Results from Experiment 1 indicated that monitoring two agents simultaneously significantly increased mental workload, decreased SA, and increased OOTL degree compared to monitoring one agent, even when total task volume remained constant. Monitoring heterogeneous agents resulted in higher mental workload than homogeneous agents. Crucially, agent type significantly impacted SA only when monitoring two agents; heterogeneous agents caused a greater loss of SA than homogeneous ones in multi-agent scenarios. Experiment 2 demonstrated that higher transparency (SAT Level 2) significantly reduced mental workload, improved SA, decreased OOTL degree, and enhanced task performance compared to lower transparency (SAT Level 1), regardless of agent type. The findings suggest that while increasing the number of agents and their heterogeneity imposes significant cognitive burdens on human operators, these negative effects can be effectively mitigated by increasing system transparency. Providing operators with insight into the agents' reasoning processes and decision logic improves situational understanding and reduces the likelihood of becoming out-of-the-loop. The study concludes that designing for higher transparency is a cost-effective and actionable strategy for improving human-multi-agent team performance, offering practical guidance for interface design and work organization in automated systems.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-nvidia on 2026-08-10; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• situational awareness

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Empirical Findings: self report data
• Theoretical Contribution: theory or model