Comparing attentional disengagement between Prolific and MTurk samples
DOI: 10.1038/s41598-023-46048-5
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study investigates whether attentional disengagement—manifested as mind wandering and multitasking—differs significantly between participants recruited via Prolific and Amazon Mechanical Turk (MTurk). The research was motivated by concerns regarding data quality in online psychological studies and conflicting prior findings regarding platform comparisons. Additionally, the authors sought to determine if manipulating the risk of negative consequences for poor performance could reduce disengagement. The researchers conducted two experiments using an adapted 1-back attention task. In Experiment 1, 160 participants (80 from each platform) were randomly assigned to either a low-risk condition (20% chance of losing points for errors) or a high-risk condition (80% chance). Attentional disengagement was measured via task accuracy, periodic thought probes assessing mind wandering, and post-task self-reports of multitasking. Individual traits related to attention and risk tolerance were also assessed. Experiment 2 replicated the design but imposed stricter MTurk reputation criteria (requiring >1000 completed HITs) and increased MTurk remuneration to match Prolific’s purchasing power parity, addressing potential confounds from Experiment 1. Results from Experiment 1 revealed substantial platform differences: Prolific participants exhibited surprisingly low levels of disengagement and high task performance, whereas MTurk participants showed significantly higher rates of mind wandering and multitasking. An interaction between risk and platform was observed; in the high-risk condition, MTurk participants performed better and disengaged less than those in the low-risk condition, but this risk effect was absent in the Prolific sample. Furthermore, correlations between individual traits and study variables differed between platforms. In Experiment 2, platform differences persisted but were reduced in magnitude after implementing stricter recruitment criteria and adjusted remuneration for MTurk. The findings indicate that recruitment platform and specific recruitment criteria significantly impact measures of attentional disengagement and task performance. The study suggests that Prolific samples may be more attentive or motivated than native MTurk samples, potentially leading to different experimental outcomes. The results also imply that while increasing the perceived risk of negative consequences can reduce disengagement, this effect may be contingent on the participant pool's baseline engagement levels. These conclusions highlight the importance of considering recruitment source and criteria when designing online cognitive studies and interpreting data quality.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | Crossref | — | — | 1 | 2026-06-19 |
| archive | success | canonical_url | — | — | 1 | 2026-06-26 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-20 |
| chunk | success | chunk | — | — | 1 | 2026-06-20 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-20 |
| promote | success | — | — | — | 1 | 2026-06-19 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-20 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: behavioral performance data