Substantial differences in attentional disengagement between Prolific and MTurk samples: Implications for experimental outcomes
DOI: 10.21203/rs.3.rs-2824983/v1
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study investigates whether attentional disengagement and experimental outcomes differ significantly between participants recruited via Prolific Academic and Amazon Mechanical Turk (MTurk). The research was motivated by concerns regarding data quality in online psychological studies and the lack of comparative evidence regarding sustained attention across these platforms. Additionally, the authors sought to determine if manipulating the risk of negative consequences for poor performance could reduce attentional disengagement, such as mind wandering and multitasking. The researchers conducted an experiment involving 82 Prolific participants and 78 MTurk participants, who were randomly assigned to either a high-risk (80% chance of losing points for errors) or low-risk (20% chance) condition. Participants completed a 1-back working memory task designed to assess sustained attention. Attentional engagement was measured through task accuracy, periodic thought probes assessing mind wandering, and post-task questionnaires regarding media multitasking. The study also included trait measures for cognitive failures, inattention, and sensation seeking. MTurk participants were screened for high approval ratings and experience, while both groups were matched for age and location. The results revealed substantial differences between the two platforms. MTurk participants exhibited significantly higher levels of attentional disengagement than Prolific participants, evidenced by lower task accuracy, higher rates of self-reported mind wandering, and greater engagement in media-related multitasking. Furthermore, MTurk participants scored higher on trait measures of inattention and sensation seeking. Crucially, the effect of the risk manipulation depended on the platform: in the MTurk sample, the high-risk condition led to significantly better task performance compared to the low-risk condition, whereas no such effect was observed in the Prolific sample. Correlation analyses also showed stronger relationships between trait inattention, sensation seeking, and task performance in the MTurk sample compared to the Prolific sample. These findings indicate that the choice of recruitment platform can significantly impact experimental results, particularly those involving attention and cognitive performance. Prolific samples demonstrated higher engagement and less susceptibility to risk-based manipulations, potentially due to lower baseline disengagement. In contrast, MTurk samples showed greater variability and responsiveness to incentives, which may offer higher ecological validity for studying real-world attentional lapses. The study concludes that researchers must consider platform-specific differences in participant engagement when designing and interpreting online cognitive experiments.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | Crossref | — | — | 1 | 2026-06-19 |
| archive | success | canonical_url | — | — | 1 | 2026-06-26 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-20 |
| chunk | success | chunk | — | — | 1 | 2026-06-20 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-20 |
| promote | success | — | — | — | 1 | 2026-06-19 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-20 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: behavioral performance data