Reaction time effects in lab- versus Web-based research: Experimental evidence
DOI: 10.3758/s13428-015-0678-9
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study addresses persistent skepticism among researchers and reviewers regarding the validity of reaction time measurements in Web-based experiments. While online research offers advantages in sample size and cost, concerns remain that increased technical and situational variance, along with software limitations, may obscure small but significant reaction time effects (e.g., differences of a few hundred milliseconds). Previous comparisons often lacked rigorous experimental control, relying on cross-sample comparisons rather than random assignment. This paper aims to provide a critical test of whether Web-based data collection is inferior to traditional laboratory methods for measuring reaction times. To isolate the effects of software technology from situational variance, the authors employed a between-subjects design with three conditions using a lexical decision task. Participants were randomly assigned to: (1) a laboratory condition using standard experimental software (E-Prime); (2) a laboratory condition using a browser-based implementation (HTML/JavaScript); or (3) a Web-based condition using the same browser-based implementation. The task required participants to judge whether six-letter strings were words or pseudowords. The study utilized the word frequency effect—a robust phenomenon where frequent words are recognized faster than infrequent words, typically yielding an effect size of 150–200 ms—as the dependent variable. This design allowed for specific comparisons: the difference between conditions 1 and 2 tested software accuracy, while the difference between conditions 2 and 3 tested the impact of situational and technical variance inherent to Web environments. The results demonstrated that the word frequency effect was substantial and statistically significant in all three conditions, with effect sizes ranging from 170 to 204 ms. There was no significant interaction between word frequency and experimental condition, indicating that the magnitude of the effect did not differ across settings. Specifically, Helmert contrasts revealed no significant difference between the E-Prime and browser-based software in the lab, nor between the lab-based browser condition and the Web-based condition. Although descriptive data showed slightly higher variability in reaction times for the Web condition, this did not compromise the detection of the primary effect. A power analysis confirmed that the study had sufficient statistical power to detect even small differences, reinforcing the conclusion that the null result was not due to insufficient sample size. The findings challenge the preconception that Web-based research is inadequate for measuring small reaction time differences. The study concludes that neither the use of browser-based JavaScript nor the increased technical and situational variance of the Web environment significantly impairs the ability to detect classic reaction time effects. These results support the validity of Web-based experiments as a viable alternative to laboratory studies, suggesting that skepticism regarding the accuracy of online reaction time data is unwarranted. The authors note that while this study complements previous work, further research across different paradigms is beneficial, but the current evidence strongly disconfirms the notion that Web-based methods are inherently inferior for reaction time measurement.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | OpenAlex-citations | — | — | 1 | 2026-06-19 |
| archive | success | unpaywall | — | — | 2 | 2026-06-25 |
| extract | success | pdftotext | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-26 |
| chunk | success | chunk | — | — | 1 | 2026-06-26 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-26 |
| enrich | failed | — | — | — | 4 | 2026-06-26 |
| promote | success | — | — | — | 1 | 2026-06-19 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-26 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: behavioral performance data