QRTEngine: An easy solution for running online reaction time experiments using Qualtrics
DOI: 10.3758/s13428-014-0530-7
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper introduces QRTEngine, an open-source JavaScript engine designed to facilitate the creation of online reaction time (RT) experiments within the Qualtrics survey platform. The authors address the growing demand for online behavioral research, noting that existing solutions often require advanced programming skills, specialized software installations (such as Java or Flash), or dedicated web servers. QRTEngine aims to lower these barriers by providing a plugin-free, cross-browser compatible tool that requires minimal coding expertise while maintaining timing accuracy comparable to laboratory standards. The engine operates by embedding JavaScript code into Qualtrics surveys, utilizing the HTML5 `requestAnimationFrame` API to synchronize stimulus onset with monitor refresh rates for precise timing. It logs detailed time audit information, including intended and calculated durations, allowing researchers to filter data based on timing precision. To validate its accuracy, the authors conducted a timing study using external chronometry (a photosensitive diode) under varying CPU and RAM loads. Results indicated that under low to high load conditions, the mean deviation between intended and actual stimulus duration was approximately 6 ms, falling within one display frame (16.67 ms) for 97% of trials. Performance degraded slightly under maximum system load, with deviations averaging 10 ms. The study also highlighted that server communication delays introduce variable intertrial intervals, averaging 1,388 ms for data transmission. To demonstrate practical utility, the authors replicated three classic paradigms online via Amazon Mechanical Turk: a Stroop task, an attentional blink task, and a masked-priming task. In the Stroop experiment, 52 participants completed 96 trials. The results successfully reproduced the classic Stroop effect, with significantly slower reaction times and higher error rates for incongruent trials compared to congruent ones. The paper further notes that QRTEngine can reproduce reliable negative masked-priming effects, which had previously been difficult to achieve with JavaScript-based methods. The significance of this work lies in providing a robust, accessible tool for online RT research that does not require participant software installation, thereby meeting the constraints of platforms like Amazon Mechanical Turk. By combining the ease of use of Qualtrics with precise timing mechanisms, QRTEngine enables researchers without extensive programming backgrounds to conduct valid behavioral experiments. The authors conclude that while online RT data contains more noise than lab-based data due to hardware variability, QRTEngine provides sufficient accuracy for many psychological paradigms, provided researchers account for timing errors through data filtering and adequate sample sizes.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | OpenAlex-citations | — | — | 1 | 2026-06-24 |
| archive | success | unpaywall | — | — | 2 | 2026-06-26 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-25 |
| chunk | success | chunk | — | — | 1 | 2026-06-25 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-25 |
| promote | success | — | — | — | 1 | 2026-06-24 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-25 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: behavioral performance data