Measuring cognitive load with subjective rating scales during problem solving: differences between immediate and delayed ratings
DOI: 10.1007/s11251-014-9328-3
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This study investigates whether subjective cognitive load (CL) ratings differ depending on when they are collected: immediately after each task or as a single delayed rating after a sequence of tasks. While subjective rating scales are widely used in educational research to measure mental effort and perceived task difficulty, it remains unclear if averaging immediate ratings yields comparable results to a single retrospective rating. The authors aimed to replicate and extend previous findings by examining both mental effort and perceived difficulty, relating these measures to performance, and determining if the discrepancy between immediate and delayed ratings is specific to cognitive load or applies to affective variables like interest and motivation. Two experiments were conducted with teacher education university students (N = 168 in Experiment 1; N = 107 in Experiment 2). Participants solved a sequence of six problems with increasing intrinsic cognitive load, measured by element interactivity. In Experiment 1, a repeated-measures design was used where students rated mental effort, perceived difficulty, interest, and motivation on 7-point Likert scales immediately after each problem and once after the entire sequence. Experiment 2 manipulated the order of problem complexity to control for recency effects. Performance was assessed by counting correctly solved problems. Statistical analyses included paired samples t-tests to compare average immediate ratings against delayed ratings, and stepwise regression analyses to identify which immediate ratings best predicted the delayed overall ratings. The results demonstrated that delayed ratings for both mental effort and perceived task difficulty were significantly higher than the average of the immediate ratings provided after each problem. In contrast, no significant differences were found between immediate and delayed ratings for affective variables such as interest and motivation. Regression analyses revealed that the delayed cognitive load ratings were best predicted by the ratings given for the most complex problems in the sequence, rather than being a simple average of all tasks. This pattern held true even when controlling for the order of presentation in Experiment 2, indicating that the effect was driven by task complexity rather than recency bias. Additionally, both immediate and delayed mental effort ratings significantly predicted problem performance, whereas perceived difficulty ratings did not consistently predict performance. The findings indicate that immediate and delayed subjective cognitive load ratings are not comparable indicators of experienced load. Delayed ratings appear to reflect the cognitive load imposed by the most complex tasks in a series, leading to higher scores than the average of immediate ratings. This discrepancy is specific to cognitive load measures and does not extend to affective variables. The study concludes that researchers must carefully consider the timing of subjective CL assessments, as delayed ratings may overestimate overall load by disproportionately weighting complex tasks. These results have implications for instructional design research, suggesting that immediate ratings may provide a more accurate representation of the average cognitive load experienced during a learning process.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | OpenAlex-citations | — | — | 1 | 2026-06-20 |
| archive | success | openalex | — | — | 5 | 2026-06-26 |
| extract | success | pdftotext | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-26 |
| chunk | success | chunk | — | — | 1 | 2026-06-26 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-26 |
| enrich | failed | — | — | — | 1 | 2026-06-26 |
| promote | success | — | — | — | 1 | 2026-06-20 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-26 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: self report data