A new model of decision processing in instrumental learning tasks
DOI: 10.7554/elife.63055
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses the disconnect between cognitive modeling traditions for reinforcement learning (RL) and decision-making. While joint RL-Evidence Accumulation Models (RL-EAMs) have emerged to study their interaction, the dominant approach combines RL with the Diffusion Decision Model (DDM). The authors argue that the DDM fails to capture crucial aspects of response time (RT) distributions during learning and has theoretical limitations regarding absolute value effects and multi-alternative choices. To resolve this, the authors propose a new model, the Advantage Racing Diffusion (RL-ARD), based on a racing accumulator architecture that incorporates both the difference (advantage) and sum of expected rewards, alongside an urgency signal. The study evaluates the RL-ARD against the standard RL-DDM and two other racing models (RL-RD and RL-lARD) using data from four instrumental learning experiments. Experiment 1 involved binary choices with varying reward probabilities to manipulate difficulty. Experiment 2 tested speed-accuracy trade-offs using explicit cues. Experiment 3 examined stimulus-response mapping reversals. Experiment 4 extended the testing to three-alternative choices with manipulated reward magnitudes. The authors employed hierarchical Bayesian methods to fit models to trial-by-trial data, assessing performance via posterior predictive distributions and the Bayesian Predictive Information Criterion (BPIC). The results demonstrate that the RL-ARD significantly outperforms the RL-DDM and other competitors. In Experiment 1, the RL-ARD achieved the lowest BPIC score (4577) compared to the RL-DDM (7673), indicating a superior balance of fit and complexity. The RL-ARD successfully captured the full shape of RT distributions, including variability and skew, as well as learning-related changes in accuracy and RTs. Crucially, the model accounted for effects of stimulus difficulty, speed-accuracy trade-offs, and contingency reversals without requiring additional mechanisms. Furthermore, the RL-ARD was the only model capable of explaining behavior in the three-alternative task, addressing the DDM’s limitation to binary choices. The analysis confirmed that both the advantage (difference) and sum terms were necessary for optimal performance, even when reward magnitude was not explicitly manipulated. The significance of this work lies in providing a more robust and flexible framework for modeling the interaction between learning and decision-making. By overcoming the DDM’s inability to handle absolute value effects and multi-option choices, the RL-ARD offers a computationally tractable basis for broader applications in cognitive neuroscience. This model allows for more accurate psychological inferences by capturing the complete RT distribution and its evolution during learning, thereby bridging the gap between RL and EAM traditions more effectively than previous joint models.
Key finding
The advantage racing diffusion model outperforms the diffusion decision model in capturing response time distributions and behavioral phenomena in instrumental learning tasks.
Methodology
simulation_modeling
Sample size: 55
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed. Discovered via author_sweep_intake on 2026-05-28 (2 acquisition events logged).
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | author_sweep | — | — | 3 | 2026-05-28 |
| archive | success | openalex | — | — | 9 | 2026-06-06 |
| extract | success | cached | — | — | 3 | 2026-06-10 |
| clean | success | clean | — | — | 1 | 2026-06-04 |
| chunk | success | chunk | — | — | 1 | 2026-06-04 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-04 |
| enrich | success | — | — | — | 1 | 2026-05-28 |
| promote | success | — | — | — | 1 | 2026-06-04 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 2 | 2026-06-10 |
| tag | success | vector_similarity | — | — | 15 | 2026-06-11 |
| verify | success | — | — | — | 2 | 2026-06-10 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-10; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Theoretical Contribution: computational model