Comment on: “Effects of Exercise Training Interventions on Executive Function in Older Adults: A Systematic Review and Meta‑analysis”

Zimmer, Philipp; Javelle, Florian; Lampit, Amit · 2021 · Crossref

DOI: 10.1007/s40279-020-01369-7

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This letter to the editor critiques the methodological rigor of a systematic review and meta-analysis by Chen et al., which investigated the effects of exercise training interventions on executive function in older adults. The authors argue that while identifying effective exercise settings for clinical translation is critical, Chen et al.’s study contains fundamental flaws that undermine the reliability of its findings. The critique focuses on three primary methodological errors: the handling of non-independent effect sizes, the inappropriate use of a fixed-effect model, and deviations from standard systematic review guidelines. The authors identify that Chen et al. treated 107 effect sizes from 33 articles as independent units of analysis, despite many outcomes being nested within single studies or reported across multiple manuscripts by the same research groups (e.g., Gothe et al. and Albinet et al.). This failure to account for dependency resulted in the double-counting of participants, inflating the total sample size from 3008 to 7023. Consequently, this error led to an underestimation of standard error, an overestimation of precision, and the assignment of excessive weight to studies reporting multiple outcomes. The authors note that established methods exist to handle such dependency, yet Chen et al. ignored them, biasing pooled effect estimates and investigations of publication bias. Furthermore, the critique highlights that Chen et al. utilized a fixed-effect model for all analyses. The authors contend this approach is inappropriate for the heterogeneous set of studies included, which varied in population, intervention type, outcomes, and risk of bias. A fixed-effect model assumes all studies assess the same underlying effect, attributing variation to chance rather than genuine differences. Given the substantial heterogeneity reported in Chen et al.’s own Cochran test results, the authors argue that a random-effects model would have been more suitable to account for between-study variability and avoid overestimating the precision of the mean effect. Finally, the authors question Chen et al.’s decision to include only studies published after 2003, ostensibly to update findings from Colcombe and Kramer’s seminal work. The authors assert that excluding studies from previous reviews defies the logic of an update and violates basic systematic review practices recommended by PRISMA and the Cochrane Handbook. They conclude that the flaws in Chen et al.’s work are rooted in basic meta-analysis principles rather than statistical modeling opinions. Therefore, the authors advise against using this meta-analysis to inform study design or clinical practice, urging the field toward more robust trials and systematic reviews.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• motorcyclist skill

Information type

What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).

• Synthesis & Review: quantitative synthesis