When is enough enough? Empirical guidelines to determine participant sample size for scene viewing studies

Hoogerbrugge, Alex J.; Hooge, Ignace T. C.; Hessels, Roy S.; Strauch, Christoph · 2025 · Crossref

DOI: 10.3758/s13428-025-02754-8

archive: archived pipeline: cataloged verified

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study addresses the lack of empirical guidelines for determining participant sample sizes in eye-tracking scene viewing studies. While standard statistical tests have established power analysis tools, researchers analyzing gaze distribution maps or areas of interest (AOIs) often rely on rules of thumb, leading to inefficient resource use or unreliable data. The authors aim to provide data-driven recommendations by analyzing the law of diminishing returns in gaze data saturation, helping researchers balance statistical power with practical constraints. The researchers analyzed two large datasets representing different experimental setups. Dataset 1 consisted of 1,248 participants viewing a single static image in a low-cost, unsupervised setting. Dataset 2 involved 115 participants viewing over 200 images in a high-cost, supervised laboratory setting. The authors evaluated sample size sufficiency by comparing spatial distribution maps using Normalized Scanpath Saliency (NSS) and Area Under the Curve (AUC) metrics, as well as AOI-based measures such as visit counts and dwell times. They employed a leave-n-out bootstrapping procedure to assess how closely smaller samples matched a benchmark of the full dataset, calculating the relative improvement in metric scores for incremental sample sizes. The results demonstrate progressively diminishing returns as sample size increases. For distribution map similarity using NSS, achieving a 5% relative increase in generalizability required increasing the sample from 13 to 20 participants in Dataset 1, or from 10 to 16 in Dataset 2. Further increases yielded smaller gains; for instance, moving from 13 to 34 participants in Dataset 1 provided only marginal additional improvement. Dataset 2 showed higher initial metric scores and lower variance due to the multiple-image design, making each additional participant more reliable for improving generalizability compared to the single-image setup. For AOI analyses, reducing outcome variance by 25% required increasing the sample from 13 to 44 participants. The findings were consistent across different image categories, indicating the guidelines are broadly applicable. The significance of this work lies in providing concrete, easy-to-use reference tables for academics and industry professionals. By quantifying the trade-off between sample size and information gain, the study enables researchers to make informed decisions about resource allocation. It clarifies that while no single sample size fits all needs, substantial gains in reliability are achieved with relatively small samples (e.g., 10–20 participants), after which returns diminish sharply. This allows for more efficient experimental design in fields ranging from psychology to marketing and human-computer interaction.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• attention allocation