DEVELOPING AN APPROACH TO MULTIMODAL QUANTITATIVE ASSESSMENT OF INTERVIEWERS’ COGNITIVE LOAD: FIRST RESULTS OF A FIELD QUASI EXPERIMENT
DOI: 10.22363/2313-2272-2018-18-4-627-637
archive: archived pipeline: cataloged verified
Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)
Summary
This paper addresses a gap in survey methodology regarding the systematic assessment of interviewers’ cognitive load. While it is established that survey instruments affect data quality, the specific impact of the mental effort required by interviewers to manage attention, memory, and motor control during data entry has rarely been quantified. The author argues that excessive cognitive load can lead to measurement errors and reduced data quality, particularly when interviewers must simultaneously maintain communication with respondents and operate data collection tools. The study aims to develop and test a multimodal approach to quantitatively assess this load, specifically examining the transition from Paper-and-Pencil Interviewing (PAPI) to Computer-Assisted Personal Interviewing (CAPI). The research was conducted as a field quasi-experiment involving 24 female interviewers (mean age 60.9) from the Russian Longitudinal Monitoring Survey (RLMS-HSE) during their training for CAPI adoption. The methodology combined three types of measures: subjective, physiological, and behavioral. Subjectively, interviewers used a modified five-point version of the Paas Cognitive Load Scale to rate the mental effort required for specific questionnaire sections. Physiologically, heart rate (HR) was continuously monitored using Mi Band 2 wristbands as an indicator of sympathetic nervous system activation. Behaviorally, screen recordings from Android tablets were used to synchronize HR data with specific questionnaire blocks and to track navigation patterns. Participants conducted test interviews with model respondents using the CAPI tablet, allowing for the comparison of perceived load between retrospective PAPI assessments and real-time CAPI experiences. The results demonstrated the feasibility and limitations of this multimodal approach. Analysis of heart rate data using linear mixed-effects models revealed significant main effects for both questionnaire sections and individual interviewers, as well as a significant interaction effect, suggesting that HR is a valid indicator of cognitive activation during survey administration. However, the lack of randomization in block order and rest intervals complicated the attribution of HR changes to specific sections. Subjectively, interviewers rated three questionnaire sections (“Internet,” “Pension,” “Women only”) as significantly easier to complete via CAPI than PAPI, with two others showing a similar trend. The Paas scale demonstrated high internal consistency (Cronbach’s alpha = 0.945 for CAPI). Technical challenges were noted, including data loss from screen recording software failures (approximately 25%) and synchronization issues with iOS devices. The study concludes that combining subjective scales, physiological indicators like heart rate, and behavioral screen recordings offers a promising framework for assessing interviewer cognitive load. The findings support the construct validity of these measures in a field setting and suggest that CAPI may reduce cognitive effort for certain complex sections compared to PAPI. However, the authors emphasize that further research with larger, more diverse samples and stricter experimental controls is needed to establish convergent validity with other physiological measures, such as pupillometry. This approach provides a foundation for optimizing survey instrument design and interviewer training to minimize cognitive overload and improve data quality.
Provenance
The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.
| Stage | Outcome | Tool | Model | Prompt | Attempts | Completed |
|---|---|---|---|---|---|---|
| discover | success | DOAJ | — | — | 1 | 2026-06-18 |
| archive | success | unpaywall | — | — | 1 | 2026-06-25 |
| extract | success | cached | — | — | 2 | 2026-06-26 |
| clean | success | clean | — | — | 1 | 2026-06-18 |
| chunk | success | chunk | — | — | 1 | 2026-06-18 |
| embed | success | embed | Qwen/Qwen3-Embedding-8B | — | 1 | 2026-06-18 |
| promote | success | — | — | — | 1 | 2026-06-18 |
| summarize | success | llm | qwen3.6-27b-prismaquant | summ-v5 | 1 | 2026-06-26 |
| tag | success | vector_similarity | — | — | 6 | 2026-06-18 |
| verify | success | — | — | — | 1 | 2026-06-26 |
Summary generated by qwen3.6-27b-prismaquant on 2026-06-26; verification: verified.
Topics
Ranked by relevance to this paper. Hover a topic for its definition.
Information type
What kind of knowledge this paper contributes, grouped by family — independent of topic (what it is about) and method (how it was studied).
- Empirical Findings: self report data, physiological data