Cortical tracking of continuous speech under bimodal divided attention

Xie, Zilong; Brodbeck, Christian; Chandrasekaran, Bharath · 2022 · Crossref

DOI: 10.1101/2022.10.29.514344

archive: archived pipeline: cataloged

Get this paper ↗ (DOI — opens at the source; we link to it, we don't host it)

Summary

This study investigates how bimodal divided attention—specifically, the allocation of cognitive resources between auditory and visual tasks—affects the neural processing of natural continuous speech. While previous research has primarily focused on simplified speech stimuli, this paper addresses the gap in understanding how crossmodal attention impacts the mapping of acoustic features into linguistic representations of varying complexity. The authors hypothesized that dual-task conditions would reduce neural tracking of speech, particularly at higher cognitive loads, with linguistic representations potentially being more affected than acoustic ones, consistent with supramodal accounts of attention. The experiment involved 16 participants who performed a visuospatial n-back task (primary task) while listening to audiobook stories (secondary task). Three conditions were tested: a high-load dual-task (3-back), a low-load dual-task (0-back), and an auditory single-task condition where participants attended to the story while ignoring visual stimuli. Electroencephalography (EEG) was recorded to assess neural responses. The analysis employed multivariate temporal response function (mTRF) encoding models to predict EEG activity from acoustic features (spectrograms, onset spectrograms) and linguistic features (sublexical, word-form, and sentence-level surprisal and entropy measures derived from information-theoretic models). Prediction accuracy was quantified using z-transformed Pearson’s correlations, and the unique contribution of each predictor was assessed by calculating the change in prediction accuracy (Δz) when specific predictors were removed from the full model. Behaviorally, the high-load dual-task condition resulted in significantly lower speech comprehension accuracy compared to the other two conditions. Neural tracking analysis revealed that despite the increased cognitive demand of the high-load condition, the tracking of most acoustic and linguistic features remained unchanged relative to the low-load condition. However, when compared to the auditory single-task condition, the dual-task conditions selectively reduced neural tracking of specific acoustic and linguistic features. Crucially, these reductions were observed primarily at latencies greater than 200 ms, while earlier latencies were largely unaffected. This temporal dissociation suggests that early sensory representations of speech are robust to crossmodal attentional competition. The findings indicate that the behavioral deficits in speech comprehension under bimodal divided attention do not stem from impaired early sensory processing. Instead, the effects manifest at later cognitive processing stages, likely involving higher-order linguistic integration. The selective impact on specific features at later latencies implies that crossmodal attention-related mechanisms are not uniform across different levels of speech processing. This work provides a systematic, holistic view of how attentional resources are allocated across acoustic-to-linguistic levels, challenging the assumption that divided attention uniformly degrades all aspects of speech processing and highlighting the resilience of early auditory representations even under high cognitive load.

Provenance

The full processing record for this entry. Every stage of this paper's journey through the pipeline is logged — what ran, with which tool and model, how many attempts it took, and when it last completed.

Summary generated by qwen3.8-27b-gittensor on 2026-08-23; verification: pending re-verification.

Topics

Ranked by relevance to this paper. Hover a topic for its definition.

• auditory