15th Speech in Noise Workshop, 11-12 January 2024, Potsdam, Germany 15th Speech in Noise Workshop, 11-12 January 2024, Potsdam, Germany

P23Session 1 (Thursday 11 January 2024, 15:35-18:00)
Stable individual differences in audiovisual benefit across three levels of linguistic structure: Exploring the role of cognitive and perceptual abilities

Jacqueline von Seth, Máté Aller, Matthew H. Davis
MRC Cognition and Brain Sciences Unit, Cambridge, United Kingdom

Individuals differ substantially in their ability to use visual cues encoded in the speaker’s facial movements to enhance speech perception. We recruited 114 normally-hearing participants to complete a three-part online experiment investigating the distribution, reliability, and predictors of individual audiovisual benefit for acoustically degraded speech. Rather than measuring changes in intelligibility due to adding visual speech, we measured the relative intelligibility of matched auditory-visual (AV) and auditory-only (AO) speech for materials at three levels of linguistic structure: meaningful sentences, monosyllabic words and isolated consonants. Participants performed speech identification tasks in AO, AV, and VO conditions in two sessions, set one week apart. Acoustic clarity was manipulated using a noise-vocoding procedure to create two levels of degradation. We matched report accuracy in high-clarity AO and low-clarity AV conditions by varying the mixing proportion of unintelligible 1-channel and intelligible 16-channel vocoded speech. In a third session, participants completed a battery of tests to assess their hearing, linguistic and cognitive skills.

We found that audiovisual benefit is stable (test-retest reliable) across experimental sessions using different item sets and significantly correlated across levels of linguistic structure and speakers. Multiple linear regression showed that individual differences in audiovisual benefit were explained by perceptual, rather than cognitive abilities. While auditory-only perception was predicted by verbal IQ, audiovisual benefit was predicted by better lipreading ability and relatively poorer hearing (estimated using the Digits-in-Noise test). These results represent the first step towards validating a short-form intelligibility-matched measure of individual audiovisual benefit and provide a foundation for further work assessing cognitive and neural (MEG) correlates of enhanced audiovisual speech benefit.

Last modified 2023-11-29 15:13:43