Everyday communication frequently includes situations with several persons speaking simultaneously. Such speech-on-speech listening is usually challenging, as understanding the person of interest requires the segregation of competing speech streams as a prerequisite to focus attention on the target talker. The features for segregation, like voice, spatial and intensity cues, are not always readily available.

In the pursuit to understand the intricacies of speech-on-speech segregation at a fine-grained temporal level, we developed a Visual World Paradigm (VWP, Tanenhaus et al., 1995) based on matrix sentences (The Oldenburg Sentence Test, Wagener et al., 1999). The VWP visually presents elements of the matrix sentences parallel to the spoken stimulus and is based on the finding that gaze fixations and speech processing are closely linked in time. Various Target-to-Masker Ratios (TMRs) were employed as cues to aid discriminating the target sentence from the masker sentence.

Our study, involving young normal-hearing individuals, unveiled that both gaze fixations and pupil dilation effectively reflected the segregation of competing sentences, yielding valuable insights into the processing of elements within the target sentence. Our data analysis included three parameters: the peak and slope of eye-gaze fixations to gauge speech processing, and the pupil dilation over time to assess the related cognitive load. The impact of varying TMRs clearly became evident, with a significant decrease in the peak and the slope of fixations and a significant increase in pupil dilation in more challenging TMR scenarios. Notably, this held true even when the corresponding speech recognition was virtually perfect.

Funding: Deutsche Forschungsgemeinschaft (ME2751/6-1)

References:

• Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science (New York, N.Y.), 268(5217), 1632–1634. doi:10.1126/science.7777863
• Wagener K., Brand T., Kollmeier B. (1999). Entwicklung und Evaluation eines Satztests in deutscher Sprache II: optimierung des Oldenburger satztests (Development and evaluation of a sentence test in German language II: optimization of the Oldenburg sentence test). Z. Audiol. 38 44–56.