Understanding speech in the presence of other speech: Perceptual mechanisms for auditory scene analysis in human listeners

We take it for granted that we can converse with other people in daily life and be understood with little, if any, noticeable effort. However, it is fairly unusual to hear the speech of a particular talker in isolation; speech is typically heard in the presence of interfering sounds, such as the voices of other talkers. The human auditory system, which is responsible for our sense of hearing, therefore faces the challenge of identifying which parts of the sounds reaching our ears have originated from the same environmental source. This involves separating those sound elements coming from one source (e.g., the voice of one talker) from those arising from other sources, and grouping them in ways that can be interpreted by higher-level processes in the brain (such as those involved in our understanding of speech). Without a solution to this "auditory scene analysis" problem, our perceptions of speech (and other sounds) would not correspond to the events that produced them. Humans have been exposed to a variety of complex listening environments over the course of evolution, and so we are generally very successful at understanding the speech of one person in the presence of other talkers. This contrasts with attempts to develop listening machines, which often fail catastrophically when confronted with complex listening environments, such as an open-plan office or a crowded party. Human listeners with hearing impairment also find these environments very difficult, even when using the latest developments in hearing-aid or cochlear-implant technology. So far, most research on auditory scene analysis has focussed on relatively simple sounds and has identified a number of general principles for the perceptual grouping and separation of sound elements. However, at least as currently understood, these principles seem inadequate to explain fully the perceptual grouping of speech. This is because the speech signal consists of a diverse and rapidly changing stream of sounds. The speech of our native language is also a highly familiar stimulus, and so by adulthood our auditory system has had many years to learn about its speech-specific properties. These properties may also assist in the successful perceptual grouping of speech. Much of the information necessary to understand speech is carried by the changes in frequency over time of a few broad peaks in the frequency spectrum of the speech signal, known as formants. The aim of this project is to investigate how human listeners presented with speech sound mixtures are able to group together the appropriate formants, and to reject others, such that the speech of the talker we want to listen to can be understood. We will do so using perceptual experiments with human listeners, in which we measure how the intelligibility of target speech (measured, for example, as the number of words reported correctly) changes under a variety of conditions. The project will explore the roles of general-purpose grouping factors (i.e., those that apply to a wide variety of sounds) and of speech-specific grouping factors, including higher-level constraints associated with the articulation of speech (i.e., the way our tongue, lips, and jaw move when we speak) and with the rules of our language. Our approach is to generate artificial speech-like stimuli with precisely controlled properties, to mix target speech with carefully designed "competitors" that offer alternative grouping possibilities, and to measure how manipulating the properties of these competitors affects the ability of human listeners to recognise the target speech in the mixture. The results of this project will not only improve our understanding of how human listeners separate speech from interfering sounds, but will also help to refine computer models of listening. Such refinements will in turn provide ways of improving the performance of devices such as hearing aids and automatic speech recognisers when they operate in noisy environments.