TY - CHAP
T1 - Multisensory integration in speech processing
T2 - neural mechanisms of cross-modal aftereffects
AU - Kilian-Hütten, Nick
AU - Formisano, Elia
AU - Vroomen, J.
PY - 2017
Y1 - 2017
N2 - Traditionally, perceptual neuroscience has focused on unimodal information processing. This is true also for investigations of speech processing, where the auditory modality was the natural focus of interest. Given the complexity of neuronal processing, this was a logical step, considering that the field was still in its infancy. However, it is clear that this restriction does not do justice to the way we perceive the world around us in everyday interactions. Very rarely is sensory information confined to one modality. Instead, we are constantly confronted with a stream of input to several or all senses and already in infancy, we match facial movements with their corresponding sounds (campbell et al. 2001; kuhl and meltzoff 1982). Moreover, the information that is processed by our individual senses does not stay separated. Rather, the different channels interact and influence each other, affecting perceptual interpretations and constructions (calvert 2001). Consequently, in the last 15–20 years, the perspective in cognitive science and perceptual neuroscience has shifted to include investigations of such multimodal integrative phenomena. Facilitating cross-modal effects have consistently been demonstrated behaviorally (shimojo and shams 2001). When multisensory input is congruent (e.g., semantically and/or temporally) it typically lowers detection thresholds (frassinetti et al. 2002), shortens reaction times (forster et al. 2002; schröger and widmann 1998), and decreases saccadic eye movement latencies (hughes et al. 1994) as compared to unimodal exposure. When incongruent input is (artificially) added in a second modality, this usually has opposite consequences (sekuler et al. 1997).
AB - Traditionally, perceptual neuroscience has focused on unimodal information processing. This is true also for investigations of speech processing, where the auditory modality was the natural focus of interest. Given the complexity of neuronal processing, this was a logical step, considering that the field was still in its infancy. However, it is clear that this restriction does not do justice to the way we perceive the world around us in everyday interactions. Very rarely is sensory information confined to one modality. Instead, we are constantly confronted with a stream of input to several or all senses and already in infancy, we match facial movements with their corresponding sounds (campbell et al. 2001; kuhl and meltzoff 1982). Moreover, the information that is processed by our individual senses does not stay separated. Rather, the different channels interact and influence each other, affecting perceptual interpretations and constructions (calvert 2001). Consequently, in the last 15–20 years, the perspective in cognitive science and perceptual neuroscience has shifted to include investigations of such multimodal integrative phenomena. Facilitating cross-modal effects have consistently been demonstrated behaviorally (shimojo and shams 2001). When multisensory input is congruent (e.g., semantically and/or temporally) it typically lowers detection thresholds (frassinetti et al. 2002), shortens reaction times (forster et al. 2002; schröger and widmann 1998), and decreases saccadic eye movement latencies (hughes et al. 1994) as compared to unimodal exposure. When incongruent input is (artificially) added in a second modality, this usually has opposite consequences (sekuler et al. 1997).
U2 - 10.1007/978-1-4939-7325-5_6
DO - 10.1007/978-1-4939-7325-5_6
M3 - Chapter
SN - 9781493973231
T3 - Innovations in cognitive neuroscience
SP - 105
EP - 127
BT - Neural mechanisms of language
A2 - Mody, Maria
PB - Springer
CY - Boston, MA
ER -