The sound of size: Crossmodal binding in pitch-size synesthesia: A combined TMS, EEG and psychophysics study

N. Bien*, S. ten Oever, R. Goebel, A.T. Sack

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Crossmodal binding usually relies on bottom-up stimulus characteristics such as spatial and temporal correspondence. However, in case of ambiguity the brain has to decide whether to combine or segregate sensory inputs. We hypothesise that widespread, subtle forms of synesthesia provide crossmodal mapping patterns which underlie and influence multisensory perception. Our aim was to investigate if such a mechanism plays a role in the case of pitch-size stimulus combinations. Using a combination of psychophysics and ERPs, we could show that despite violations of spatial correspondence, the brain specifically integrates certain stimulus combinations which are congruent with respect to our hypothesis of pitch-size synesthesia, thereby impairing performance on an auditory spatial localisation task (Ventriloquist effect). Subsequently, we perturbed this process by functionally disrupting a brain area known for its role in multisensory processes, the right intraparietal sulcus, and observed how the Ventriloquist effect was abolished, thereby increasing behavioural performance. Correlating behavioural, TMS and ERP results, we could retrace the origin of the synesthestic pitch-size mappings to a right intraparietal involvement around 250 ms. The results of this combined psychophysics, TMS and ERP study provide evidence for shifting the current viewpoint on synesthesia more towards synesthesia being at the extremity of a spectrum of normal, adaptive perceptual processes, entailing close interplay between the different sensory systems. Our results support this spectrum view of synesthesia by demonstrating that its neural basis crucially depends on normal multisensory processes. (PsycINFO Database Record (c) 2011 APA, all rights reserved) (journal abstract)
Original languageEnglish
Pages (from-to)663-672
Number of pages10
JournalNeuroimage
Volume59
Issue number1
DOIs
Publication statusPublished - 2 Jan 2012

Keywords

  • Crossmodal
  • Synesthesia spectrum
  • Parietal cortex
  • TMS
  • ERP
  • GRAPHEME-COLOR SYNESTHESIA
  • AUDIOVISUAL INTEGRATION
  • MULTISENSORY INTEGRATION
  • INDIVIDUAL-DIFFERENCES
  • MENTAL REPRESENTATION
  • SUPERIOR COLLICULUS
  • PARIETAL CORTEX
  • SPEECH SOUNDS
  • HUMAN BRAIN
  • SYNAESTHESIA

Cite this