Multisensory synchrony of contextual boundaries affects temporal order memory, but not encoding or recognition

Vincent van de Ven*, Guyon Kleuters, Joey Stuiver

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

We memorize our daily life experiences, which are often multisensory in nature, by segmenting them into distinct event models, in accordance with perceived contextual or situational changes. However, very little is known about how multisensory boundaries affect segmentation, as most studies have focused on unisensory (visual or audio) segmentation. In three experiments, we investigated the effect of multisensory boundaries on segmentation in memory and perception. In Experiment 1, participants encoded lists of pictures while audio and visual contexts changed synchronously or asynchronously. After each list, we tested recognition and temporal associative memory for pictures that were encoded in the same audio-visual context or that crossed a synchronous or an asynchronous multisensory change. We found no effect of multisensory synchrony for recognition memory: synchronous and asynchronous changes similarly impaired recognition for pictures encoded at those changes, compared to pictures encoded further away from those changes. Multisensory synchrony did affect temporal associative memory, which was worse for pictures encoded at synchronous than at asynchronous changes. Follow up experiments showed that this effect was not due to the higher dimensionality of multisensory over unisensory contexts (Experiment 2), nor that it was due to the temporal unpredictability of contextual changes inherent to Experiment 1 (Experiment 3). We argue that participants formed situational expectations through multisensory synchronicity, such that synchronous multisensory changes deviated more strongly from those expectations than asynchronous changes. We discuss our findings in light of supportive and conflicting findings of uni- and multi-sensory segmentation.

Original languageEnglish
Pages (from-to)583-597
Number of pages15
JournalPsychological Research
Volume87
Issue number2
Early online date28 Apr 2022
DOIs
Publication statusPublished - Mar 2023

Keywords

  • EVENT SEGMENTATION
  • INTEGRATION
  • LONG-TERM-MEMORY
  • MUSIC
  • ORGANIZATION
  • PERCEPTION
  • PREDICTION ERROR
  • SITUATION MODELS

Fingerprint

Dive into the research topics of 'Multisensory synchrony of contextual boundaries affects temporal order memory, but not encoding or recognition'. Together they form a unique fingerprint.

Cite this