Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals

A.W. de Borst, G. Valente, I.P. Jääskeläinen, P. Tikka

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

In the perceptual domain, it has been shown that the human brain is strongly shaped through experience, leading to expertise in highly-skilled professionals. What has remained unclear is whether specialization also shapes brain networks underlying mental imagery. In our fMRI study, we aimed to uncover modality-specific mental imagery specialization of film experts. Using multi-voxel pattern analysis we decoded from brain activity of professional cinematographers and sound designers whether they were imagining sounds or images of particular film clips. In each expert group distinct multi-voxel patterns, specific for the modality of their expertise, were found during classification of imagery modality. These were mainly localized to the occipito-temporal and parietal cortex for cinematographers and auditory cortex for sound designers. We also found generalized patterns across perception and imagery that were distinct for the two expert groups: they involved frontal cortex for the cinematographers and temporal cortex for the sound designers. Notably, the mental representations of film clips and sounds of cinematographers contained information that went beyond modality-specificity. We were able to successfully decode the implicit presence of film genre from brain activity during mental imagery in cinematographers. The results extend existing neuroimaging literature on expertise into the domain of mental imagery and show that experience in visual versus auditory imagery can alter the representation of information in modality-specific association cortices.
Original languageEnglish
Pages (from-to)428-438
Number of pages11
JournalNeuroimage
Volume129
DOIs
Publication statusPublished - 1 Apr 2016

Keywords

  • MVPA
  • fMRI
  • Mental imagery
  • Film
  • Expertise
  • Naturalistic stimuli
  • EARLY VISUAL-CORTEX
  • HUMAN AUDITORY-CORTEX
  • WORKING-MEMORY
  • FUNCTIONAL-ANATOMY
  • CORTICAL ACTIVITY
  • HESCHLS GYRUS
  • NEURAL BASIS
  • MINDS IMAGE
  • FMRI
  • AREAS

Cite this

@article{7a3f617993254e36884cc32946c43437,
title = "Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals",
abstract = "In the perceptual domain, it has been shown that the human brain is strongly shaped through experience, leading to expertise in highly-skilled professionals. What has remained unclear is whether specialization also shapes brain networks underlying mental imagery. In our fMRI study, we aimed to uncover modality-specific mental imagery specialization of film experts. Using multi-voxel pattern analysis we decoded from brain activity of professional cinematographers and sound designers whether they were imagining sounds or images of particular film clips. In each expert group distinct multi-voxel patterns, specific for the modality of their expertise, were found during classification of imagery modality. These were mainly localized to the occipito-temporal and parietal cortex for cinematographers and auditory cortex for sound designers. We also found generalized patterns across perception and imagery that were distinct for the two expert groups: they involved frontal cortex for the cinematographers and temporal cortex for the sound designers. Notably, the mental representations of film clips and sounds of cinematographers contained information that went beyond modality-specificity. We were able to successfully decode the implicit presence of film genre from brain activity during mental imagery in cinematographers. The results extend existing neuroimaging literature on expertise into the domain of mental imagery and show that experience in visual versus auditory imagery can alter the representation of information in modality-specific association cortices.",
keywords = "MVPA, fMRI, Mental imagery, Film, Expertise, Naturalistic stimuli, EARLY VISUAL-CORTEX, HUMAN AUDITORY-CORTEX, WORKING-MEMORY, FUNCTIONAL-ANATOMY, CORTICAL ACTIVITY, HESCHLS GYRUS, NEURAL BASIS, MINDS IMAGE, FMRI, AREAS",
author = "{de Borst}, A.W. and G. Valente and I.P. J{\"a}{\"a}skel{\"a}inen and P. Tikka",
year = "2016",
month = "4",
day = "1",
doi = "10.1016/j.neuroimage.2016.01.043",
language = "English",
volume = "129",
pages = "428--438",
journal = "Neuroimage",
issn = "1053-8119",
publisher = "Elsevier Science",

}

Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals. / de Borst, A.W.; Valente, G.; Jääskeläinen, I.P.; Tikka, P.

In: Neuroimage, Vol. 129, 01.04.2016, p. 428-438.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals

AU - de Borst, A.W.

AU - Valente, G.

AU - Jääskeläinen, I.P.

AU - Tikka, P.

PY - 2016/4/1

Y1 - 2016/4/1

N2 - In the perceptual domain, it has been shown that the human brain is strongly shaped through experience, leading to expertise in highly-skilled professionals. What has remained unclear is whether specialization also shapes brain networks underlying mental imagery. In our fMRI study, we aimed to uncover modality-specific mental imagery specialization of film experts. Using multi-voxel pattern analysis we decoded from brain activity of professional cinematographers and sound designers whether they were imagining sounds or images of particular film clips. In each expert group distinct multi-voxel patterns, specific for the modality of their expertise, were found during classification of imagery modality. These were mainly localized to the occipito-temporal and parietal cortex for cinematographers and auditory cortex for sound designers. We also found generalized patterns across perception and imagery that were distinct for the two expert groups: they involved frontal cortex for the cinematographers and temporal cortex for the sound designers. Notably, the mental representations of film clips and sounds of cinematographers contained information that went beyond modality-specificity. We were able to successfully decode the implicit presence of film genre from brain activity during mental imagery in cinematographers. The results extend existing neuroimaging literature on expertise into the domain of mental imagery and show that experience in visual versus auditory imagery can alter the representation of information in modality-specific association cortices.

AB - In the perceptual domain, it has been shown that the human brain is strongly shaped through experience, leading to expertise in highly-skilled professionals. What has remained unclear is whether specialization also shapes brain networks underlying mental imagery. In our fMRI study, we aimed to uncover modality-specific mental imagery specialization of film experts. Using multi-voxel pattern analysis we decoded from brain activity of professional cinematographers and sound designers whether they were imagining sounds or images of particular film clips. In each expert group distinct multi-voxel patterns, specific for the modality of their expertise, were found during classification of imagery modality. These were mainly localized to the occipito-temporal and parietal cortex for cinematographers and auditory cortex for sound designers. We also found generalized patterns across perception and imagery that were distinct for the two expert groups: they involved frontal cortex for the cinematographers and temporal cortex for the sound designers. Notably, the mental representations of film clips and sounds of cinematographers contained information that went beyond modality-specificity. We were able to successfully decode the implicit presence of film genre from brain activity during mental imagery in cinematographers. The results extend existing neuroimaging literature on expertise into the domain of mental imagery and show that experience in visual versus auditory imagery can alter the representation of information in modality-specific association cortices.

KW - MVPA

KW - fMRI

KW - Mental imagery

KW - Film

KW - Expertise

KW - Naturalistic stimuli

KW - EARLY VISUAL-CORTEX

KW - HUMAN AUDITORY-CORTEX

KW - WORKING-MEMORY

KW - FUNCTIONAL-ANATOMY

KW - CORTICAL ACTIVITY

KW - HESCHLS GYRUS

KW - NEURAL BASIS

KW - MINDS IMAGE

KW - FMRI

KW - AREAS

U2 - 10.1016/j.neuroimage.2016.01.043

DO - 10.1016/j.neuroimage.2016.01.043

M3 - Article

VL - 129

SP - 428

EP - 438

JO - Neuroimage

JF - Neuroimage

SN - 1053-8119

ER -