Cortical Tracking of Surprisal during Continuous Speech Comprehension

Hugo Weissbart, Katerina D Kandylaki, Tobias Reichenbach

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Speech comprehension requires rapid online processing of a continuous acoustic signal to extract structure and meaning. Previous studies on sentence comprehension have found neural correlates of the predictability of a word given its context, as well as of the precision of such a prediction. However, they have focused on single sentences and on particular words in those sentences. Moreover, they compared neural responses to words with low and high predictability, as well as with low and high precision. However, in speech comprehension, a listener hears many successive words whose predictability and precision vary over a large range. Here, we show that cortical activity in different frequency bands tracks word surprisal in continuous natural speech and that this tracking is modulated by precision. We obtain these results through quantifying surprisal and precision from naturalistic speech using a deep neural network and through relating these speech features to EEG responses of human volunteers acquired during auditory story comprehension. We find significant cortical tracking of surprisal at low frequencies, including the delta band as well as in the higher frequency beta and gamma bands, and observe that the tracking is modulated by the precision. Our results pave the way to further investigate the neurobiology of natural speech comprehension.

Original languageEnglish
Pages (from-to)155-166
Number of pages12
JournalJournal of Cognitive Neuroscience
Volume32
Issue number1
Early online date3 Sep 2019
DOIs
Publication statusPublished - Jan 2020

Keywords

  • BRAIN POTENTIALS
  • DYNAMICS
  • LANGUAGE
  • OSCILLATIONS
  • RESPONSES
  • SEMANTIC CONTEXT
  • SENTENCES
  • SENTENTIAL CONSTRAINT
  • TIME
  • WORD

Cite this

Weissbart, Hugo ; Kandylaki, Katerina D ; Reichenbach, Tobias. / Cortical Tracking of Surprisal during Continuous Speech Comprehension. In: Journal of Cognitive Neuroscience. 2020 ; Vol. 32, No. 1. pp. 155-166.
@article{0c01723b1149407cb559ff7c19a4f4af,
title = "Cortical Tracking of Surprisal during Continuous Speech Comprehension",
abstract = "Speech comprehension requires rapid online processing of a continuous acoustic signal to extract structure and meaning. Previous studies on sentence comprehension have found neural correlates of the predictability of a word given its context, as well as of the precision of such a prediction. However, they have focused on single sentences and on particular words in those sentences. Moreover, they compared neural responses to words with low and high predictability, as well as with low and high precision. However, in speech comprehension, a listener hears many successive words whose predictability and precision vary over a large range. Here, we show that cortical activity in different frequency bands tracks word surprisal in continuous natural speech and that this tracking is modulated by precision. We obtain these results through quantifying surprisal and precision from naturalistic speech using a deep neural network and through relating these speech features to EEG responses of human volunteers acquired during auditory story comprehension. We find significant cortical tracking of surprisal at low frequencies, including the delta band as well as in the higher frequency beta and gamma bands, and observe that the tracking is modulated by the precision. Our results pave the way to further investigate the neurobiology of natural speech comprehension.",
keywords = "BRAIN POTENTIALS, DYNAMICS, LANGUAGE, OSCILLATIONS, RESPONSES, SEMANTIC CONTEXT, SENTENCES, SENTENTIAL CONSTRAINT, TIME, WORD",
author = "Hugo Weissbart and Kandylaki, {Katerina D} and Tobias Reichenbach",
year = "2020",
month = "1",
doi = "10.1162/jocn_a_01467",
language = "English",
volume = "32",
pages = "155--166",
journal = "Journal of Cognitive Neuroscience",
issn = "0898-929X",
publisher = "MIT Press",
number = "1",

}

Cortical Tracking of Surprisal during Continuous Speech Comprehension. / Weissbart, Hugo; Kandylaki, Katerina D; Reichenbach, Tobias.

In: Journal of Cognitive Neuroscience, Vol. 32, No. 1, 01.2020, p. 155-166.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Cortical Tracking of Surprisal during Continuous Speech Comprehension

AU - Weissbart, Hugo

AU - Kandylaki, Katerina D

AU - Reichenbach, Tobias

PY - 2020/1

Y1 - 2020/1

N2 - Speech comprehension requires rapid online processing of a continuous acoustic signal to extract structure and meaning. Previous studies on sentence comprehension have found neural correlates of the predictability of a word given its context, as well as of the precision of such a prediction. However, they have focused on single sentences and on particular words in those sentences. Moreover, they compared neural responses to words with low and high predictability, as well as with low and high precision. However, in speech comprehension, a listener hears many successive words whose predictability and precision vary over a large range. Here, we show that cortical activity in different frequency bands tracks word surprisal in continuous natural speech and that this tracking is modulated by precision. We obtain these results through quantifying surprisal and precision from naturalistic speech using a deep neural network and through relating these speech features to EEG responses of human volunteers acquired during auditory story comprehension. We find significant cortical tracking of surprisal at low frequencies, including the delta band as well as in the higher frequency beta and gamma bands, and observe that the tracking is modulated by the precision. Our results pave the way to further investigate the neurobiology of natural speech comprehension.

AB - Speech comprehension requires rapid online processing of a continuous acoustic signal to extract structure and meaning. Previous studies on sentence comprehension have found neural correlates of the predictability of a word given its context, as well as of the precision of such a prediction. However, they have focused on single sentences and on particular words in those sentences. Moreover, they compared neural responses to words with low and high predictability, as well as with low and high precision. However, in speech comprehension, a listener hears many successive words whose predictability and precision vary over a large range. Here, we show that cortical activity in different frequency bands tracks word surprisal in continuous natural speech and that this tracking is modulated by precision. We obtain these results through quantifying surprisal and precision from naturalistic speech using a deep neural network and through relating these speech features to EEG responses of human volunteers acquired during auditory story comprehension. We find significant cortical tracking of surprisal at low frequencies, including the delta band as well as in the higher frequency beta and gamma bands, and observe that the tracking is modulated by the precision. Our results pave the way to further investigate the neurobiology of natural speech comprehension.

KW - BRAIN POTENTIALS

KW - DYNAMICS

KW - LANGUAGE

KW - OSCILLATIONS

KW - RESPONSES

KW - SEMANTIC CONTEXT

KW - SENTENCES

KW - SENTENTIAL CONSTRAINT

KW - TIME

KW - WORD

U2 - 10.1162/jocn_a_01467

DO - 10.1162/jocn_a_01467

M3 - Article

C2 - 31479349

VL - 32

SP - 155

EP - 166

JO - Journal of Cognitive Neuroscience

JF - Journal of Cognitive Neuroscience

SN - 0898-929X

IS - 1

ER -