Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation

Thai-Son Nguyen; Sebastian Stüker; Jan Niehues; Alex Waibel

doi:10.1109/ICASSP40776.2020.9054130

Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation

Thai-Son Nguyen^*, Sebastian Stüker, Jan Niehues, Alex Waibel

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

Abstract

Sequence-to-Sequence (S2S) models recently started to show state-of-the-art performance for automatic speech recognition (ASR). With these large and deep models overfitting remains the largest problem, outweighing performance improvements that can be obtained from better architectures. One solution to the overfitting problem is increasing the amount of available training data and the variety exhibited by the training data with the help of data augmentation. In this paper we examine the influence of three data augmentation methods on the performance of two S2S model architectures. One of the data augmentation method comes from literature, while two other methods are our own development - a time perturbation in the frequency domain and sub-sequence sampling. Our experiments on Switchboard and Fisher data show state-of-the-art performance for S2S models that are trained solely on the speech training data and do not use additional text data.

Original language	English
Title of host publication	ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher	IEEE
Pages	7689-7693
Number of pages	5
ISBN (Electronic)	978-1-5090-6631-5
ISBN (Print)	978-1-5090-6632-2
DOIs	https://doi.org/10.1109/ICASSP40776.2020.9054130
Publication status	Published - 2020
Event	45th International Conference on Acoustics, Speech, and Signal Processing - Online, Barcelona, Spain Duration: 4 May 2020 → 8 May 2020 Conference number: 45 https://2020.ieeeicassp.org/

Conference

Conference	45th International Conference on Acoustics, Speech, and Signal Processing
Abbreviated title	ICASSP 2020
Country/Territory	Spain
City	Barcelona
Period	4/05/20 → 8/05/20
Internet address	https://2020.ieeeicassp.org/

Keywords

sequence-to-sequence
self-attention
data augmentation
speed perturbation
sub-sequence
Data Augmentation
Self-attention
Sequence-to-sequence
Speed Perturbation
Sub-sequence

Access to Document

10.1109/ICASSP40776.2020.9054130

Cite this

@inproceedings{a3a1c015ef8949928bfc9ac53f0e22ff,

title = "Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation",

abstract = "Sequence-to-Sequence (S2S) models recently started to show state-of-the-art performance for automatic speech recognition (ASR). With these large and deep models overfitting remains the largest problem, outweighing performance improvements that can be obtained from better architectures. One solution to the overfitting problem is increasing the amount of available training data and the variety exhibited by the training data with the help of data augmentation. In this paper we examine the influence of three data augmentation methods on the performance of two S2S model architectures. One of the data augmentation method comes from literature, while two other methods are our own development - a time perturbation in the frequency domain and sub-sequence sampling. Our experiments on Switchboard and Fisher data show state-of-the-art performance for S2S models that are trained solely on the speech training data and do not use additional text data.",

keywords = "sequence-to-sequence, self-attention, data augmentation, speed perturbation, sub-sequence, Data Augmentation, Self-attention, Sequence-to-sequence, Speed Perturbation, Sub-sequence",

author = "Thai-Son Nguyen and Sebastian St{\"u}ker and Jan Niehues and Alex Waibel",

note = "DBLP's bibliographic metadata records provided through http://dblp.org/search/publ/api are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.; 45th International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 ; Conference date: 04-05-2020 Through 08-05-2020",

year = "2020",

doi = "10.1109/ICASSP40776.2020.9054130",

language = "English",

isbn = "978-1-5090-6632-2",

pages = "7689--7693",

booktitle = "ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

publisher = "IEEE",

address = "United States",

url = "https://2020.ieeeicassp.org/",

}

Nguyen, T-S, Stüker, S, Niehues, J & Waibel, A 2020, Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation. in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 7689-7693, 45th International Conference on Acoustics, Speech, and Signal Processing, Barcelona, Spain, 4/05/20. https://doi.org/10.1109/ICASSP40776.2020.9054130

Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation. / Nguyen, Thai-Son; Stüker, Sebastian; Niehues, Jan et al.
ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020. p. 7689-7693.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceeding › Academic › peer-review

TY - GEN

T1 - Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation

AU - Nguyen, Thai-Son

AU - Stüker, Sebastian

AU - Niehues, Jan

AU - Waibel, Alex

N1 - Conference code: 45

PY - 2020

Y1 - 2020

N2 - Sequence-to-Sequence (S2S) models recently started to show state-of-the-art performance for automatic speech recognition (ASR). With these large and deep models overfitting remains the largest problem, outweighing performance improvements that can be obtained from better architectures. One solution to the overfitting problem is increasing the amount of available training data and the variety exhibited by the training data with the help of data augmentation. In this paper we examine the influence of three data augmentation methods on the performance of two S2S model architectures. One of the data augmentation method comes from literature, while two other methods are our own development - a time perturbation in the frequency domain and sub-sequence sampling. Our experiments on Switchboard and Fisher data show state-of-the-art performance for S2S models that are trained solely on the speech training data and do not use additional text data.

AB - Sequence-to-Sequence (S2S) models recently started to show state-of-the-art performance for automatic speech recognition (ASR). With these large and deep models overfitting remains the largest problem, outweighing performance improvements that can be obtained from better architectures. One solution to the overfitting problem is increasing the amount of available training data and the variety exhibited by the training data with the help of data augmentation. In this paper we examine the influence of three data augmentation methods on the performance of two S2S model architectures. One of the data augmentation method comes from literature, while two other methods are our own development - a time perturbation in the frequency domain and sub-sequence sampling. Our experiments on Switchboard and Fisher data show state-of-the-art performance for S2S models that are trained solely on the speech training data and do not use additional text data.

KW - sequence-to-sequence

KW - self-attention

KW - data augmentation

KW - speed perturbation

KW - sub-sequence

KW - Data Augmentation

KW - Self-attention

KW - Sequence-to-sequence

KW - Speed Perturbation

KW - Sub-sequence

U2 - 10.1109/ICASSP40776.2020.9054130

DO - 10.1109/ICASSP40776.2020.9054130

M3 - Conference article in proceeding

SN - 978-1-5090-6632-2

SP - 7689

EP - 7693

BT - ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - IEEE

T2 - 45th International Conference on Acoustics, Speech, and Signal Processing

Y2 - 4 May 2020 through 8 May 2020

ER -

Improving Sequence-To-Sequence Speech Recognition Training with On-The-Fly Data Augmentation

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this