Abstract
Speech-to-speech translation (S2ST) converts input speech to speech in another language. A challenge of delivering S2ST in real time is the accumulated delay between the translation and speech synthesis modules. While recently incremental text-to-speech (iTTS) models have shown large quality improvements, they typically require additional future text inputs to reach optimal performance. In this work, we minimize the initial waiting time of iTTS by adapting the upstream speech translator to generate high-quality pseudo lookahead for the speech synthesizer. After mitigating the initial delay, we demonstrate that the duration of synthesized speech also plays a crucial role on latency. We formalize this as a latency metric and then present a simple yet effective duration-scaling approach for latency reduction. Our approaches consistently reduce latency by 0.2-0.5 second without sacrificing speech translation quality.
Original language | English |
---|---|
Title of host publication | Proceedings of INTERSPEECH 2022 |
Publisher | International Speech Communication Association (ISCA) |
Pages | 1771-1775 |
Number of pages | 5 |
Volume | 2022-September |
DOIs | |
Publication status | Published - 2022 |
Event | 23rd Annual Conference of the International Speech Communication Association - Incheon, Korea, Republic of Duration: 18 Sept 2022 → 22 Sept 2022 Conference number: 23 https://www.interspeech2022.org/ |
Publication series
Series | Interspeech |
---|---|
ISSN | 2308-457X |
Conference
Conference | 23rd Annual Conference of the International Speech Communication Association |
---|---|
Abbreviated title | INTERSPEECH 2022 |
Country/Territory | Korea, Republic of |
City | Incheon |
Period | 18/09/22 → 22/09/22 |
Internet address |
Keywords
- speech translation
- text-to-speech
- low-latency