Microsoft presents VALL-E speech synthesis research

Microsoft researchers describe VALL-E in a paper submitted on 5 January: a system that can generate speech in an unfamiliar speaker’s voice from a three-second recording, according to their experiments.

Photo: efes, 2015. Original photograph via Wikimedia Commons. CC0 1.0. Recompressed; responsive crops.

It also takes the recording’s transcript and the text to speak. The approach predicts audio codes, then decodes them into sound. Training used 60,000 hours of English audiobooks.

The authors report better listener-rated naturalness and voice similarity than YourTTS on LibriSpeech and VCTK. The reported gains come from the team’s own tests.

The paper acknowledges missed or repeated words and limited accent coverage. It also flags impersonation risks and proposes detection tools as a possible response.