DiffSinger: Diffusion Acoustic Model for Singing Voice Synthesis.
2021
Singing voice synthesis (SVS) system is built to synthesize high-quality and
expressive singing voice, in which the acoustic model generates the acoustic
features (e.g., mel-spectrogram) given a music score. Previous singing acoustic
models adopt simple loss (e.g., L1 and L2) or generative adversarial network
(GAN) to reconstruct the acoustic features, while they suffer from
over-smoothing and unstable training issues respectively, which hinder the
naturalness of synthesized singing. In this work, we propose DiffSinger, an
acoustic model for SVS based on the diffusion probabilistic model. DiffSinger
is a parameterized Markov chain which iteratively converts the noise into
mel-spectrogram conditioned on the music score. By implicitly optimizing
variational bound, DiffSinger can be stably trained and generates realistic
outputs. To further improve the voice quality, we introduce a \textbf{shallow
diffusion mechanism} to make better use of the prior knowledge learned by the
simple loss. Specifically, DiffSinger starts generation at a shallow step
smaller than the total number of diffusion steps, according to the intersection
of the diffusion trajectories of the ground-truth mel-spectrogram and the one
predicted by a simple mel-spectrogram decoder. Besides, we train a boundary
prediction network to locate the intersection and determine the shallow step
adaptively. The evaluations conducted on the Chinese singing dataset
demonstrate that DiffSinger outperforms state-of-the-art SVS work with a
notable margin (0.11 MOS gains). Our extensional experiments also prove the
generalization of DiffSinger on text-to-speech task.
Keywords:
- Correction
- Source
- Cite
- Save
- Machine Reading By IdeaReader
31
References
6
Citations
NaN
KQI