README.md

    Tacotron 2 (without wavenet)

    PyTorch implementation of Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions.

    This implementation includes distributed and automatic mixed precision support and uses the LJSpeech dataset.

    Distributed and Automatic Mixed Precision support relies on NVIDIA's Apex and AMP.

    Visit our website for audio samples using our published Tacotron 2 and WaveGlow models.

    Alignment, Predicted Mel Spectrogram, Target Mel Spectrogram

    Pre-requisites

    1. NVIDIA GPU + CUDA cuDNN

    Setup

    1. Download and extract the LJ Speech dataset
    2. Clone this repo: git clone https://github.com/NVIDIA/tacotron2.git
    3. CD into this repo: cd tacotron2
    4. Initialize submodule: git submodule init; git submodule update
    5. Update .wav paths: sed -i -- 's,DUMMY,ljs_dataset_folder/wavs,g' filelists/*.txt
      • Alternatively, set load_mel_from_disk=True in hparams.py and update mel-spectrogram paths
    6. Install PyTorch 1.0
    7. Install Apex
    8. Install python requirements or build docker image
      • Install python requirements: pip install -r requirements.txt

    Training

    1. python train.py --output_directory=outdir --log_directory=logdir
    2. (OPTIONAL) tensorboard --logdir=outdir/logdir

    Training using a pre-trained model

    Training using a pre-trained model can lead to faster convergence
    By default, the dataset dependent text embedding layers are ignored

    1. Download our published Tacotron 2 model
    2. python train.py --output_directory=outdir --log_directory=logdir -c tacotron2_statedict.pt --warm_start

    Multi-GPU (distributed) and Automatic Mixed Precision Training

    1. python -m multiproc train.py --output_directory=outdir --log_directory=logdir --hparams=distributed_run=True,fp16_run=True

    Inference demo

    1. Download our published Tacotron 2 model
    2. Download our published WaveGlow model
    3. jupyter notebook --ip=127.0.0.1 --port=31337
    4. Load inference.ipynb

    N.b. When performing Mel-Spectrogram to Audio synthesis, make sure Tacotron 2 and the Mel decoder were trained on the same mel-spectrogram representation.

    Related repos

    WaveGlow Faster than real time Flow-based Generative Network for Speech Synthesis

    nv-wavenet Faster than real time WaveNet.

    Acknowledgements

    This implementation uses code from the following repos: Keith Ito, Prem Seetharaman as described in our code.

    We are inspired by Ryuchi Yamamoto's Tacotron PyTorch implementation.

    We are thankful to the Tacotron 2 paper authors, specially Jonathan Shen, Yuxuan Wang and Zongheng Yang.

    项目简介

    🚀 Github 镜像仓库 🚀

    源项目地址

    https://github.com/nvidia/tacotron2

    发行版本

    当前项目没有发行版本

    贡献者 17

    全部贡献者

    开发语言

    • Jupyter Notebook 81.2 %
    • Python 18.7 %
    • Dockerfile 0.1 %