Model Card for Kyutai TTS See also the pre print research paper, the project page, the Colab example, the GitHub repository, and the repository of voices. This is a model for streaming text to speech (TTS). Unlike offline text to speech, where the model needs the entire text to produce the audio, our model starts to output audio as soon as the first few words from the text have been given as input. This model is actually 1.8B parameters, not 1.6B as the name might suggest. Model Details The model architecture is a hierarchical Transformer that consumes tokenized text and generateds audio tokenized by Mimi, see the Moshi paper. The frame rate is 12.5 Hz and each audio frame is represented by 32 audio tokens, although you can use less tokens at inference time for faster generation. The backbone model is 1B parameters, and the depth transformer is 600M parameters and uses partial weight sharing similar to Hibiki. The audio is shifted by 16 steps (1.28 sec.) with respect to the text, and the model uses an acoustic/semantic delay of 2. Model Description Kyutai TTS is a decoder only model for streaming speech to text. It leverages the multistream architecture of Moshi to model text strea…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy