Text-to-Audio Generation Using Instruction Tuned LLM and Latent Diffusion Model

Excellent. Some of the theory here goes back to Oct/2021 and beyond [1].

The riffusion.com [2] guys made this practical. Also, my video showing high-level overview and examples [3].

3. Riffusion high-level overview: https://youtu.be/olkLVGcvib8

Could it generate speech with enough training?

And what is SOTA for TTS these days?

https://google-research.github.io/seanet/audiolm/examples/ is capable of generating speech that is syntactically correct and semantically coherent (without ever being conditioned on text).

Anyone have the actual Gradio space link? The one that is on the site is broken.

interesting, seems like it could take the job of a movie looper with very little additional tuning.