I have no recommendations, but I'm curious if someone has tried to train a TTS on the data made by one of the commercial services. Generating data would be very cheap, labels perfect, and there would be less noise than in the human datasets.
Why do that instead of using an existing dataset like Common Voice?
Since you would be learning from another AI and not humans, there would be much less variation in the way words are pronounced and you would have a lot more data.
What's the goal? What would be the benefit of training on a single TTS speaker
To have your own AWS TTS, which you can run offline and for free.