Hacker News Reader

Comment by jamez | Hacker News Reader

The main issue is that there was no sniffling symbol in the transcript. And the generated text wouldn't contain it either, because (thankfully) they are pruned out of written interviews that I used to train the model.

View on HN

Thanks for the explanation. I had some assumptions but wasn’t totally sure how this was trained.

How would you make it sniffle in a natural way, too? It’s not a usual speech mannerism, and the way he does it is distinct. I wouldn’t know how to efficiently represent it with text. Maybe it’s easier than I’m imagining.

Just stop the audio output every 5 seconds and include a sniffle sounds, at least that's what it sounds like in real life haha

The TTS model is trained on two things: speech samples and their transcript. If you add enough sniffle-symbols every time a sniffle appears in the speech, I am confident the model would pick up on that. And then you would be able to replicate a sniffle in the generation part. The more time-consuming bit would be to add in the training data for the language model those sniffle-symbols, so that they would be organically added in the text in the text-generation phase.

But seriously, it's not worth it. I think he's a brilliant man with an idiosyncratic speech, let's leave it to that.

I agree, I personally don't hear his sniffles when I'm listening to him intently. It's irrelevant. I was mostly curious if and how, generally speaking, a model could be trained to sniffle. Now that you describe it though it seems fairly clear, so thanks!

in all seriousness how do you not hear them?

I assumed it would difficult to include, it was just something I noticed about him

Reply on news.ycombinator.com