Hacker News Reader

Comment by senkora | Hacker News Reader

It is possible to synthesize an English voice with a 1.5MB model using http://cmuflite.org/ or some of the Apple VoiceOver voices, which is just crazy to me. Most of the model is diphone samples for pairs of phonemes.

I don't know of any way to go smaller than that with software. I tried, but it seems like a fundamental limit for English.

View on HN

I don't know of any way to go smaller than that with software. I tried, but it seems like a fundamental limit for English.

If you include "robotic" speech, then there's https://en.wikipedia.org/wiki/Software_Automatic_Mouth in a few tens of KB, and the demoscene has done similar in around 1/10th that. All formant synths, of course, not the sample-based ones that you're referring to.

SAM is awful but at the same time tantalisingly close (one of the demos of it apparently draws on a great reverse engineered version by Sebastian Macke and refactoring efforts by a couple of others including me - I spent too many hours listening to SAM output...) - especially when comparing to the still awful Festival/Flite models -, that I keep wanting to see what a better generic formant synth used as constraint on an ML model would produce.

That is, instead of allowing a generic machine learning model to output unconstrained audio, train it on the basis of letting it produce low bitrate input/control values for a formant synth instead, and see just how small you can push the model.

Me as a blind dude that is listening to a sinth all the time I don't cair about the size of the thing I just want good SQ.

That's fair, but those two are in constant tension. We can do very good speech if we ship gigs of samples. Better quality smaller models make it easier to get better quality speech in more places.

You probably could by what I call the "eastern-european method." Record one wave period of each phoneme, perhaps two for plosives, downsample to 8 or 11 kHz 8 bit, and repeat that recording on-the-fly enough times to make the right sound. If you're thinking "mod file", you're on the right track.

For phonetically simple languages, such a system can easily fit on a microcontroller with kilobytes of RAM and a slow CPU. English might require a little bit more on the text-to-phoneme stage, but you can definitely go far below 1MB.

That's effectively what eSpeak-ng is doing.

For the CMU flite voices they represent the data as LPC (linear predictive coding) data with residual remainder (residual excited LPC). The HTS models use simple neural networks to predict the waveforms -- IIRC, these are similar to RNNs.

The MBROLA models use OLA (overlapped add) to overlap small waveform samples. They also use diphone samples taken from midpoint to midpoint in order to create better phoneme transitions.

Most of the links there are dead.

That's a descendant of Festival Singer, which was well respected in its day.

What's a current practical text-to-speech system that's open source, local, and not huge?

Flite.

That's another spinoff of Festival Singer. What's good from the LLM era?

Depending on your definition of "huge", you might find Piper TTS fits your requirements: https://github.com/rhasspy/piper

The size of the associated voice files varies but there are options that are under 100MB: https://huggingface.co/rhasspy/piper-voices/tree/main/en

The speak & spell did it with a 32kB [edit, previously incorrectly wrote 16kB] ROM and a TMS0280.

I always wanted something like Flite but for Spanish.

Reply on news.ycombinator.com