Hacker News Reader

Comment by dom96 | Hacker News Reader

This really makes me want to build a Amazon Echo/Google Nest/etc replacement that's open hardware, open source and most importantly recognises voice completely offline. I find that I don't use these smart devices for much more than setting timers anyway so this seems like an easy project.

I just wonder what system requirements Whisper has and whether there are open source voice recognition models that are specifically built for embedded devices.

View on HN

I really want all this too. The smallest model is ~80mb and the largest is 3gb. Not sure about system requirements yet; but models that small suggest this may be doable locally on a single board computer.

Edit: According to this comment[0] the base model runs in real time on an M1 CPU. The tiny model apparently decodes an audio file twice as fast. These are promising results.

[0] https://news.ycombinator.com/item?id=32927360#32929739

I'd be interested to see how well it performs on something like an RPi. M1 is pretty beefy.

For an offline (non-streaming) model, 1x realtime is actually kind of bad, because you need to wait for the audio to be available before you can start processing it. So if you wait 10 seconds for someone to finish speaking, you won't have the result until 10 seconds after that.

You could use really small chunk sizes and process them in a streaming fashion, but that would impact accuracy, as you're significantly limiting available context.

To be more precise the original comment said "M1 Max" which in itself is significantly beefier a bare "M1"

This is only one side of the coin, you still need really good models for Speech Synthesis and then be able to have it all working in almost real time, ideally locally on device.

As far as TTS goes, Mycroft.ai[0] has released a decent offline one.

[0]https://mycroft.ai/

pico2wave with "-l=en-GB" option to get the British lady voice is pretty decent (way better than the other voices it does for some reason).

I'm pretty sure mycroft sends your speech snippets to Google for processing so it's not exactly offline.

https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/customiz...

I'm currently trying to setup a deepspeech server on my raspberry pi to see if it works ok for commanding spotify.

Edit: just realised you said `TTS` not `STT`

Are you thinking about reimplementing Mycroft?

The Mycroft has done a lot of cool and important work in the field to ship an actual personal assistant product (stuff like wake word detection).

hah, of course someone had the idea already and executed on it. But yeah, basically that but without the screen (probably would go a long way to decrease the cost, $299 is pretty steep for such a device)

One thing they don't touch much on is the STT, as they use models from third parties. You could definitely do something that utilizes this model and then feeds the tokens to some of their parsing code. I've been working on something similar to this, but burned out around adding the STT portion [0].

[0]: https://github.com/Sheepybloke2-0/trashbot - It was called trashbot because the final implementation was going to look like oscar the grouch in a trashcan displaying the reminders.

Well, you can always install Mycroft on a Pi, or on your computer.

Almond is also interesting as a voice assistant, though I think it doesn't perform speech recognition itself.

Reply on news.ycombinator.com