Hacker News Reader

Comment by navbaker | Hacker News Reader

Seamless Streaming looks really promising! We just had a new employee start a few months back with profound hearing loss and our company had no idea what to do with him from an accessibility standpoint. They threw out solutions like Dragon, not realizing those solutions are not real-time.

He ended up rolling his own solution by standing up Whisper in one of our clusters and writing a basic front end and API to take his laptop’s mic input and chunk it every few seconds to send to the model and get back text in pseudo-realtime. We got him a pretty beefy Alienware so he wouldn’t be tied to the cluster GPUs. I can’t wait to see what he does with these new models!

View on HN

Just wanted to say you’re a great employer to be so incredibly accommodating to the point you get them an Alienware and let them roll an accessibility solution

We need more support for employees like this!

Second this!

Also, what about Apple’s latest M3 series chips? Are this in the same realm as Alienware in terms of AI compute?

I think generally the consensus of Apple Silicon is that they're great _for a laptop_, but still aren't going to beat a dedicated graphics card + high-end CPU like i9/Ryzen 9. Biggest thing going for apple is the performance/watt though which is critical for a laptop.

I think this is missing the main reason to use Apple Silicon, which is that your dedicated graphics card probably has 24GB or less of RAM, whereas e.g. an M2 Ultra Mac Studio can have 192GB of RAM with a far superior memory bandwidth to anything on x86. This is important because even a "small" LLM like Llama2 13B would require quantization to fit in the 24GB RAM that the dedicated graphics card will give you, whereas the Mac could run Llama2 70B without quantization (at FP16).

Whisper doesn't need that much RAM though.

They definitely are in terms of energy efficiency

They're better than most consumers x86 CPUs but worse than using a GPU. Where they shine is when the ML model can't fit the GPU's VRAM since you have better options for ram size with macs.

>Just wanted to say you’re a great employer to be so incredibly accommodating to the point you get them an Alienware

So gracious, to give a software developer some hardware to run the software they need to work, that costs a whopping nothing more than what other people in the industry get on the average.

>and let them roll an accessibility solution

"You're such a good employer! You let your employee build their own accessibility ramp to the back entrance in their own time, and even got them a mortar spatula to do so!" We need more support for employees like this!

>We need more support for employees like this!

And less support for employers like this.

Not sure why you're being downvoted. Literally the equivalent of building your own ramp.

I didn't downvote, but I considered doing so because nowhere that I saw in GP does it say in his own time, and that's a critical piece of the equation. Hallucinating that datum means they got the argument wrong, and worse they were harshly critical of the company based on that wrongly assumed information.

It reminds me of the Homer Simpson quote, "I don’t mind being called a liar when I’m lying, or about to lie, or just finished lying, but NOT WHEN I’M TELLING THE TRUTH!" I would be equally critical if it was warranted, but when it isn't it's deeply unfair to the accused.

If the person wanted to build their own ramp, and the employer let them do it on the clock, that's a completely different scenario than the employee having to come in during their off-hours to build the ramp just so they can go to work.

Yeah, it wasn’t on his own time. He had a full budget and this was right in line with stuff he had already done research in anyway, so he just went for it.

> He had a full budget and this was right in line with stuff he had already done research in anyway, so he just went for it. reply

So you're saying, not only you didn't pay him extra, but that the company got to benefit from him building the system as it was already in line with your other projects?

Unless his working hours were reduced, he did it in his own time.

Unless his pay was increased, he did it in his own time.

Unless the expectations for him were scaled down, he did it in his own time.

Merely allowing them to hack on a project that is in line of his work is exactly like having someone build their own ramp because "it's already in line with other construction they did on the project".

>nowhere that I saw in GP does it say in his own time

Nowhere did I see that GP said the employee was paid extra, had the expectations on other projects reduced in writing and deadlines shifted, or had his working hours reduced at same pay.

Saying "oh hey, you can work on this during 9-to-5 as long as you get your other shit done on time" means the project was done in his own time.

>Not sure why you're being downvoted. Literally the equivalent of building your own ramp.

Because we are on Hackernews, where everyone likes to think themselves a scrappy startup owner, and not a person with a disability who might need accommodations from one.

Awesome! I love hearing about places making the effort to be inclusive.

As someone who’s profoundly deaf myself, another less technical approach is to install Rogue Amoeba’s Loopback, and use it to pipe audio from a given app into a tool like Google Meet or Otter.ai using the Loopback device as the audio source. This effectively provides real time captions for anything running on your existing machine.

Clever use of Google Meet as a tool! Also, Google Pixel phones now provide realtime captions to any speech playing on the phone (Accessibility > Live Caption). You can also choose a "preferred language" and the captions will be automatically translated to that language from other languages.

Google Chrome [1] also has captioning built-in [2], so this could also work from a plain page that hooks into the loopback device. Pretty sure it's using the same TTS backend that Google Meet uses.

The nice thing about Chrome feature is you can move the caption box around and keep it in the foreground while doing other things, although styling options seem limited (the text might be a little small for some).

[1] on desktop, not sure about mobile

[2] via chrome://settings/accessibility -> Live Caption

>Awesome! I love hearing about places making the effort to be inclusive.

The extent of the effort being getting their employee a slightly-more-expensive-than-average tool that would enable them to do their job better regardless of the disability?

Such inclusive, much pat-yourself-on-the-back, wow.

"We gave our woodworking shop employee a quality saw so that they'd make their own accessibility ramps!"

What would you have them do instead?

>What would you have them do instead?

Why instead? They didn't a bad thing, just nothing beyond what the law requires.

>What would you have them do instead?

A simple answer for you, BTW:

Pay a student $15/hr to transcribe all meetings in real time for the employee.

Or pay money to professionals in the field to set up a solution. You know, people who get paid to do just that.

Jesus.

I have literally been told in job interviews that the company would not be “allowed” to hire me because I’m hearing impaired, so yes, making an effort to support an employee’s disability and their needs is worth recognizing.

>I have literally been told in job interviews that the company would not be “allowed” to hire me because I’m hearing impaired,

That is literally grounds for a lawsuit.

>so yes, making an effort to support an employee’s disability and their needs is worth recognizing.

...if the effort were actually made. Which is not the case.

Merely being compliant with the law is a very low bar, and we should recognize that they did not do anything beyond what the ADA requires them to.

So what? Okay, in the case of a ramp, if you need one you probably are going to have difficulty building one. So pay employee Sally to build it instead, absolutely.

But hearing loss does not impair standing up servers and software. They can pay the employee who probably is the expert at this, the guy with the hearing loss, or go task Emil to go do it to ... avoid 'appearances'?

> They can pay the employee who probably is the expert at this, the guy with the hearing loss,

Aaaaaaaaand who said they paid him?

Not the OP, for sure. It seems like the disabled person set the system up in their own time to help the company communicate with him in meetings.

>or go task Emil to go do it to ... avoid 'appearances'?

They could hire Emil to live-transcribe the speech into text.

They could hire the services of a professional company that would set up speech recognition for their meetings.

They could have done a lot of things.

What they did is nothing. Gee, a software company gave their employee a laptop, how totally unheard of!

Why are people here so willing to pass this off as some act of charity?

We definitely explored using these tools, but we’re constrained by government sponsor rules regarding data protection in our day to day work. We can use ZoomGov captions, but most of the other tools weren’t approved. It looks like Windows 11 has a real time solution of some kind, but we’re still stuck on 10.

I have profound hearing loss and rely on the Windows 11 captions. They are absolutely best in class and head/shoulders above any other automatic captioning I've used. My coworkers have a variety of accents (Hispanic, Eastern European, South African) and it does a great job with all of them.

Additionally, it supports multiple languages (only one at a time sadly), so I also use it for Japanese captions and it's equally great there.

Whisper is pretty good for speech to text, and can be done with in a resource constrained environment. I tried a demo running in a browser using WASM on my phone and even the tiny model is not bad.

Problem with whisper is its not really optimized for command recognition versus general dictation.

- Whisper processes 30 second audio chunks. So if you process 5 seconds of audio you have to pad it out with 25 seconds of silence. Hence a loss of efficiency with wasted CPU / GPU cycles on 25 seconds per chunk in the case above.

- Whisper most likely can't handle hundreds of commands much less than a thousand performantly.

- Whisper doesn't handle short commands very well with a degree of accuracy post processing commands from free dictation utterances.

Command dictation should be weighted higher than general dictation when decoding.

I work with a little under 1500 of commands dragon naturally speaking. DNS is hot garbage as a program despite it has the best accuracy to date with the feature of commands and dictation in one utterance. You get to pay $750 for the privilege m

I've yet to see a free and open source speech recognition engine that can handle both dictation and commands with a high degree of accuracy.

Please please let me know if there's alternatives out there. I would definitely pay to support an open source project like this that focuses on command and dictation.

Most solutions out there that are open source nowadays focus so much on iot command recognition with intents. That's not well suited for controlling your computer with grammars containing voice commands.

Is 30s the input size set by the model, or programs that wrap the model? Is it how it's trained?

It's a property of the model itself.

> Input audio is split into 30-second chunks, converted into a log-Mel spectrogram, and then passed into an encoder.

https://openai.com/research/whisper

Check out Willow! It does essentially this, using WebRTC. It doesn't handle the near-real-time response yet, but it does stream the audio to the server and the change would be pretty minor.

> Check out Willow! It does essentially this, using WebRTC. It doesn't handle the near-real-time response yet, but it does stream the audio to the server and the change would be pretty minor.

Simply voice to text is not what's needed for dictating commands. Unless I can load commands of on the fly and decode utterances that may be useful.

The client would need to be able to send its commands to the server on the fly.

I recommend checking out: https://talonvoice.com/

It's not open source nor does the author intend to open the stack.

Do they need realtime transcription?

Computer: webcaptioner.com Android: Live Transcribe (g.co/livetranscribe) iOS: Live Caption with the 'mic' icon enabled.

Web conferencing: Meet, Zoom, Teams all support realtime CC, which is pretty good.

Y’all should turn that into a product, or at least open source it and get the positive PR + helping others

> Y’all should turn that into a product, or at least open source it and get the positive PR + helping others

There you go. https://github.com/dictation-toolbox/dragonfly

That's very nice of you

>He ended up rolling his own solution

>That's very nice of you

...doesn't compute.

What exactly was nice here?

> We got him a pretty beefy Alienware so he wouldn’t be tied to the cluster GPUs.

Probably this.

As opposed to what, an IBM 286 that everyone else gets?

An "Alienware" is not a Ferrari, those things cost about the same as any other professional-level workstations.

Reply on news.ycombinator.com