Hacker News Reader

Comment by retrac | Hacker News Reader

Deaf relay is done by transcriptionists. The end user relies on deaf relay in a way where automatic systems are not really an option - possibly in a life and death way if someone is calling their pharmacy or doctor or whatever. It's common for the relay agent to use their human intelligence to navigate a phone menu or get the right party on the line, to rephrase and clarify, since typing is so slow. That's actually a great benchmark for when AI will be ready. Can it replace humans for that task? At present absolutely not. (Trust me I've tried the various speech recognizers. Add a little noise or a mumbly person and it can degrade into uselessness. And the AI isn't going to interrupt someone immediately and then to repeat the last phrase.

Anyway. You need to register for relay service with the provider. Using without would be unauthorized.

View on HN

Speaking as someone who's deaf and uses these services a lot: for speech to text, the AI stuff is getting rather good.

I'm not saying it's perfect for every situation, but I have a very high success rate using InnoCaption[0] for captioned phone calls, including to places like restaurants with a lot of noise going on in the background. InnoCaption does both live person and AI-based captioning; since they started offering the AI-based option I've left that on, and I've never had to switch to human operators to continue a conversation.

That said - I'm not deaf from birth (lost my hearing in elementary school), so I voice for myself and that does simplify the process. I have used the old school text-only relay services and that was always such a miserable experience for me that I would crawl over broken glass to avoid making phone calls, especially going through phone trees. That's one area that relay operators still have a major advantage on. IIRC, Google's Pixel phones are supposed to be able to navigate phone trees for you, but since I use iOS I have no personal experience there.

[0] https://www.innocaption.com/

The phone tree stuff on Pixel is decent but nowhere near 100% reliable or robust.

If it hears and understands an automated system speaking out a phone tree, it will start to list the options and you can tap on them. Usually works but often doesn't recognize that a phone tree is happening. Other times it recognizes the phone tree, but mistranscribes the options.

As a non-deaf person, it's a handy UX improvement. But I wouldn't recommend that anyone rely on it.

These services are indeed great for those that need them. I received one or two years ago when I worked at a computer shop. Unfortunately they were always scammers, abusing the system.

Yea, FCC needs to do something about the scammers. They're causing a lot of shops to not accept relay calls because of this.

I can't really understand speech these days without the captions to go with it. But I encounter discrepancies with AI generated captions very often. As in, I heard something and from context I know I'm right and the AI is wrong. With Whisper and other deep learning based speech systems in particular - they can generate very plausible misinterpretations - sounds similar and is grammatically plausible - but not what was said. Of a kind that a person with semantic understanding of what's going on would not make. So I am a little leery of them for that reason. I rely on it every day for generating captioning to video and so on. I don't find any iteration I've tried reliable or comfortable for interactive use.

> I encounter discrepancies with AI generated captions very often. As in, I heard something and from context I know I'm right and the AI is wrong.

I've been noticing this as well. It's becoming a common problem. Also, many times I've noticed that if I hadn't heard the speech being captioned and only had the captioning to go by, I would have had little chance of correctly understanding what was actually said.

[Applause] on YouTube transcripts, short two or three syllable sentence fragments, and absolute nonsense are the only ones I’d be able to reliably detect sans audio. But doubt YouTube captions are state of the art given how poor it is.

I’ve received one call in my life from such a relay service. I worked the phones at a retail shop selling specialty sports equipment. It was a normal call aside from some delays due to the typing and a brief introduction to explain that the person on the call was a middle man. It’s great that these services exist.

Thanks for accepting that.

A lot of stores don't accept them anymore because of scumbag scammers abusing the system, leaving us dependent on a friend or family member to make our calls.

That infuriates me on your behalf. It’s so hard to make something nice to help people.

Not an American, but I would have thought that sort of thing would be against the ADA?

It is. But when the front line person has received a bunch of spam calls they start hanging up when they hear the relay disclaimer at the start of the call, whatever the law says.

Similar issue here in Canada.

I don't know if you're outside of US that's done differently but in US, it's not called "deaf relay", it's just relay services.

The relay service is not for deaf people, it's also for people with speech issues, deafblind, blind, and so on.

> transcriptionists.

Also, not as such in US. It's interpreter (usually for ASL/video call) or operator. Relay service relays messages in two ways; voice the message from the end user and then type from voice to the end user.

AI has to be absolutely perfect to replace humans for these relay calls because as you said, when it comes to medical situations, or legal issues; who would be at fault for miscommunication that led to some issues?

In US, you must also verify your identity to the relay service as federal regulations has step up the identity requirements recently. I had to do a video call with VRS to show my state ID that I'm in the state and registered with the state's relay services.

(I'm not from US)

I do appreciate GP using the term "deaf relay", as this confirmed my understanding of the StackExchange post. The term "relay" is so broad that, without context, I'd assume "relay service" in telecom is whatever relays your call across vendor/state/national service borders, or something.

Have you tried Whisper large? I'm always amazed what it can pick up.

Yes. I use it every day and it's what I was benchmarking against mentally.

Just look at the automatic closed captions on any news or sports broadcast, or enable them on a YouTube video. They are pretty good, but there are still some obvious errors that you'll see within just a few minutes.

My first thought after reading the post was also more or less "hang on - practically usable, local, realtime speech recognition functionality in a 20-30 year old phone? Where is the error?"

Your post explains it very well...

Reply on news.ycombinator.com