Real-time Continuous Transcription with Live Transcribe | Hacker News Reader

Real-time Continuous Transcription with Live Transcribe

ai.googleblog.com

105 pointsiron001330 comments

This app is cool and useful, but a major piece of the puzzle is missing.

Communication between a Deaf person and a hearing person is a two-way street, and this tool really only addresses one of those streets.

The tool transcribes the audible speech of the hearing person, allowing the Deaf person to read the transcription.

But if the Deaf person wants to sign a response, they're out of luck. Instead, they need to type their response on the device.

That's OK, but in a "this'll make do in a pinch" kind of way. The ideal is that both the Deaf person and the Hearing person are able to communicate without typing.

There is some pretty cool research around signing-to-text translation -- Matt Huenerfauth https://huenerfauth.ist.rit.edu is doing some really interesting stuff, for example -- but as far as I know it's not ready for prime time.

>Communication between a Deaf person and a hearing person is a two-way street, and this tool really only addresses one of those streets.

Could have not said it better myself.

Repairing one lane of a two way street is still useful. :)

As a deaf person, I use https://www.ava.me/ most of the time. It allows me to host a group conversation and have speakers labeled if everyone else uses the app but that's a chore to set up sometimes. It's also quite pricey at $30 per month and some months I use it for less than a hour. It can voice what you type if you want, but I normally voice for myself. I also used to use Keep Notes & the 'mic' icon in a pinch, but that would turn itself off if it didn't pick up any speech in some time.

There is a keyboard button on the toolbar (portrait mode). When you press it, a keyboard shows up so that the user can communicate back to the speaker.

I read about it and tried it out, and I couldn't figure out how to use it. After typing my text, I kept looking for the button to trigger text to speech.

But, it just dawned on me that the idea is to type something and then show it to the other person. Which works, it just wasn't what I was expecting.

Ah yes. Speech to text would be cool, but I guess the speech would get heard by the app again.

> This app is cool and useful, but a major piece of the puzzle is missing

While I agree pieces are missing when sign communication is needed, just this piece can still life changing for the hard of hearing (who vastly outnumber the deaf). This is for my mother in-law who stopped going to group lunches because she felt so left out, even with hearing aids. If this app works as advertised, there are going to be tablets mounted on the walls running it. And maybe a phone adapter.

Although there is a lot of hype around AI/ML, there are a ton of pragmatic things coming out of the research much like this Live Transcription by Google that I think will be incredibly helpful to people around the world.

The thing I am most excited about is that most of this work is being done in the open, in the very least much of this is being open sourced by Google, Facebook, and other giants. For all the heat they have been taking lately I do think they deserve to be applauded for this and Of course this is mostly happening in order to sell us cloud services but it cannot be understated how helpful this will be to many around the world.

Another thing I’m excited about is what I can build with these things, and even more what others will create with these models as building blocks in their applications.

As an example, and complete self promotion here, I was able to use some open source models by Google on TensorFlow to build a cross platform App that can read Articles to you using these neural networks. The amazing thing is I built it mostly on nights and weekends, which shows how easy some of this is to work with now, you can check it out here if you like https://articulu.com

Great job with the app! Which open source model did you use to generate the speech?

Really wish they let people save a transcript.

Agreed.

I love them doing accessibility stuff, and I can see this being useful for specific people in my life. But when I initially read the announcement I thought about using it to transcribe business meetings.

Obviously the accuracy of this won't be "court reporter"-levels, but for casual note taking it would be "good enough."

Google Docs has a voice typing functionality [1]. I used Soundflower [2] to send audio playback to a digital input device, and had Google Docs listen to this device. This worked "okay" for transcribing a blog post from a voice memo.

[1] https://qz.com/work/1087765/how-to-transcribe-audio-fast-and... [2] https://github.com/mattingalls/Soundflower

I think it has some privacy implications. Lot of people will feel uncomfortable if they feel their conversations are being saved in someone else's device or cloud.

This is a design problem. An important one to solve, but still, solvable.

I'll add that I'd feel MORE comfortable with just a transcript being saved than I would an audio file (which is what happens now). Deniability.

The issue is that recording without consent is illegal in many jurisdictions.

Auto-transcribing likely falls in a grey zone, because you could argue that it is being recorded before transcription (which is true).

So GPDR limits having your Google Home device listen in all the time because, well that is creepy, but maybe you want a transcript so let us listen in on your conversations please.

Given that the service is free, you have to know that Google has a plan to monetize that data with someone you don't know yet. When you find out who that is you won't be able to 'undo' the transcriptions you have done in the past.

I also think it is ridiculous that this needs "the cloud". I had pretty good speaker dependent transcription working on an Intel 486 processor, and so find it difficult to believe that given literally 500x the compute power and 2000x the memory on a typical phone you can't do this all locally?

now they just need to bolt it onto Glass or some other HUD. IRL subtitles. Could put a mic array on the frame too, and then get an indication of where the speaker is so you can be alerted even if you aren't facing the person.

Plus, this:

https://www.youtube.com/watch?v=zL6ltnSKf9k

Have the HUD have a dot. When you're wearing the glasses, you aim the red dot on the speaker. The transcription could come just from that speaker.

Just a thought.

What’s the best (currently available) HUD in glasses/sunglasses format?

how long until we start seeing computer vision applications that translate sign language? It's a pretty hard problem, but definitely something that's feasible with the right methods.

Curious: how do you imagine people using this?

Wouldn't this be used in the same way as with any other language? I.e. customer only knows english sign language, but I don't, luckily my cash-register has a camera that can translate sign language to my language?

https://www.blackenterprise.com/kenyan-engineer-creates-high...

It's already there, but the different sentence creation + difficult in understanding moves that can vary a lot makes it not viable now.

What is the current state of open-source CSR/transcription?

Not sure why this requires to be connected to the cloud... ah probably to bill people.

Also not sure why this requires an ad-ware laden OS.

Probably because it wasn't cheap to develop.

Google's ASR (automatic speech recognition) is conducted on their Cloud infrastructure. In fact almost everyone's ASR is done online because the database of speech profiles is huge. Big data is the secret sauce that makes modern ASR superior to previous attempts.

Reply on news.ycombinator.com