Ask HN: Can you please add transcription to vocal messages in messaging apps? | Hacker News Reader

Ask HN: Can you please add transcription to vocal messages in messaging apps?

I know that there are employees of big tech companies lurking around here. I have a feature request.

Can you please add the ability to see the transcript of a vocal message on messaging apps like Teams, Signal, Telegram, Messenger, Discord, Slack, Whatsapp, etc?

I know that live transcription is already a thing so it must be feasible, right?

This feature would be super useful to see what was said without having to playback a bunch of vocal messages.

I had the idea of doing this myself a long time ago but I basically needed to recreate the entire messaging infrastructure just to add this feature. So I donate the idea here.

View on HN

100% agree, personally I just prefer typing but seems like I am not in the majority. I very often receive audio replies to my typed messages and it’s very frustrating. 70% of the time, I’m not in a position where I can easily listen to the audio vs just reading text. The other 30% of the time I’m simply annoyed as I do not want to be forced to listen to the audio.

I'd like to add to this that the transcription doesn't even need to be 100% accurate to be useful. It'll mostly be used to get the gist of what was said and target more specifically which audio message requires to be re-listened to instead of having to listen to everything again.

You just described the Google Voice voicemail experience

I don't even need a TTS feature, just let me mark my chat as only-text please. People that send audios tend to consume more of my time than people who write because they are making up their mind as they speak.

They seem to think: why waste my time putting my thoughts into words when I can waste everyone else's time while I figure out my ideas?

This seems like such an obvious Appley feature for iMessage, especially now that transcription can be done on-device.

They've just added automatic OCR of photos as well to allow copy and paste from pictures of text.

Yeah, they already do it for voicemails as well which is nice, so it's surprising they wouldn't take the next step and implement it for iMessage voice. Though I will say the voicemail transcriptions aren't amazing in quality.

This would be a big step forward in improving accessibility for the hard of hearing.

There are several Telegram bots for this. But of course very questionable in terms of privacy. I had the idea to write one myself, I found an open source project to host it myself, but I don't remember the name.

Fortunately, the voice messages have largely stopped with my contacts, so I didn't pursue it any furth

Please. It’s so (seemingly) obvious that I don’t understand why it’s not a thing yet. These people aren’t idiots, are they? This is obvious, isn’t it? Is it organizational failure? Or is the technology really, actually, not there yet?

I think you’re overthinking it. It’s certainly feasible, I’m sure it’s been thought of and brought up many times in meetings, It’s just not profitable. The answer, like with many things, is simply money I believe. The compute resources needed to implement & maintain this type of feature at scale, for even a single platform would probably cost many millions of dollars.

So unfortunately, I doubt we will get this anytime soon. They would have to spend huge amounts of money & resources to add this, for essentially no financial gain.

Oh yes, I've forgotten things and looked at my phone and gone "I know it's on here, which app was it?" Make all the messages texts searchable

And whilst we are at it: TRANSCRIBE THE PHONE CALLS TOO!!!

Basically transcribe everything and make it searchable. Wish that was more common for podcasts and videos.

There's a really important point that everybody is missing here. Transcribing phone calls is, in a sense, equivalent to recording a call. Having automatic transcription may not even be legal without letting the other party know.

I would love the opposite: text messages that can be go from text-to-speech. Specially useful when doing chores and those long threads some people write. Or one of those busy groups where everyone is saying something. Just hit play and let the phone read them all. (I use WhatsApp)

Or is this a feature I am unaware on phones?

It is.

I think you can ask Siri or Google Assistant to read your new messages (by accessing the phone notification contents), but I'm not aware of in-app features.

Google Assistent can read out your messages, but i haven't tried out of it can read from one specific group chat.

Siri can read text from apps (messages, email, etc.): "Hey Siri, read the last WhatsApp from John".

It will ask permission for access the first time you try.

Wechat has this feature

Twilio has a few ways to do this, including audio that streams through Twilio.

https://www.twilio.com/speech-recognition

Disclaimer: I work for twilio, but am not representing them in this post

May be this should be an OS feature rather than a Per App feature?

I wish they could transcribe my Voice Mail too. Whenever I have a voice mail, auto call the mail box and transcribe it for me. 99% of it are junk. But just in case there is one real voice message.

FYI Slack does this already for all uploaded audio & video clips, and it works great.

Would be great if they added this for audio clips. Does Slack support vocal messaging? It has been a long time since I used Slack.

Yup, it does

Isn't this a privacy issue with E2E encryption?

If it's done on device is there still a security risk?

Not as long as the transcript is created on device.

Voice-to-text transcription can be done entirely on-device, although I'm not sure what the quality of existing solutions is. This is a barrier but not a showstopper.

The questions then become --- can you trust the system on-device transcription to preserve privacy or can you justify the size of the datasets for transcription if you've got to include it/download it for secure offline use; and can you make it work reasonably well in the languages your users use. (bonus points if it can reasonably detect the spoken language which may not match the written language of the phone).

If it’s sent to the cloud still only kind of, since many of these services aren’t E2EE to begin with.

At least for iOS they already do on-device transcription for voicemails, but not voice messages sent via other means.

I have been using on device transcription on my old Pixel. Here is a demo of the 2 years old technology: https://youtu.be/zfOPF069Xn0

But what I'm looking forward is the new technology in the Pixel 6. Besides much better transcription, caption and translation (see the video completely) it could even solve the real problem by letting people send text messages by speaking into theirs phones with better quality than before.

https://youtu.be/tSAYtwTqppA

Zenly also has this feature

zenly lets you do it already

fdfd

Reply on news.ycombinator.com