Hacker News Reader

Comment by mellosouls | Hacker News Reader

In the not too distant future we may see integrations with always-on recording devices (yes, I know, shudder) transcribing our every conversation and interaction and incorporating the text in place of the current custom-corpus style addenda to LLMs to give a truly personal and social skew to the current capabilities in the form of automatically-compiled memories to draw on.

View on HN

To me, the value of a local-LLM is that it can hold my life's notes and i d talk to it as if it was my alter ego until old age. One could say, it's the kind of "soul" that outlasts us

You know what's funny, that episode of black mirror about that I thought was so unbelievable when I saw it

holy sh*t. that's so true! that could definitely be possible.

besides the synthetic body, we have the text interaction, the text-to-speech in a persons voice, and avatar generation/deep fakes. almost the entirety of that episode is available today, which i didn't believe was even ten years away when i saw it.

referring to s2e1: Be Right Back

it really asks great questions about image/reality too

Imagine training a GPT on your own whatsapp/fb/instagram/linked/emails conversations: all the conversations, posts. A huge part of our life is already happening online, and the conversations with it. It is not too much work to simply take that data and retrain GPT.

i initially tried to download a bunch of my reddit comments and try to get it to write "in my style" but i think i need to actually go through the fine tuning process to do that well.

what is the name of that episode?

I actually meant Be Right Back, s2e1. https://www.imdb.com/title/tt2290780/

"After learning about a new service that lets people stay in touch with the deceased, a lonely, grieving Martha reconnects with her late lover."

I am more and more convinvced that we are living in a timeline described in https://en.wikipedia.org/wiki/Accelerando (at least the first part and I would argue that we have it worse)

Look at David Shapiro's project on GitHub, not Raven but the other one that is more fleshed out. He already does the summarization of dialogue and retrieval of relevant info using the OpenAI APIs I believe. You could combine that with the Chrome web speech or speech-to-text API which can stay on continuously. You would need to modify it a bit to know about third party conversations and your phone would run out of battery. But you could technically make the code changes in a day or two I think.

I'm not too sure Siri/ Google Assistant doesn't do this already, but to serve us ads.

If Siri or Google were doing this, it would have been whistleblown by someone by now.

As far I as understand, Siri works with a very simple "hey siri" detector that then fires up a more advanced system that verifies "is this the phone owner asking the question" before even trying to answer.

I'm confident privacy-sensitive engineers would notice and flag any misuse;

That would also be crazy expensive and hard to do well. They struggle with current speech reco that’s relatively simple, and can’t do this more complex always listening thing at high accuracy and identifying relevant topics worth serving an ad on even if they wanted to and it wasn’t illegal. This is always the thing people would say for Alexa and Facebook too. The reality is people see patterns where there aren’t any or forget they searched for something that they also talked about and that’s what actually drove the specific ad they saw.

A high-end phone is quite capable of doing automatic speech recognition continuously, as well as NLP topic analysis. The last years voice activity detection has moved down into the microphone itself, to enable ultra low power always-listening functionality. It then triggers further processing of the potentially-containing-speech audio. Modern SoC have dedicated microcontroller/microprocessor cores that can do further audio analysis, without involving the main cores or the OS. Typically deciding if something is speech or not. Today this is usually doing Keyword Spotting (hey Alexa etc). These are expected to get access to neural accelerators chips, which will further improve power efficiency and eventually having sufficient memory and computer to run speech recognition. So the technological barriers are falling one by one.

I worked on Alexa (and Cortana before that). I’m aware of the current tech. The tech barriers for doing this at high accuracy cheaply are still very much there.

Which barriers do you considered to be the most problematic (and why)? I think that using todays CPU/GPU on a modern phone, one would be able to run a model capable of a Word Error Rate of under 10%. I mean, it would take up maybe 1GB of RAM. And battery life would be impacted, but still a quite usable phone? And that seems like it would give workable quality data for some topic / user modelling? I am assuming one would limit triggers to cases where the person speaking is within say 2 meters of the phone, which seems ok limitation for a phone - ulike for a home device like Alexa).

They're not. A breach of trust at that level would kill the product instantly.

Call me jaded but I don't believe that anymore. They might lose 20%. Maybe that's enough to kill but I honestly believe people would just start rolling with it

> I’m not too sure Siri/ Google Assistant doesn’t do this already, but to serve us ads.

If it did, traffic analysis would probably have revealed it.

I talked to an Amazon Echo engineer about how the sound recording works. They said there is just enough hardware on the device to understand "hello Alexa" and then everything else is piped to the cloud.

Currently, ML models are too resource intensive ($$) for always on-recording.

Reply on news.ycombinator.com