Hacker News Reader

Comment by dist-epoch | Hacker News Reader

Weird way of attacking Microsoft, when this is a feature users will soon want everywhere.

The latest Android update already introduced screen sharing with Gemini. Their web app has that too.

It wont be long until people complaining here about DRM/Microsoft will have an always on AI watching their screen by their own choice.

View on HN

I'm normally not one to attack the messenger and just attack the message, but lay off the crack.

99% of users don't want anything even remotely like this. The thought of a single database (even encrypted) that could contain random login/password information, personal information, etc. and easily exfiltrated by whatever new zero-day of the week is NOT pleasant in the slightest.

> this is a feature users will soon want everywhere

Some users - the less privacy-conscious. Many others (who probably frequent this site) actively do (and will) not.

If I could run the model locally I would do that, but sending screenshots of everything I do + metadata to microsoft is way too much for me because, to start, I don’t want them selling my data to advertisers.

Recall only runs locally - it doesn't send any data off-device, and doesn't work if you don't have an "AI+" chip.

It’s still not fully user-controllable, which is a critical distinction. It remains local-only until Microsoft decides otherwise, and MS can always put in hooks that makes it easy for them to exfiltrate specific data that was technically harvested locally on a per-user or per-demographic basis. The level of trust required is truly extraordinary.

> and MS can always put in hooks that makes it easy for them to exfiltrate specific data that was technically harvested

Just like they always can put hooks into Windows to do the same thing. And Google can put hooks into Android. And Apple into macOS.

With AI that processes periodically-captured screenshots, the threat is an order of magnitude greater. It’s always been possible for companies to indiscriminately copy data, but cost and risk of detection have made doing so an expensive and risky proposition. AI flips that on its head and makes it possible to target individuals and groups with incredible precision and reduces the volume of data that needs to be transmitted to almost nothing.

> MS can always put in hooks that makes it easy for them to exfiltrate specific data

MS can issue an update any day to just copy all drives you currently have attached to Azure, if we're going to put on our tin foil hats.

Er, isn't that how onedrive works? It's not a "tin foil hat" move to point out that that's exactly what does happen to users who aren't paying attention and opting out, and it's equally valid to extrapolate that they might continue similar behaviors with new features.

No, OneDrive doesn't upload all data from all attached drives.

You're right, it only started uploading people's most important data without clear and deliberate setup, not all their data.

That's more than enough to make these worries not tinfoil hat.

It's also opt-in.

It is fully user controllable and allways was.

Not in the fullest sense. It can be turned off (for now), but its behavior once enabled is subject entirely to Microsoft’s whims.

Full user control is what you’d have if e.g. you were running a FOSS Recall analogue powered by the local LLM of your choice on some flavor of Linux. That setup will only ever do what the user intends it to and barring supply chain exploits, cannot go rogue.

> That setup will only ever do what the user intends it to

There are tons of times I've had stuff running on a Linux box do things I didn't intend it to do. Often even with software I wrote!

I guess you're one of those people who only ever writes perfect code that exactly does what you intend the first time.

Alright, let’s rephrase it to be a bit more pedantry-proof: A Linux-based FOSS analogue of Recall built on a self-hosted LLM of the user’s choice will never actively undermine the user’s privacy or sell them out as long as they’ve vetted all software involved.

There’s always the possibility of vulnerabilities and exploits but that’s not the point.

The behavior of Windows is subject to Microsoft's whims. Microsoft could just as easily have pushed an update to Windows XP that exfiltrated sensitive screenshots too. The existence of a user-facing feature changes nothing at all here

See again: Raymond Chen's "other side of an airtight hatchway" analogy. Microsoft already have ring 0 access to your machine

it doesn't send any data off-device... YET.

"Free cloud storage for your recalls, we will only scan it for bad thoughts not for good thoughts, we promise!"

Oh, is it that lightweight? I’ll look forward to the open source equivalent then. I try not to rope myself into services that may change for the worse later, but I’ve got nothing against the idea.

Reply on news.ycombinator.com