How the new Microsoft Recall feature fundamentally undermines Windows security | Hacker News Reader

How the new Microsoft Recall feature fundamentally undermines Windows security

doublepulsar.com

89 pointsspeckx47 comments

I think Recall is a window into the future of computers—but this version isn’t going to be successful. It may not even be possible to succeed with this model at all.

I think a lot of people would consider it a _feature_ if they could simply ask their computer “what was my banking password?” and not understand at all the difference between storing credentials in Recall vs storing them in LastPass.

For me the killer compromise is that 3 months isn’t _nearly_ long enough. This wouldn’t be useful to me until the history extends to more than a decade. I don’t always need that, but for anything less than that I very rarely need the help—it’s at most a convenience, not a new capability.

>if they could simply ask their computer “what was my banking password?”

And the computer would reply "I'm sorry, I have no record of Baking Pastor", based on how well these AI assistants understand me.

As 99.99% of HN comments are based on 5-10 year old data, how recently and which chatbot you asked?

Alexa. This week.

As of yesterday Siri still fails to distinguish “fifteen” and “fifty” about half the time when I set a timer, even when I’m careful about enunciation.

Perhaps a day will come when tools like Siri will ask for a confirmation whether you meant "fifteen" or "fifty" instead of making a guess when they aren't fully certain that they heard you correctly.

Yeah, and any updates to the model that make Recall attractive, cannot go back in time and reanalyze the past to be more useful.

At least they will learn a lot from this, in 6-12 months and perhaps a new generation of NPUs we might have something attractive.

I would consider something like Recall to be a great feature if I were confident that I was the only one who had access to the collected data.

But I'm not confident enough with any system or OS for that -- and I have zero confidence in the likes of Microsoft, Google, OpenAI, Facebook, etc.

That’s really where I think the future of these features lives—in figuring out the trust model and security story for providing it. (Maybe it’s even impossible…)

The only way I can imagine being comfortable with it is if I'm using it on a machine that is permanently air-gapped from the internet.

That complete of a record is extremely sensitive and high-value. The potential damage from having it misused is far too high to take any actual chance with.

It would be great, if I trusted my computer.

If it were entirely local and everything else on my computer was entirely local unless I was actively trying to connect to something external, fine.

But instead everything is being sent to the void and monetized as much as possible while telling me the absolute minimum about it.

I’m highly skeptical of the conclusions here. Many corporations already record periodic screenshots of employee devices. There’s also a bunch of hand waving of “security incident here, therefore hackers will access your machine”.

The crux of the argument boils down to > If you have malware running on your PC for only minutes, you have a big problem in your life now rather than just changing some passwords.

Ok. How often does malware run for just a few minutes? Honestly if there’s malware running on any of my machines, it’s not clear the outcome is any different here. Sure there’s a hypothetical scenario where it’s worse. But it’s also just as likely to grab a copy of my password manager while it has everything unencrypted. And it’s also way more likely to persist for days or months, making access to historical data moot as it’ll just install its own key logger.

> I’m highly skeptical of the conclusions here. Many corporations already record periodic screenshots of employee devices. There’s also a bunch of hand waving of “security incident here, therefore hackers will access your machine”.

In highly specific situations, like customer care, yes. Also these machines tend to be super locked down, no admin rights for the users.

And yes this practice is risky. I managed some PCs for customer care and one of them was taken in for repair by the service desk and replaced, and redeployed without wiping it. It ended up in payroll and when we found out their manager was furious (quite understandably so).

Yes malware can install its own keylogger. But it's an extra step once again.

Hackers are one risk. Authorities another.

It won't take long before the demand access to your stored search data.

Of course only to prevent major crimes like terrorism, child abuse and copyright infringement.

Great!

This OS will, by design, waste 10% of rather limited SSD space, on a feature that I don't need, that may or may not work, and that also introduces a security risk on top of slowing things down.

It's going to be very useful™, as infinite browser history and bookmarks are to some people, I'm sure.

As for me, this sounds like a wonderful way to invent entirely new types of digital clutter.

Hoping there'll be a no-nonsense way to disable this "feature" in the "Pro" OS versions at least, along with Cortana/Copilot.

Likely, this will be illegal, especially if users aren't aware their screens are being recorded. Furthermore that screenshots of activity are stored in the Azure cloud.

Developers building it are probably thinking about Security Considerations, but are getting pushback from product managers. So data leak is imminent.

It's one thing for this data to be held locally and under extreme protection through the OS.

But Azure cloud? How can this be legal?

Azure Cloud isn’t mentioned in the article.

I overtrusted the original commenter. My bad... Thanks for the correction.

The way we treat corporations doing bad things in America:

It's probably not illegal...

And if it is, they're probably not going to get caught...

And if they do, the government probably won't go after them...

And if it does, they're probably not going to be found guilty...

And if they are, they probably won't face any consequences...

And if they do, nobody's going to go to jail.

No the announcement was clear that the storage is local. But that doesn't protect against malware.

Not to mention that, even taking them at their word on that for the sake of argument, storing the data locally is far from sufficient. Can we be sure that the data is only ever processed locally and that the results of that processing stay local?

>> No the announcement was clear that the storage is local.

But I just bought a new computer and need to recall stuff from the old one.

Or, I am using my laptop / tablet computer and need to recall stuff done on my desktop comptuer. (Or similarly, I am using my smartphone and need to recall stuff done on my laptop.)

All they have to do is convince the user of the advantages of sending their recall data to the cloud and the user will agree to do so.

> Furthermore that screenshots of activity are stored in the Azure cloud.

Where does it say that screenshots will go to the cloud?

I was under the impression that Recall was designed to be local-only - all processing done locally, storage local too (the article quotes 25GB of local storage for three months of screenshots).

Microsoft isn't really specific about what is local and what isn't. Sure, they are very specific that the storage of the screenshots is local. What about all the processing? As far as I see it they're far less specific about that.

And who says this will stay local and isn't a small T&C update away from being shared with 'trusted partners'?

The article and the source video note these screenshots are stored locally: "Satya point out the processing and data storage is done locally on the device"

I believe the questionable legality also extends to any kind of video calls taking place on systems with this turned on. If Recall indeed "does not perform content moderation", then having it on would include having screenshots of all video calls.

Especially in many-party consent jurisdictions recording video calls by taking screenshots at a sufficiently high frequency without the explicit consent, or at least option to opt out, of remote parties is probably illegal.

Disclaimer: IANAL and this isn't legal advice.

I would like them to fix file search first. That's what I want to "recall"

To add to this - did you notice the terribly ineffective demo scenario?

In the video, the WSJ reporter searches for the term "Brown leather bag" and ends up finding what appears to be an AI generated / stock image of a brown leather bag inside a PDF that has zero relevant textual content about brown leather bags.

Of all the things a user might be looking for with that search term, it's not going to be this!

>In essence, a keylogger is being baked into Windows as a feature.

The author does realize that windows 10+ keeps a timeline of all your copypasta right? Windows will even upload it to the cloud lol.

The keylogger is already here.

Keylogger > copypasta logger

This is explicitly mentioned in the article too

Wat? I am not aware of this. Does this also happen when you use Windows without an Microsoft account?

Nope it doesn't.

You have to turn on "sync across devices" and have a MS account. One of the many excellent reasons to avoid one. Just in case MS makes a "mistake" and turns this one on by default as well.

The built-in clipboard manager is limited to 25 entries. That's why I'm still using Ditto (also the ability to search).

I'm sure the cloud keeps all copypastas indefinitely, but you have to manually enable clipboard sync for that to happen.

Let's not be disingenuous.

It's funny. I have really wanted a product like this for a while. There is absolutely no way I would consider a non-FOSS program to do this though.

I would not touch this..

But it got thinking, if git ever gets replaced would it be by something akin to this that is constantly monitoring all keystrokes in your repo folder and you never make a commit as such, the ai repo upstream just sorts it out..

Isn’t it what `jj` is doing more or less? (Apart from the AI upstream)

Perhaps the system's secure boot integrity process can confirm with a giant desktop visual indicator, whether Recall is disabled or enabled by policy.

2003, https://en.wikipedia.org/wiki/DARPA_LifeLog

> LifeLog was.. to be "an ontology-based (sub)system that captures, stores, and makes accessible the flow of one person's experience in and interactions with the world in order to support a broad spectrum of associates/assistants and other system capabilities". The objective of the LifeLog concept was "to be able to trace the 'threads' of an individual's life in terms of events, states, and relationships", and it has the ability to "take in all of a subject's experience, from phone numbers dialed and e-mail messages viewed to every breath taken, step made and place gone".

2007 Microsoft Research, https://www.microsoft.com/en-us/research/video/the-microsoft...

> The SenseCam is a personal, wearable camera developed by Microsoft Research in Cambridge, UK, and used as a lifelogging device in projects like MyLifeBits.. is based on wearing the SenseCam for lifelogging of ‘events’ during your day, and generating a fast-forward movie of the event as the memory recall interface.

2010 Microsoft Research, https://www.microsoft.com/en-us/research/publication/now-let...

> Lifelogging technologies can capture both mundane and important experiences in our daily lives, resulting in a rich record of the places we visit and the things we see.. Previous work has demonstrated that Lifelogs can aid recall, but that they do many other things too. They can help us look back at the past in new ways, or to reconstruct what we did in our lives, even if we don’t recall exact details. Here we extend the notion of Lifelogging to include locational information.

https://www.microsoft.com/en-us/research/project/mylifebits/

> MylifeBits is a lifetime store of everything. It is the fulfillment of Vannevar Bush’s 1945 Memex vision including full-text search, text & audio annotations, and hyperlinks. The book Total Recall.. is the culmination of our thoughts regarding MyLifebits and the larger CARPE research agenda .. Gordon Bell has captured a lifetime’s worth of articles, books, cards, CDs, letters, memos, papers, photos, pictures, presentations, home movies, videotaped lectures, and voice recordings and stored them digitally. He is now paperless, and is beginning to capture phone calls, IM transcripts, television, and radio.

HN ranking history for this thread: https://hnrankings.info/40433884/

MicroSoft, as usual, being MediocreSoft. At scale

Brought to you by high-level decision-making involving survivors zero of domestic abuse.

Asking for a friend. Does this mean the end of incognito mode on your browser?

Incognito mode was the end of incognito mode.

Only for Chromium-based browsers.

How do you opt out?

How long before governments require all your data to be shared with them "for safety"?

How long before companies get to mine your data for advertising and "training better AI / ML assistants"?

How long before criminals get your data for hacking, extortion, etc. ?

How long before you cannot opt out?

Who is this feature really supposed to serve?

> How long before governments require all your data to be shared with them "for safety"?

This is a good point IMO.

If you'd tell someone in the 90s that the government (or a corpo on their behalf) would scan all their drives they'd be laughing at you.

Now when everyone uses OneDrive and Dropbox this has become "normal".

I can be wrong on this, but if the screenshots are being uploaded, they can be used to train AI models on how to effectively use the computer and navigate. Copilot will then become better at doing things on autopilot and Microsoft will profit.

Reply on news.ycombinator.com