Hacker News Reader

Comment by joenot443 | Hacker News Reader

Their "Statement" is remarkably aloof for having brought down flights, hospitals, and 911 services.

"The issue has been identified, isolated and a fix has been deployed."

Maybe I'm misunderstanding what I read elsewhere, but is the machine not BSODing upon boot, prior to a Windows Update service being able to run? The "fix" I see on reddit is roughly:

Workaround Steps:

1. Boot Windows into Safe Mode or the Windows Recovery Environment

2. Navigate to the C:\Windows\System32\drivers\CrowdStrike directory

3. Locate the file matching “C-00000291*.sys”, and delete it.

I'm horrified at the thought of tens of thousands of novice Windows users digging through System32 to delete driver files; can someone set my mind at ease and assure me this will eventually be fixed in an automated fashion?

https://www.crowdstrike.com/blog/statement-on-windows-sensor...

View on HN

> Their "Statement" is remarkably aloof for having brought down flights, hospitals, and 911 services.

Their lawyers certainly won't allow mentioning such dramatic (is "dramatic" appropriate here?) consequences.

It cannot and will not be fixed in an automated fashion.

Of course it can be fixed in an automated fashion; it just requires effort. The machines should have netboot enabled so that new validated operating system images can be pushed to them anyway, so you just write a netboot script to mount the filesystem and delete the file, then tell the netboot server that you're done so it doesn't give you the same script again when it reboots.

It's like two hours of work with dnsmasq and a minimal Linux ISO. The only problem is that much of the work is not shareable between organisations; network structures differ, architectures may differ, partition layout may differ, the list of assets (and their MAC addresses) will differ.

Edit: + individual organisations won't be storing their BitLocker recovery keys in the same manner as each other either. You did back up the recovery keys when you enabled BitLocker, right? Modern cryptsetup(8) supports a BITLK extension for unlocking said volumes with a recovery key. Again, this can be scripted.

If the affected organizations had such an organized setup, they probably won't need crowdstrike in the first place. The product is made so that companies that don't understand (and won't invest) in security can just check that box by installing the software. Everyone is okay with this.

> so you just write a netboot script to mount the filesystem and delete the file

Because writing such a script (that mounts the filesystem and delete a file) under stress and time constraint is a great idea? That's a recipe for a worse disaster. The best solution, for now, is to go PC by PC manually. The sole reason the situation is as is was the lack of backstage testing.

> I'm horrified at the thought of tens of thousands of novice Windows users digging through System32 to delete driver files; can someone set my mind at ease and assure me this will eventually be fixed in an automated fashion?

Nope. Both my orgs (+2000 each) have sent out a Google doc to personal emails on using CMD Prompt to delete that file.

Anyone with technical experience is being drafted to get on calls and help people manually delete this file.

So no bitlocker on the system?

And I guess if they use bitlocker then they need to enter the key as well? Imagine doing that to thousands of computers

The 1000's of laptops my wife's work uses are bitlockered. I went to fix the issue, when I found that out. I wonder if they will be giving out the keys or if IT will require hands on to those laptops to fix it.... what a shitshow.

Good luck to Joe Schmoe in the IT dept who has to do this over and over flawlessly

Reply on news.ycombinator.com