HNHacker News
TopNewBestAskShowJobs

timbelina

6 karma · joined July 20, 2024

submissionscomments
timbelina··on Technical Details on Today's Outage
Cheers Ananay!

So if I put this all together:

a) The driver (sensor) csagent.sys includes code that hasn't checked with a tool like Valgrind or ASan or something and so includes some kind of memory management bug.

b) Since n, n-1 and n-2 versions of the sensor all died equally spectacularly, that bug as been around for at least three versions of csagent.sys.

c) The bug can be triggered by getting the csagent.sys to swallow a shitty channel file and since csagent runs in kernel mode, when it crashes it BSOD's the system.

d) Someone at Crowdstrike uploaded a shitty channel file as part of an update process that apparently happens many times a day.

Am I on the right track so far? If so, there's no/inadequate memory management checks in the csagent driver, and either:

1)There were also no checks before the borked channel file was uploaded because of a failure to follow process, or because there was no process, but whatever the case it was an accident.

or

2) Someone uploaded on purpose, not by accident, the borked channel file intending for a nasty outcome (probably not BSOD)

I can't believe that there are not a million checks and balances in place to let (1) happen, but as my grandma used to say, "Don't assume malice where stupidity will do" :-)

timbelina··on Technical Details on Today's Outage
OK, and another question:-) Can tools like Valgrind and ASan pick up the kinds of errors that are described in those two links from my previous post?
timbelina··on Technical Details on Today's Outage
I was reading these two threads:

https://x.com/perpetualmaniac/status/1814376668095754753?s=4...

https://x.com/ananayarora/status/1814269058088304760

The authors explain the coding error and coredump well, but I'm lost: Is the buggy code that they're describing the channel file, or some kernel code that consumes the channel file? Is there a way to tell?

timbelina··on Technical Details on Today's Outage
Can someone aim me at some RTFM that describes the sensor release and patching process, please? I'm lost trying to understand: When a new version 'n' of the sensor is released, we upgrade a selected batch of machines and do some tests (mostly waiting around :-)) to see that all is well. Then we upgrade the rest of the fleet by OU. However, 'cause we're scaredy cats, we leave some critical kit on n-1 for longer. And some really critical kit even on n-2. (Yeah, there's a risk in not applying patches I know but there are other outage-related risks that we balance; forget that for now) Our assumption is that n-1, n-2, etc are old, stable releases, and so when fan and shit collided yesterday, we just hopped on the console and did a policy update to revert to n-2 and assumed we'd dodged the bullet. But of course, that failed... you know what they say about assumptions :-) So in a long-winded way that leads to my three questions: Why did the 'content update' take out not just n but n-whatever sensors equally as effectively? Are the n-whatever versions not actually stable? And if the n-whatever versions are not actually stable and are being patched, what's the point of the versioning? Cheers!