Ugh ... I've been personally #Crowdstruck (again)
Remember 1 day after #CrowdStruck Microsoft said 8.5 million systems were affected? That was massive downplaying. It was only how many had uploaded crash reports! Which probably only fixed systems could, not the vast majority that were still bluescreened.
https://www.theregister.com/2024/07/29/microsoft_crowdstrike_incident_report/
Post-CrowdStrike, Microsoft to discourage use of kernel drivers by security software

8.5 million crashed machines estimate based on incomplete info

The Register
How a cheap barcode scanner helped fix CrowdStrike'd Windows PCs in a flash

This one weird trick saved countless hours and stress – no, really

The Register

Work, the gift that keeps on giving

Just finished after ~13 hours of fun, hard work, but fun

Will I go back on Monday ? Darn right, I will

Even though I’m not in SRE, I’m close enough to see how awesome they are, after the past week or so ( including #CrowdStruck and, perhaps, what’s going on in la belle Paris ), my ❤️ is with them

#HugOps #LoveYourSRE #LifeIsGood

And now ….. 🥃 🥃 🥃 💤 🛌

@quoll I don’t know, but I cant imagine the kernel driver itself downloading files, let alone to System32\drivers. CrowdStrike must surely support managed install and updates. If not, IT would have had to work around it, because allowing all of your production servers to patch themselves at any given moment is an invitation for disasters to happen.

I would like to see more discussions around how IT organizations have and should have dealt with this.
#CrowdStruck

#ReverseEngineering the #CrowdStrike #CrowdStruck issue has been quite interesting to say the least. After digging into a variety of technical brief's it appears that CrowdStrike's driver kit which is signed by Windows to access the Kernel apparently had a function that took two params. The first was 0 and the second being 0x9C which likely was meant to be an offset into a struct. In X64 architecture, when the R8 register loaded this value, it found there was not a valid address there and therefore BSOD.

If this is confusing and you would like to learn more about ReverseEngineering I do have a free series on my GitHub here -> https://github.com/mytechnotalent/Reverse-Engineering

GitHub - mytechnotalent/Reverse-Engineering: A FREE comprehensive reverse engineering tutorial covering x86, x64, 32-bit/64-bit ARM, 8-bit AVR and 32-bit RISC-V architectures.

A FREE comprehensive reverse engineering tutorial covering x86, x64, 32-bit/64-bit ARM, 8-bit AVR and 32-bit RISC-V architectures. - mytechnotalent/Reverse-Engineering

GitHub
@da_667 i think c-suite is on hook for breaches but probably not for this other than losing 500m in vested stock option value? will be interesting to see how it pans out, the contrapositive is that it is actually amazing things like this don't happen more often #crowdstruck

I fly cross country today, and zero issues with the flight (thanks, United), or any evidence of problems during departure

On arrival, though, still some lingering issues....

#crowdstruck

@quoll @w7voa All good points. But what about the receiving end? What organization doesn’t patch a master image first, before applying the patch to other servers in the organization? And why patch every single server at the same time? And why no automatic rollback of virtual app servers to the last functional backup on repeated failure to boot after the patch? (And yes, a file such as this should be treated as and applied together with other frequent category patches).
#CrowdStruck