The root cause is that they made it expect 21 arguments, but only gave it 20.
This was not caught in testing because their test environment did not represent the same conditions as their anticipated customer environment, and they put in a wildcard for the missing parameter instead of having a testing path that would validate that parameter.
Left unsaid, but very the fuck pertinent, is that the rest of us who give a fuck have this nasty habit of checking that something we're going to load into a process....has the -right fucking format- for the process, and then we use this cute little concept called an "error message" to let the operator know if -something is the fuck missing-.
Innovative, I know. Top-right quadrant thinking.
Oh, whoops, my mistake:
They -allowed- wildcards in the 21st field initially, and then -disallowed- them but didn't test -that- change.
Nice touch with putting the dates in; y'all are at least compatible with the disaster podcast convention that shit starts getting serious when there's a timestamp, so, credit where it's due.
I'm....I'm gonna have to sit with this one for a moment.
Because what it says about their development processes is -fucking fascinating-
So what they're saying here is that the -sensor binary-, at the -time of compilation-, did not validate that the definitions file had the correct number of fields.
But
............you don't -do- that at compile time.
Before the compile as part of your overall process for adding code, to make sure that everything that this code connects to has been adjusted, yeah, that's...that's how software review works.
On execution, when you're -loading- the definitions file, having it check that it's got the number that it was expecting, yes, I was screaming at that up above.
But neither of those are at compile time. Why are they bringing up compile time.
Also, this is not one finding. This is -multiple- findings:
1. the actual lack of validation
2. the lack of effective review process exposed by this, where an invalid state was not caught during the development of the new type
3. the lack of effective testing that did not include, e.g., invalid configuration files -to test such a mechanism in normal operative contexts-
Three is more than one, guys.
WHY IS THIS A MITIGATION
YOU ARE NOT DOING IR RIGHT NOW. THIS IS A POST-INCIDENT REVIEW.
MITIGATIONS ARE DURING THE INCIDENT. POST-INCIDENT FINDINGS GET FUCKING
R E M E D I A T I O N S
YOU ARE USING THE WRONG WORDS FOR WHAT YOU ARE DOING
The phrase "input pointer array" appears in the next para, which means "we are doing silly shit with C++ because we're leet yo"
Languages that don't make you do your own fucking pointer math exist for a fucking reason.
Their 'mitigation' here is to bother to check that they're still in allocated memory, something which is only a problem by their choice.
So they talk about how their test cases weren't broad enough in the next para, and they promise swearsie-realsie that they'll put in test scenarios that "better reflect production usage"
Buuuut I don't see one -really fucking obvious standout test case- that, given the context above, really the fuck ought to be separated out:
They say nothing about whether they're gonna test the -failure- of the sensor.
If you ain't testing with invalid inputs and other abuses to bound the behavior of your binary, then you're not testing its full envelope of behavior and you cannot assert anything meaningful about its suitability for production.
Car manufacturers do crash tests to make sure you don't fucking impale your face on the steering column; this is the exact same fucking principle.
There's a -lot- of fascinating subtlety and discussion to be had around testing generally,
but this is kindergarten level horseshit. Maybe when they stop eating the crayons we can talk about the more interesting bits.
So there's a logic error here alright but it sure the fuck ain't with their agent's parsing, which....this is repeating items 1 and 2, but from a different level of abstraction.
This is turd-polishing.
More to the point:
Why the everliving fuck are you hard-coding a specific number of channels into your fucking agent,
when 'channels' are a tagging convention and have no pertinence to the detection logic,
and you could just -fucking allocate the resources to hold the content based on the configuration itself-
You -utter- -assholes-
You are -creating a problem for yourself- and then -doubling down on doing it wrong-
.......
Mister Holmes, sir, we have a -mystery- on our hands!
Why, just this morning the lad Simpkins came into Scotland Yard with the most astonishing tale and -
Mister Holmes, the mudlarks are in an absolute uproar, you must have heard from the Irregulars -
London's entire sewer system has been -scoured utterly bare-
There is -no shit-, Sherlock!
yeah they don't even try to dress this one up
Problem is, they completely fail to talk at all about staged deployment for -any other part of the product- so uh.
Also as one of their mitigations they're deigning to allow customers whether to accept the new content.
You know, the -base expectation- from -literally everyone else-
But only about this 'channel' content. Not anything with the actual definitions or the agent binary itself; none of that is mentioned at all.
Completely the fuck missing.
So, see, what this -looks- like they're saying is that they've got third parties in to review the code and process.
But those are two separate clauses.
They have two third-parties in to review the -sensor code-
-and-
They are conducting a review of process.
But they are not actually -saying- that the third parties are involved in the process review at all - only the code review.
Perhaps someone ought to ask them to clear that the fuck up.
It's that sticky "we" there, y'see?
"We" -could- be implied to mean the set of crowdstrike, vendor 1, and vendor 2.
But "we" can also refer to crowdstrike the company, or to the personnel of that company.
"We" is one of those words that has -very- tricky scope to it, and can be used to lie to you right to your face.
This whole technical details section is exec-pandering crap.
-this- little fucker is funny tho, 'cuz it implies that if you have an input that cannot be parsed with regular expressions, clownstrike can't handle it.
The next part appears to be an extract from some guy at MS's blog about this shit -
whiiiich pads out the last half of the document and since it isn't clownstrike's work, but just shit they lifted from someone else's blog, doesn't matter
So yeah, only the first six pages have any content on them; two of the findings are duplicates and are just there to pad for length; they -missed- a bunch of other findings; they are committed to a known-broken sensor operations regime and have no clear plans to fix the underlying architectural issues exposed -by- this; and they don't have anyone left in the place who can fucking write worth a damn.
Complete fucking clownshoes. If I were their customer I would be calling for their literal, heart-ripped-from-chest, blood for this.
Also:
Who the everliving fuck -audited- this pile of shit?
Who signed off that -this- was suitable for deployment to federal computers?
Who the fuck did their audit and why the fuck did they not catch -any- of this?
........y'all.
I -really- suspect that this document was an LLM summary of some collation of internal documents that got some light editing.
......who has a very uncomfortable looking smile on his blog profile, damn. That must hurt to make that expression.
......there's nothing on here about licensing; anyone know if MS lets you just, like, steal half a blogpost to crap into your document to fill out space?
.....yeah, looking back, that fixation on defining why "channel 291" was pertinent?
A person wouldn't keep referring to it by number, but would use a pronoun phrase. "the updated channel" or similar.
And then with that mishmash in that other para with the whole 20 vs 21 thing.....
Yeah, that's not how a person would write that at all. Certainly not someone who actually understood what was going on.
Yeah.
This is not a professional report in any sense.
None of this has been handled according to professional standards.
The level of negligent disrespect evinced by Crowdstrike utterly beggars belief, and I seriously question why they have the certifications they do, given the flagrant and obvious process violations this report makes evident.
And I seriously question who signed off on certifying them, because this goes well beyond the normal vagaries of corporate incompetence; even for an industry that assumes bad behavior, lack of compliance, and failure, this is a standout.
This goes into levels of negligence akin to civil war era profiteers, who similarly got paid wagonloads of cash for providing a shoddy - literally, that's what the word was coined to reference - defective product for government use.
this is even more fundamental than that.
This is at the level where "actually check your -whole- work" needs to be fixed first before we can get to such innovative concepts as discussing what an input is.
@munin I guess I interpret "input" to also mean the input to functions. Even in a single-developer project, to just assume that you definitely have never and will never make a mistake in your code seems arrogant, to say the least. And this definitely isn't a single dev, so they're assuming that *nobody* would ever make any mistakes and that technology always functions perfectly. (no verification of data downloaded of the internet?)
Thank you for the breakdown of this clusterfuck.
np. I was in the mood to tear into something today and that happened to be a fun chewtoy.
Saving this for later perusal, and for passing to people who really ought to know about it (including me).
Thank you.
Calling @Chartodon Spine ...
Your chart is ready, and can be found here:
https://www.solipsys.co.uk/Chartodon/112920897187312739.svg
Things may have changed since I started compiling that, and some things may have been inaccessible.
In particular, the very nature of the fediverse means some toots may never have made it to my instance, in which case I can't see them, and can't include them.
The chart will eventually be deleted, so if you'd like to keep it, make sure you download a copy.