Delve - Fake Compliance as a Service - Part I

How Delve managed to falsely convince hundreds of customers they were compliant and then lied about it when exposed and called out

DeepDelver

80% of Compliance has always been a performative box checking exercise.

They delivered the product that every company wanted - make the box checking faster.

> 80% of Compliance has always been a performative box checking exercise.

You're making the same mistake as most people do: it's 80% box checking but that doesn't make it performative, the box checking is here so that the dude who checked the box become legally responsible for what's happening if they haven't done what they said they did.

If you didn't check that box you could always claim you didn't know you weren't supposed to do what you did. As soon as you've checked “yes, I'm doing things in the approved way”, this excuse disappears.

> the box checking is here so that the dude who checked the box become legally responsible for what's happening if they haven't done what they said they did.

Maybe so, but how often are small companies actually sued for compliance survey misrepresentations? My most positive look at such surveys, after filtering out all the nonsense, is sometimes they flag something we've missed in our self-directed efforts.

Not really, and I kinda envy you that you haven't really worked up close with compliance-related people.

A lot of compliance is basically corruption - while in country A, you might fall out of a window if you don't buy from the right people at 10x prices, but in 'civilized' country B, you have to buy from vendor X (who has the necessary paperwork), at 10x prices, or you wont be able to sell the product - and there are a million ways that they can turn the levers to kick you out of their markets, or at least make you pay protection money to these compliance organizations.

The systems of grift are very sophisticated, and very obvious to anyone but the people perpetuating and participating in them. As they say,iyt is difficult to get a man to understand something, when his salary depends upon his not understanding it.

A lot of compliance software is griftware - Sonarqube is a prime example - most engineers don't think it adds value, and the 'analysis' it produces is incredibly shoddy, but like a lot of cybersecurity products, it relies on a authoritarian company culture, certification TP conditional on using the software and achieving a good score etc and alarmist language with nice dashboards. A classic example, is it tags public fields in Java as a security issue. And then the management see that you are writing 'insecure code'.

And literal mouthbreathing idiots in upper management eat this shit up, or use it as a punitive measure against the devs who by their very nature do all the meaningful work.

I'm not saying all compliance is worthless, but if you approach quality from first principles, a 'compliant' product usually has to clear a very low bar of quality. And compliance usually keeps the quality low, and prices high, by forcing potential competitors out of the market.

And compliance can keep quality low in other ways, I've seen firsthand - by making devs work on BS tasks, or preventing improvements and fixes to codebases, because they're not tracked appropriately by whatever change management system.

I was incredibly wary of doing hacky solutions in these places, not out of a sense of commitment to quality, but the fact that once management sees your hacks WORK (kinda), all requests to clean up the garbage will be stonewalled.

Thankfully LLMs make this busywork very easy, through making this papermill garbage, and nitpicking busywork very easy, which I feel will bring at least some positive change in the world (at least to those who do meaningful work)

Sonarqube did not flag public fields as a security issue by default the last time I used it — however it has found several real vulnerabilities for me before.

It did by default for me, and there are a bunch of other poorly implemented analyses, such as it incorrectly flagging Dictionary keys in C# as mutable, or opinionated stuff like it disliking certain names and patterns, forcing me to make arbitrary changes that often cost performance, readability or API cleanliness.

Or insane stuff like it doing a blanket-ban on security related code in the app (but importing a third party lib that does the same is fine).

The analyses in general are low quality and you can see not a lot of effort or thought went into them.

They are not the product - compliance, and dashboards for boomers is.

I'm curious about what did it detect for you? In my experience it stops very obvious bad patterns like using string manipulation to submit SQL (which in certain circumstances might even be fine, even necessary), but it can't really trace non-obvious security issues (like tracing a value through the code, making sure its valid on every codepath), it just doesn't have the compiler machinery to do that.

There is no relation between checking a box and becoming legally responsible for the vast majority of certifications.

The company may be legally in troble if the planets are aligned but that's all.

Compliance is crazy sucky - I remember there being a case when one of our vendors was harvesting data like crazy, and we went after them. It was grossly in violation of GDPR, like as bad as it could get.

When we reached out to them, they showed us a cert about how they were GDPR compliant, issued by a huge brand-name consulting firm.

In the paper they said they implemented certain standard-mandated cryptographic measures to 'anonymize' the data. Thing is, they implemented them wrong on purpose, so that they could actually identify users by inverting hashes with a rainbow table.

There was a lot of BS legal reasoning in there but the bigname firm signed off on it. Oh and at the bottom, it had a provision, that if the company were to be sued for breach of GDPR, the consluting firm would not be liable any way.

But this was good enough for tons of companies and govt agencies to just use that software.

So that's what compliance certs get you.

Yes, I know it first-hand.

At least in cybersecurity, there are no certifications that "certify" that you are secure. There are plenty of them that will assess your processes, their execution, etc., but the reality of the risk is next door. This is typically the case for ISO 27001, which has ISO 27002 (the ex British Standard from the 90s) that theoretically governs the controls you should have in place. But it simply does not work.

When you have a major leak, this is usually a company with half a page of certifications, but, hey, mistakes happen. The key problem that these mistakes come from is a fundamentally wrong approach to cybersecurity, but nobody cares.