New blog post: Nobody knows how the whole system works
https://surfingcomplexity.blog/2026/02/08/nobody-knows-how-the-whole-system-works/
New blog post: Nobody knows how the whole system works
https://surfingcomplexity.blog/2026/02/08/nobody-knows-how-the-whole-system-works/
@norootcause what are these tools effective at? we used to think deleting code was a thing to be proud of.
how can we be sure it won't go away when it's currently utterly unprofitable and heavily subsidized? do we suppose the subsidization will continue (until morale improves)?
let's say I agree the tools are effective under the right circumstances. Now we've introduced an expensive external dependency we don't control, which also happens to harm learning. why?
@norootcause quite liked this — I feel like at one point it was possible to understand a system end to end, but the scope of technology has expanded so much in the past ~40 years that there’s no way to grasp everything end to end anymore.
We are all depending on the work others have done, all the time.
@norootcause actually, I'm one of the people who knows how the whole system works for certain things. We do exist, we're just exceedingly rare and invisible due to being over the age of 34 and generally dismissed by the less skilled.
And complexity is just the bullshit excuse trotted out by people who have absolutely no clue what they're doing. No, I don't mean technically.
I mean 90% of knowing a system to end is three things they dismiss out of hand: intent, compromise, and reasoning.
@norootcause Grafana is a stellar example of that.
What is the *intent* of Grafana? "Monitoring"? That's not an intent, that's a vague aspiration.
Monitoring what? Maybe we can set boundaries. ... okay "everything but half-pretending it's sorta separate" is not a boundary.
And compromises... yeaaaaah, those aren't. "We used Z because we couldn't make Y work" is not a compromise.
It's just cranking out LoC and 'features' as though these were the only metrics.
@norootcause Possibly unrelated, but this reminds of something I read a long time ago.
https://briandbuckley.com/2012/09/12/who-knows-how-to-make-a-mouse/
@norootcause
In 1977, I was part of a team at BNR, working on the first all-digital telephone switch (DMS-100). We thought that we could apply our ComputerScience knowledge to build a nicely layered set of abstractions in an elegant framework. And we failed in our attempt to encapsulate a century of analog telephony engineering ... so, we were loaned a veteran telephony engineer, and all day programmers went to his office and said "George, we have a thingamabob connected via SF signalling to a whatchamacallit, and the thingamobob does 3 off-hooks and an on-hook and the watchmacallit responds ... and then ... and then ... and what happens next?" and George would think for a bit or consult from his bookshelf of binders and give the answer, which was duly incorporated into the code that was becoming more byzantine every day.
At one point, I was asked to interface an 1894 Stromberg-Carlson switchboard to our fancy digital switch (telephone companies were loathe to retire equipment that had been fully depreciated).
(The DMS-100 operating system and programming languages, however, were elegant, as was the overall architecture of the system. (For example, we were using relational databases before Oracle; and a OOP before C++.) But the signalling was a hodgepodge of special cases because that's how telephony engineers thought processes worked.)
@norootcause I think they key difference for me is that while I might not know exactly how pre-LLM software works, I trust that it’s coherent and I can drop down a layer when needed
Whereas slop is slop under the hood - it’s not designed for understandability or debugability - if no one is reading the code then in practical terms it’s been optimized to not be readable
@norootcause The error the post makes, I think, is confusing knowing the details with understanding the principles.
An example.
I was working in Perl on VMS on a data interface to a remote system. The Perl parsed well structured DEC database (RDB?) output, generated by an Ada program. To make the parser, I had revised the data structure from its documentation into Perl, comparing the Ada code.
Another team were working to replace the VMS/Ada system with a Solaris/C++ system. They were generating the same DB output and wanted a test.
My Perl parser complained: one of the 3 core transactions from the new system failed sanity checks.
I dug out the Solaris C++ code from ClearCase. By the end of the afternoon, I traced the bug: a misplaced & character at the bottom level of the new team's code. It had been there for years in a core object.
Now, did I understand everything here? No, but general knowledge, and layers of reasonably coherent design, saved the day.
LLM output thwarts that type of analysis.