putting the sociopathic billionaires to one side for a moment, there are lots of smaller tech leaders who *are* in fact congruent with the kinder, gentler stereotype from the aughts of really smart, clever people who think fast and talk fast and try to make a "win-win" scenario out of their business

the problem is, this kind of person, who is exemplified by a lot of mozilla leadership, has to have a particular orientation towards leadership and problem solving. tech filters aggressively for time-to-market, which means it filters for leaders who are comfortable making confident decisions under uncertainty

this means that good leaders develop a way to kind of vibe out a technology, to quickly develop a partial understanding based on several carefully-chosen data points. even at the best of times, they become overconfident in their facility with this skill, because they are disproportionately rewarded for it. but also a lot of them *do* have a track record of trying out emerging tech and developing a sufficient model of it by intelligently interpolating between the features they can quickly test

LLMs are *absolute concentrated brain poison* for these folks. They try out the LLM to see if it can solve a few simple problems and then they extrapolate to more complex problems. Wrongly. They infer from social cues in their cohort, which are absolutely fucked by the amount of synthetic money (and maybe fraud?) driving a subprime-bubble type mania. They infer from the plausibility of its outputs, which are absolutely fucked because the job of these models is to produce plausible outputs.

lest we feel superior in *our* ability to clock LLM garbage, this disaster among elites is a microcosm of something even worse that LLMs and their sister technologies of shitcoins and spambots are harbingers of: we *all* have heuristics that we need to use to make sense of the world, and heuristics work because of an implicit presumption of good faith in many interactions. most presumptions can be hacked if some significant plurality of actors in a system are always seeking maximum advantage

like, consider open source projects. casual code review works because you assume the code is written by a human and you presume that person is taking pride in their work. if you prompt a spambot to emit maximum PRs under your name so you can pad out your resume and land one of the 7 remaining jobs at FAANG, to stand out among the 5,000 applicants for every opening, the reviewer has no chance. they can't possibly contend with that volume of junk. projects will shut down, eventually.

even with an equivalent volume of PRs, code reviewers *must* use heuristics to judge how much effort to pour into any given line of code. what's the contributor's reputation, do you trust them to know what's going on, what's going on in this file, is this a tricky area of the code, are there subtle mistakes to look out for. spambots flatten the probability of a mistake into a uniform statistical distribution instead of following human patterns

so you have to be equally vigilant on every single line of code, which is way harder than you think (even if you already know that it's pretty hard). and you have to exercise that level of vigilance *because* the contributor didn't care enough to pay attention to the code themselves. it is a recipe for burnout.

the same is true of social heuristics. you might think of me as an AI critic, negative to the point of cynical viciousness. but, dear reader, I am embedded in the same social fabric as these tech leaders. I have friends I respect tremendously caught up to varying degrees with the AI bubble; it's impossible not to be. While I certainly have active social links to other skeptics, I would REALLY like to believe that my other friends are doing good, meaningful work, ethically.

so, to the extent that I am biased, I am actually biased in the *opposite* direction, actively looking for an "out" and willing to meet people more than halfway. it just so happens that LLMs are, as a wise person once said, "shit from a butt", and my *particular* heuristics do not allow for many handwaving shortcuts in this specific area

@glyph I encountered an enthusiast citing the Anthropic stat about the amount of AI code they were publishing, and while it wasn't in a situation where I needed to respond, my immediate thought was "So you're thinking the statements of a publisher of largely greenfield code with no regulatory oversight, the ability to devolve blame to their customers, and a strong vested interest should be taken at face value when judging suitability for updating existing applications under strict regulations?"

@ancoghlan I am being pretty charitable in the thread because a charitable read does exist, but it is simultaneously true that a lot of people are just blasting out their metacognitive deficits on main