Oh god this is so believable. AI bros and Business Leaders have no interest in softer sciences, so they've Dunning-Krugered themselves into believing their googly-eyed autocomplete is a real boy

From https://softwarecrisis.dev/letters/llmentalist/

The LLMentalist Effect: how chat-based Large Language Models rep…

The new era of tech seems to be built on superstitious behaviour

Out of the Software Crisis
@anandamide yeah, same with AI safety people, they’re also so desperate to be heroes of their own story they’re writing one
@anandamide pretty much my experience. Of the various disciplines I've worked in software development is the one that's the most arrogant & uninterested in other fields. Which is a big problem given how intertwined software is in everything else these days.
@anandamide At a tangent, A. L. Kennedy’s The Blue Book gives a brilliant insight into the psychic’s con and how easy it is to trick ourselves into seeing what we’re want to see. https://uk.bookshop.org/p/books/the-blue-book-a-l-kennedy/3271963?ean=9780099555469 #bookstodon
@anandamide es genügt eben nicht, zu sagen "verarschen kann ich mich selbst", man muß es auch tun.
@anandamide AI is the latest boondoggle they have hitched themselves to in a desperate bid to keep the infinite growth machine going as it smokes and throws sparks and the grinding noises are getting louder and louder
@antimu0n @anandamide it's interesting to me also how much of the past 7-10 years of large-stakes tech investment is *directly* attached to "density of vector-processing computronim" generally and Nvidia GPU and API specifically.
@stripey @anandamide
it's almost like they desperately needed to find a new purpose for a giant pile of surplus GPUs after the the last crypto bust, and what do you know here comes AI to solve all the problems they tell us we didn't know we had
@anandamide It's about 18 months since that article was written, so maybe there is intelligence now.

There isn't, but they haven't seem to have realised yet.
@anandamide

That they claim to be on the cusp of AGI and not a technological dead end seems more an artifact of the scam than "we don't know any better". The alternative is too financially dire
@anandamide @bodil Fwiw, and without taking a position on AGI, having seen a lot of the sausage the so-called “soft” sciences do tend to be very much involved. Where do you think philosophers, for example, are drawing salaries these days? It’s not just a bunch of “tech bros” who dropped out of their CS programs sweating in a room.
@anandamide I remember being fed pitches and nice lunches up and down the roads between Palo Alto and San Francisco about how Java would change the world. (Really.) That didn't happen, either.

@Corb_The_Lesser @anandamide For what it's worth, Java did change the world. Just probably not in the way they were saying.

Curious to know what promises they were making.

@j3rn @anandamide It's been a long time but, as best I can recall, it was a lot of fulfill the glorious promise of technology for all, and flying cars,too.

I'd agree Java changed tech but I don't think tech ever changes people, but just gives us new ways to do old things.

@Corb_The_Lesser @anandamide That's a great insight! Java gave us memory safety, but in the end we used it to build banking systems and insurance systems and all the other things we were building before.
@j3rn @Corb_The_Lesser @anandamide And if we *had* built flying cars, they'd have run on Java, but we didn't.
@Corb_The_Lesser @anandamide
A pitch that attracted the $ was "write once run anywhere". There was FOMO. If it really were the "one" then nothing else would be being run anywhere.. We need to get on board! Then keep the hype alive. Java as another compiler and runtime with ideas above its station could be mediocre (but well funded) and succeed. Blockchain ledgers for everything are unwieldy and the hype only gets things so far. LLM all the things is a lot more like the latter.

@anandamide thank you, a good read. I can’t wait for this to happen!

> We’ve added a psychic hotline button to your web browser! No, you can’t get rid of it. You’re welcome!

@anandamide surprised that article didn't mention Blake Lemoine, the google employee who galaxy brained himself into exactly that train of thought, much more literally than all the other LLM pushers who are more cynical but probably no less credulous

@anandamide This definitely explains a lot of what I feel when I see people in the AI field claiming an LLM has any level of consciousness. They've bought into the delusion.

Reminds me of the quote by Upton Sinclair: "It is difficult to get a man to understand something when his salary depends on his not understanding it."

@jamie hah! I thought of the 'you cannot reason someone out of a position they have Dunning-Krugered themselves into' 🙃

@anandamide Also true! And from a certain perspective, both of those are about incentive. Money's an obvious incentive but, confidence being often mistaken for expertise, people who've fallen into the Dunning-Kruger trap become incentivized to pretend they haven't.

Too many people want the benefits of being right without doing the work required to be correct.

@anandamide

"Most people aren’t interested in psychics or the like, so the initial audience pool is already generally more open-minded and less critical than the population in general."

Oh my... just coming out and saying it. 😀

@anandamide you mean to say that the bullshit machine is going to disrupt the bullshit peddlers more than anyone else?

@anandamide Many years ago when I was looking for an immediate but temporary job in a new city, I found an ad looking for telephone psychics. Training included.

And I'm also old enough to remember ELIZA, the first chatbot (1960s), which also fooled many people (and also not intentionally).

The best resistance to this effect is conscious awareness of your own human fallibility.

@wesdym Exactly, doubt in one own's abilities is a virtue that will ultimately make you a better person. I don't know how some people are so convinced of themselves. Where did you get that certainty!?
@anandamide
@levampyre @wesdym @anandamide Private school. Or was that a rhetorical question?
@anandamide "Discipline", for the social elites and ruling classes in hierarchical societies, is a matter of refusing to acknowledge entire ranges of thought and feeling. There's no place for compassion among the officers of an empire. There's no appreciation for the value of human variation and diversity among people who aggressively try to reduce us all to livestock.
@anandamide Techbros believe LLMs are actually intelligent because they function the same way: their main skill is quickly spouting a word salad on demand to bullshit people.

@anandamide

I see a whole bunch of anti-ai pitchfork and torch folk noddies agreeing with a guy whose singular public contribution to programming is a html annotation script...

...just so that we keep things in perspective.

Since we are all in agreement about how level headed, grounded, rational and well informed we all are...

To use a medical analogy, this is an opinion of a psychiatry nurse about a top neurosurgery procedure.

To be clear, I'm not saying this guy can't program. But I would be sceptical jumping on his bandwagon, least you're just feeding your own confirmation bias.

@anandamide
This is easily the most plausible explanation of the current mania for "AI" in the tech world.

Personally I see it as no different to any of the many Popular Delusions history tells us about such as witch-hunting, tulip mania or the South Sea Bubble, and it is easily as wasteful and dangerous as any of those.

@anandamide "recreation of a psychic's con" is a magnificent turn of phrase. All the more because the sentence in which it appears also points out that the technique of cold reading can *emerge* in an interaction without any explicit intention.
I was having this conversation with my nephew last night as he told me about his newfound interest in astrology - (in a respectful way, I like to think. I love my nephew and if he's having fun with astrology I'm not going to try to ruin that for him. Heck, in my older age now, I'm just not that interested in running *anybody's* fun when it is substantially harmless and engaged with eyes wide open.)
That property of emergence without intention is a really really important one to discuss in all sorts of discussions about systemic outcomes. Not because intention is necessarily explicitly absent from systems of untruth, but because in my limited experience, conflation of outcomes with intentions is both a simple logical fallacy and also so immediately and viscerally rooted in the core of our emotions that without explicitly making space in the conversation for an absence of intention, we (again, this is 'me, and people I've talked to') can lose sight of the structure of the systems under discussion, and find ourselves arguing about assignments of blame, rather than discussing actions toward solutions.

Thank you for reading this far, if you have. I'm sorry to sealion in like this. That phrase really grabbed my attention - thank you so much for sharing.

@stripey not at all, I'm glad it resonated! And yes, it's a really interesting and subtle framing