“our workplace LLM mass delusion – ava's blog”

https://blog.avas.space/llm-circus/

> How are we all collectively supposed to forget and move past experiencing the respectable elders in an institution having completely and totally embarrassed themselves in the name of "progress"?

our workplace LLM mass delusion

The in-house sessions showing off the tech are embarrassing; led by the wrong people, and should likely not even happen in the first place.

ava's blog
@baldur I am, sadly, not at all surprised that places/sectors who have fought digitalization tooth and nail for decades and now, finally, begin to feel the pain, that those are the most susceptible to IT snake oil.
@OmegaPolice @baldur that is insane... I think this place would be shit even if there wasn't any AI...

@baldur
Far too relatable. Both the "company feel" and the "AI competency".
Like, I always reach to a point where anything one would delegate to AI is:
A) Trivial, which means you don't need AI in the first place
B) Mundane, which means you want to automate parts of and AI isn't reliable enough to do it (anyone who knows what SLA and "three 9" means would toss AI booster out of the window)
C) Complex, at which point AI won't magically provide one with the expertise

Long story short, if AI is a tool or supposed to be one, then it's a really bad tool.

@rawenwolf @baldur One wonders whether "tool" even is the right word. Which other tool "can" do anything, rather than being designed for a clear, narrow purpose?

I'll run with "text toy".

@baldur
For some reason LLMs manage to create more FOMO in management types than any technology before it.

Sure, "it can replace many employees" is part of it (as always the "who the fuck is supposed to buy our products when every company does this" and bad quality are ignored).
And so is "people who don't like to think logically can suddenly automate things (badly)"..

But I think just those factors don't explain the cultish mass-delusion going on here

@Doomed_Daniel @baldur honestly I think a good chunk of it is the sycophancy.

LLMs sound like the upper management that such people surround themselves with. so they sound like the smartest=highest paid people they know. constant agreement, constant ego-boosting, constant insane waste underneath that is trivially obvious if they would actually leave their bubble and go look.

and I think LLM companies know this, and train for it.

@groxx @baldur
Yeah, that's probably a factor.

But I think that executives demanding more LLM usage even if they don't really use it themselves is a thing (I've heard about one instance at least)..

The belief that the company they're responsible for must use to not be left behind seems very strong - though of course at some point it just turned into peer pressue

@Doomed_Daniel @baldur if my hypothesis holds, all you have to do is train your LLM to say LLMs improve productivity, and the execs will eat it up just like the last UI redesign.

I don't see any reason why LLM sellers wouldn't immediately do this, it's free (and largely undetectable) advertising, you just have to seed a little marketing phrases in the fine tuning phases, and helps bend all discussion and summaries and everything it touches in a direction that makes you more money. They've already demonstrated they have zero morals when money is on the line.

@Doomed_Daniel @baldur like, I think it applies even in cases where the morals are fine, just not preventative enough. Every single LLM company trains their bot on their own site and marketing materials because they don't want it to make shit up about themselves, and it's easy to do and easy to catch mistakes and add more training. That alone is fine training that aligns it to their marketing spiel instead of reality.

Add in that execs are often the ones either making purchasing decisions or who are trying to be pleased by purchasing decisions, and you end up with bots that favor sycophancy.
Or they go with some other company that does, and the problem still compounds.

@baldur It seems inexplicable, but the eagerness comes from the promise this tech is a willing slave to replace the messy humans that will work without complaint & not need a 401k, health insurance, a vacation, or time off for a school function or to care for a relative.

It only needs to work well enough to force you to do more for less & not complain about it.

@baldur If you are forced to use that shit prod it till it generates a business proposal that says:
-ai(llm) is shit and people are cheaper
-getting rid of management is a good idea (zelf governing teams)
-getting rid of bonuses for upper management is a good idea
-using only people for client contacts makes the company stand out
-making cookies is the direction the company should move to

Then present that!

@baldur I once read a post that today's AI is a first tech in like the whole IT history that we are forced to use even when nobody really knows if it's good for the job. Instead of being it the other way around. Letting people adopt it naturally because it is actually good. Welcome to the brave new world.
@radekcrlik @baldur I'm not disagreeing with the spirit of the quote, but I can think of other examples (windows, Java, C++, Intel architecture, enterprise versions of scrum, any method that evolves estimating how long a project will take...)
@baldur it's a double-whammy because we just had a similar mask-off moment with "leaders" due to COVID-19 (pun very much intended). After having been shown that our bosses are absolutely down to literally sacrifice our lives in order to keep up mere appearances of corporate productivity and have zero interest in even the cheapest disease mitigation measures unless dragged kicking and screaming into them by the employees whose health is on the line, we're now being shown that said bosses also have absolutely no business sense or idea about how to effectively run the company, and are instead mostly operating off of vibes, which ironically makes their position the only one in the company you could replace with an LLM without losing anything of value.
@baldur I read the article, all about people using ChatGPT and Copilot without any digital cultural background. Of course they are as "fools", out of place, just by not knowing what they are doing. (ie : advising to save a shady mail in the computer desktop… to show it to a bot later…)
@meylodie @baldur what makes you assume they have no 'digital cultural background'? are they from 2006?

@baldur

I wonder what is going to happen when I share this internally in a corporation badly affected by the llm mind virus.

Course correction doens't seem likely at this point. The cultists will feel exposed, make a show of "laughing" off the message in Dunning-Kruger fashion, attempt to shoot the messenger and that'll be the end of it :-(

@baldur
That's the neat part! We don't!