Whatever improvements ChatGPT o1-preview may have to offer, they've introduced a whole new layer of bullshit with the "status messages" that make up nonsensical tasks that it pretends to be doing.
This matters—a lot—because this particular bullshit is not a limitation of the system.
This is an active choice by OpenAI to mislead people about what the system is doing. It’s Wizard of Oz shit.
The fact that they use their own bullshit generator to do it is icing on the irony cake.
To be clear: I'm not saying this is just fake "loading" screen. Each phrase corresponds in some sense to something the model is doing, and we can get more information about that (see below).
My problem is that these phrases are not accurate representations of what the system is doing.
Look at the language. Over and over again, imputes a sort of cognitive agency that LLMs simply don't have.
"I’m piecing together"
"I’m noting"
"I’m thinking through"
"I noticed that"
"Hmm, I’m thinking about"
LLM's don't note, think, or notice.
There's an even bigger problem here, which is that at best these are summaries of output stages that are chained together in the model, NOT descriptions of the processes that generated those output stages.
OpenAI should be well aware that LLMs are not able to accurately report *why* they did something. They are only able to make up post-hoc rationalizations based on their context window including their output.
So these are at best guesses, not true descriptions of motivations or processes.
@ct_bergstrom I think I see what you mean? to those who dont understand how its handling these things, especially folks who think in explicits or people who are new to how they work, and/or learning themselves.
They may take what they are saying completely out of context, possibly assigning agency to where there is none vs to how they are really operating.
I think to a degree its somewhat predatory, and inaccurate. Makes me wonder if they have a rationalization safety switch?
@EVDHmn @ct_bergstrom
Yes, predatory and inaccurate, to the point where I classify all claims of "AI" as outright commercial fraud. It isn't what they claim (an artificial intelligence) and it can't do what they claim (analyze, calculate, or identify the main point of an essay).
It just statistically processes "the most likely answer to a question that includes those keywords" or "these phrases appeared most often in the essay, and are often grammatically connected in this way">
@ngaylinn @ct_bergstrom
There is no “there” there.
OpenAI and #SamAltman are #BrokenStairs: https://www.superversive.co/blog/on-broken-stairs
Stop ignoring broken stairs. It doesn't work. Coined in 2012, the metaphor "broken stair" or “missing stair” is used to describe a person within a social group or organization who is known to be problematic, untrustworthy, or harmful, but whose behavior is tolerated or ignored by the grou
@drwho As I've explained elsewhere, there's a big difference between not knowing why I chose chocolate ice cream instead of strawberry tonight, and not having any idea what the hell I've been doing for the past 15 minutes.
About 8 beers difference, in my case.
@ct_bergstrom I get that, but I wasn't being funny or hammering on your registers. I was thinking about stuff like Lemon and Portor (1993), Delgado (1969), and Velmans (2003).
Consciousness as thin layer of paint over top of the preconscious and unconscous structures.
@ct_bergstrom I guess the challenge is that most people, including me, don’t have any clue how these models work behind the scenes. You could simply omit these phrases. But somehow they want to encourage people to use these tools and make them user-friendly. They decided to sprinkle some glitter over the output. Is this so bad? Sometimes it’s a thin line between marketing hype and user experience improvements.
But yes, the focus should stay on teaching users how to use these tools.
> OpenAI should be well aware that LLMs are not able to accurately report *why* they did something. They are only able to make up post-hoc rationalizations based on their context window including their output.
That actually sounds pretty human-like to me
all natural language is metaphorical, especially user interface text, so I don’t mind the anthropomorphic tone of its messages; furthermore, coders have always explained their algorithms in the first person (“okay first i hash the password, then i query the db to find the user record, then compare the fields”); nitpicking these terms is a distraction from the real problem, which is whether they are accurate or lies or just more opaque bullshit
(a lie requires intent to deceive)
@ct_bergstrom this is a whole new *spectacular* level of fraudulent bs, we need to coin a brand new word to define it ...
I wonder if ChatGPT could come up with a clever one...
But as somebody said earlier, the question is not worth the stupid amount of heat and water needed for it to answer...
@ct_bergstrom "It’s Wizard of Oz shit."
😂
Computers shouldn’t lie.
That includes tongue in cheek references to the classic “reticulating splines” loading screen joke from The Sims. If your audience isn’t in on the joke, you aren’t just kidding, you’re deceiving.
Too bad we forgot to make that obvious rule into law before it was too late. ☹️
[update: those status captions might not be jokes or lies! https://mastodon.social/@jamiemccarthy/113311612204003571 but LLMs and their hypers have now trained us to mistrust their every utterance so 🤷♀️]
Wouter
Carl T. Bergstrom
Everyday.Human Derek
Kathmandu
Nate Gaylinn
Coach Pāṇini ®
The Doctor
2xfo
PointlessOne 
gurix
Kuba Orlik
Veronika Cheplygina
Tom Bellin 
Karen Strickholm
Tucker Teague
Alex Chaffee
Mike J👹🐀 🤘🏻
Laura M Castro
Elisa Fadda-🔺
tessarakt
Neia masks
ch0ccyra1n is leaving fedi soon
Chara
Mae
lightspeed
Colin from Edinburgh.
aburka 🫣
ジギーくん
Tristan Nguyen
heapwolf
Lili
mosttoast