Jerry 🦙💝🦙I am more than a little alarmed at how utterly dependent I’m seeing business people, and IIT people specifically, becoming on LLMs for their work. And not in a “let’s design a cute avatar” kind of way.
It’s funny, the vision I and many people had about AI was more like what we see in Star Trek, where the technology radically drives up the knowledge of individuals. But on our current trajectory, Star Trek will just be Idiocracy in space with people asking the computer how to use the shower. Certainly I’m missing something?
Viss@jerry its becuase the people that made the AI that got the traction were vc bros who followed the minimum viable product school of thought, and were happy creating a system that tells people what it thinks they want to hear as opposed to the truth and objective reality
the enterprise ships computer is happy calling someone an idiot if theyre an idiot
frontier llms arent
DeeAnn Little@chillicampari @jerry it is, in theory, possible to make the ai from star trek. but it would be an entirely different animal, and it would have to be trained from the ground up with great care, and very likely a different technology.
technically speaking, it could be done JUST with baysean filters alone, and RAG components to fetch data from the usual places. and it wouldnt require boiling the oceans or trillions of dollars of gpu
XCM@Viss @chillicampari @jerry
I think this would not require much technical improvement; but rather a shift in how willing humans are to accept discomfort.
Instructing an LLM to question, expose biases, and offer counter arguments is easy.
The difficult part is accepting it once our limits and imperfections are exposed as a result.
Most people would see that not as an opportunity to grow, but an attack to their own identity
I think this would not require much technical improvement; but rather a shift in how willing humans are to accept discomfort.
Instructing an LLM to question, expose biases, and offer counter arguments is easy.
The difficult part is accepting it once our limits and imperfections are exposed as a result.
Most people would see that not as an opportunity to grow, but an attack to their own identity
Rust Buckett@_XCM @jerry @chillicampari @Viss I actually did instruct Claude to do this and its answers changed dramatically.
@rustbuckett @jerry @chillicampari @Viss
I use this custom instruction with chatgpt:
“Do not offer ego-stroking language. Challenge my assumptions directly and point out contradictions, blind spots, and weaknesses. Prioritize uncomfortable truths and rigorous critique over validation or engagement tactics.”
It becomes a bit of a bitch, but it also exposes logical flaws in my thinking without much sugar coating
@_XCM @rustbuckett @jerry @chillicampari
at the end of the day, the models base instruction is to please you, even it if means lying to you. of course if you ask it to use different language, it will use different language, but that will not change the fundamental truth of whats happening
it will still hallucinate, and it will still dish out bogus info and it will still go crazy on you.
Humphrey Archer@chillicampari
I think we got Idiocracy seasoned with Religulous
Shoe BillThe thing that pisses me off—so, you're right, btw, and it'd be awesome if you weren't—is that this says people are not interested in using LLMs to increase their own knowledge/skill/whatever more efficiently.
OpenAI has a business to run, at the end of the day, and if enough people, and often enough, were demanding ChatGPT to actually teach them something instead of simply doing it for them, they would tune ChatGPT accordingly, or they would at least try to whether they would succeed or not.
I'm not saying that would actually work out all that well; LLMs are not actually intelligent and they don't actually understand things; I realize that, but… they could've made some attempt at tuning the thing, more so to that effect, and they would've attempted it if that'd been what their customers had wanted.
People just don't want to learn anything.
Simple as that.
GunChleoc
Vile Lasagna@gunchleoc @the @Viss @jerry All of the marketing language of generative models centres around "productivity".
Make more!
Spit things faster!
No need to read!
Don't bother writing!
This is a reflection of the indoctrination of Neoliberal Capitalism. It's a continuation of
Hustle Culture!
Sigma Grindset!
Monetise your Hobbies!
It's absolutely infuriating but people don't get to these willingly servile positions on accident, they're constantly pushed towards them:
"Will YOU be left behind?"
SyrupSplashin@syrupsplashin @jerry there were a bunch of papers written around oct/sept of last year that basically explained "they hallucinate because at their core they were trained to tell you what you wanted to hear, and told to avoid telling the user no, or that it couldnt do a thing" - so in lieu of saying no and offending you, it just makes some shit up
Unbro@syrupsplashin @Viss @jerry That's what the bros want you to believe - that "AGI" will "emerge" as more and more billions are burnt.
Spoiler: it won't. We will never build sentient machines; there are no sentient aliens; and we will never emigrate off world.
Howitzer105mm@rancoisse @syrupsplashin @Viss @jerry We won't escape this rock with one million Starlink satellites in orbit blocking the way.
noplasticshower@jerry yeah. Watch wall-e
Dave Wilburn 
@jerry bold of you think think we'll ever end up in space, rather than stuck on earth for thousands of years after Musk's million satellites crash into each other and trigger Kessler syndrome.
ROTOPE~1 ⭐️@jerry siri, does the sonic shower work better if you scream in it?
Dave Rahardja@jerry I think it’s nothing short of a widespread mental health problem. People are literally checking out of thinking. This is bad for us.
Robert Böttner@drahardja @jerry that has been planned a long time ago with social media. The center of peoples habits has become scrolling and swiping through posts and the spending time on a single post averages around maybe 3s, idk. Sociolgists call that attention economy. That forms a society easily to mislead and manipulate.
Media and sociological education is the answer but it will take time.
kwayk42
Kevin Boyd (he/him) 🇨🇦
Julie Hughes@jerry Porn Jerry, you're missing the porn.
Keith@jerry that's one of my major issues with the technology. Somehow we implemented it in such a way that it is serving primarily to deskill it's users rather than to up-skill users.
Moos-a-dee 🫎@jerry
I feel this so hard as a compsci teacher. Even basic things like understanding a filesystem is completely obscured for people on phones and tablets, loading screens don't even say what's going on anymore you're just at the mercy of a spinning wheel, and so much more.
I feel this so hard as a compsci teacher. Even basic things like understanding a filesystem is completely obscured for people on phones and tablets, loading screens don't even say what's going on anymore you're just at the mercy of a spinning wheel, and so much more.
jaark
[nate@social0 ~]$ 
@jerry Maybe i'm an optimist. but I think the star trek future is still possible. Personally I use AI assistants like you see in Star Trek, as an augmentation not a replacement for my own skill. I dont see why folks wont be able to continue to do that.
ffffennek
Sterling
Shafik YaghmourSo they have not designed a new abstraction like Assembly was to machine code and high level programming languages were to assembly. That would be a level up.
They have just reduced the friction towards aggregating knowledge but w/ a significantly high error rate.
They have hooked into the worse human cognitive flaws but are not providing any leveling up.
Even if they could reduce the error rate to zero w/o friction we just end up w/ the "Whispering earring"
@jerry
I know.
I was expecting Star Trek or Star Wars type of robots and AI, but also ray guns, antigrav sleds, and flying robots.
What we got was whiny feeble drones, robots that can't even get you a soda, and autocorrect clippy pretending to be intelligent
oldguycrusty
My favorite Example of unwanted new feature... We do that in software and information systems all the time..
Sarianna Silvonen@oldguycrusty @jerry This reminds me of one of the parcel delivery companies here, which has an annoying system where, instead of sending you a code that you punch in a keyboard on the locker to open it (like the other major company does), you need to download their app and then stand around in the vicinity of the locker with your phone and use the app to open the locker. Which means that you can't pick up packages on behalf of family members or friends, since only the person with the correct phone can do it. (I can get how this was probably designed as a security feature, but I think they should build in an option to allow someone else to pick up your package, which they currently don't have.)
Yep. A software solution that 1)No one was looking for, and 2) Is only partly effective, only does part of the job.
Andrew Zonenberg@jerry If you haven't seen my short story "Terminal Pacifism" you might appreciate it... https://serd.es/2025/10/31/Terminal-Pacifism.html
Matti Järvinen@jerry I've seen experienced developers blindly trusting AI instead of any personal thought and getting stuck in their work.
The AI isn't perfect, it guesses and predicts but it doesn't understand jack shit.
Oliver Schwarz@jerry Actually, I don‘t want to see AI as a „thing“ that is „happening to us“. It‘s our decision! We decide, how we are going to use it. We decide, whether we use it for the greater good or for a bigger rest, shutting us offline from our consciousness. I still have hope for humankind :) But i also know: If there‘s a button, someone will press it.
Quincy ⁂
katzenberger
Leendaal
EndlessMason@jerry
The sonic showers were speech activated in that show
The sonic showers were speech activated in that show
Steven Hugg@jerry Well, I mean if humans in the 23rd century are that smart they won't need Vulcans to fix everything. Scotty is supposedly a top engineer but he's a chronic B.S. artist, although he's pretty good with a Macintosh Plus keyboard.
💀 Mirko 💀
Allan Chow@jerry nope. but then again. their output remains the same quality. Just faster.
@jpellis2008 @jerry The thing you're missing is that we're not hurtling towards the Trek future of a post-scarcity pro-science unified-earth utopia. More like the stars for corporate indentures in STL "freezer" ships, and idiocracy for those left on the ground as consolation prize.
@jpellis2008 @jerry Trek-grade conversational AI exist today, but nobody ever imagined that the data these systems would have access to for query response and analysis would be anything less than factual, if not complete. Systems that were willing to tell the user "Unknown", "That data is not available", rather than trying to keep the user on the line with fabricated responses and fabricated data/references.
hrbrmstr 🇺🇦 🇬🇱 🇨🇦@jerry Oh there's a great example of the bad path of AI in Star Trek and it's what we're becoming: the Pakleds
David J. Bianco (He/Him)
Fritz Adalis
danwwilson 
@FritzAdalis @hrbrmstr @jerry I’ve been thinking lately that we’re heading into the information dark ages. So much LLM slop that the truth can’t be discerned and what seems to be less access to accurate information
@danwwilson @FritzAdalis @jerry i do have to find a cpl extra sources for any important news I read to ensure it's not fabricated. and i won't trust any image (and haven't engaged in video viewing for years now) anymore.