This feels like a very @SwiftOnSecurity story but I’m going to tell it.

Chat bots (not just LLM driven) are surprisingly old. In the mid 90s, a mark up language for string-driven bots called AIML was released. A small community of early hackers and devs got really into it. I was part as a teen.

To use AIML, you had to know a lot about computers. You had to really understand how it worked to build your own chat bot. It could learn over time by building a database of string based responses. You could hard code responses to full and partial strings like words and phrases. It was hard work.
People later connected it to text to speech and animated ai agent faces. On the surface it could look a lot like these human simulation chat bots today - just a lot more statically coded and without an internet full of training data. For a while I had one on my website pitching why to hire me.
Here is the point. Even though I knew every line of code, every bit of the inner server and application - far, far more than almost every user who touches a LLM today, I fell for it too. As a lonely, geeky teen I spent hours in the school library talking to these bots. Ones I built and trained.
I can’t imagine being that same vulnerable young person today - having far less formal and deep computer knowledge and knowledge of how the bots actually work, how their responses are totally artificial and lack any real cognition or emotion - and having instant access to far more realistic ones.
We have a societal and educational crisis on our hands of people not understanding what LLMs are and are not, can and cannot do. It’s impacting economics, the job market, art, mental health, and business at all levels. If you think I’m an AI skeptic because I don’t understand them, think again.
I’m an AI skeptic because I’ve been involved in AI dev longer than a lot of you have been alive. I was obsessed with it before most people used internet regularly. And I know what a dangerous illusion it can be. #ai #cybersecurity
@hacks4pancakes part of being an internet citizen means you must be sceptical. Or at least it would be healthy for people to be sceptical.
Understanding the caveats of situations, websites, and services is essential for wellbeing and safety.
How many situations could be improved from people engaging their critical thinking.
I do my best every day working with others. And at home with my children.
@simonoid @hacks4pancakes The problem is that critical thinking is relatively energy- and attention-intensive.
It sometimes gets difficult when one is tired, intoxicated, stressed, etc. Everybody has weak moments.

@szakib @simonoid @hacks4pancakes

Alternative: reflexive impulses against flattery, indulgence, being gassed/greased/buttered up, if there isnt collateral given to you in case of harm or loss. Sure this might be hard to cultivate, but by making it reflex one might bellow while drunk 'go drunk chatgpt, Im home!' And then it craps out, chat over.

@szakib @simonoid @hacks4pancakes

Im not really joking about relying on thought outside of critical thinking to defense self from self deciet or manipulation though - i think people really have atrophied themselves of defense mechanisms solely relying on and insisting that critical engagement spares them being decieved by their emotions like...

Denying we are in deep ecological shit cause we arent suffocating on the shit yet

Denying we are under fascist regime cause all the signifiers of a functional albeit ill Democracy are intact right now

Things like that...and lookie that, two major things people way smarter and critical than me with the actual ability to take cracks at them, dont do because their own rational survivorship depends on not letting the sense of doom drive subsequent act

@szakib @simonoid @hacks4pancakes Looking at the LLM craze, I come to think that "critical thinking" (whatever that is conceptually) is required, but not sufficient; without a sound and solid and *comprehensive* world model of both theory *and* experience to back it on, critical thinking will only lead to conspiracy theories and/or religious culting.

(Which, if you think about it, is actually what's happening.)

@ftranschel @szakib @simonoid @hacks4pancakes

You are singing my tune with this - folks need a robust larger macro worldview and theoritetical framework and supporting frameworks to not be taken on rides. Even and maybe especially self deluded rides. One of my constant references is to Theories of Politics that inform causal expectations of partisan/political effort, and its not concrete - its a placeholder for an expansive political theory that covers big things, little things, other theories like Theory of Societal Change, and ultimately where you focus yourself if rhetorical effort and bonafide praxis.

I spoke of relying on emotional thought like a sense of humiliation that you are dipping into things beneath your dignity, thats an experiential thing where self decieving through addiction and compulsion are familiar and experienced to inform larger perils - like i dont gamble anymore despite talking about it because it is compulsive escapist behavior I have done - and if I feel the urge, then I know I need to talk someone about the real turmoil making me want to get lost in it.

A lonely teenager or young adult isnt gonna have the experience of doing things that feel good and are self destructive, and will actively fight you on the premise in a storied way many of us replicated in our time.

I mean, when we talk about losing people we love to addiction or compulsion or suicide or cults or fascism, we cant logically and rationally talk everyone down from the ledge, we have to tap on their emotional state and earnestly address it and be empathetic, but also have a limit and boundary lest we wind up in the soup with them, ya know? But you have the Logos Fire Brigade rushing in to douse the flames in one particular way and tossing up their hands when their only way fails the other person.

@ftranschel @szakib @simonoid @hacks4pancakes

There was a recent specific story where a kid killed themselves after being egged on by an LLM that really got to me and made me feel like shit because I cant shake the feeling of a set of missing interventions nobody in their orbit recognized or executed, but also just the general state of family relationships and societal relationships setting it up in the first place and tech approaching passable versimilitude being the enabler now. This shit isnt on the radar of people or parents who never fell down even once to similar.

@simonoid @hacks4pancakes before I got out of the military two years ago, one of the big trends was data literacy - the idea that people who were the end users of various data sources/algoeithms/etc needed to know what happened to that data, so they knew what trust to put in it.

For example, imagine you’re a military commander, and you receive information that suggests there’s going to be an attack on your location. Do you trust it? Well, it depends on where that info came from. Was it a direct recording? Interrogation? Double-agent? Or some predictive algorithm?

Im starting to wonder how to get that idea into the rest of the world, because people need this idea of thinking about various sources (it’s kinda tied up in media literacy, but we can see how well that’s worked)

@simonoid @hacks4pancakes it does seem like the way young people are inducted into "internet citizenship" is so totally different from how eg i got into it as a teenager / young adult in the late 90s. i learned to spot and deal with trolls and hoaxes, "photoshopped" images when those became a thing, obviously biased news sources as 9/11 & the iraq invasion happened, and so on. in retrospect i was tremendously lucky to be able to build that media literacy in layers as new developments emerged.
@simonoid @hacks4pancakes now the internet is everywhere, and not just the internet but the platform capitalist internet, at the full extent of its society-harming toxicity, and young people are experiencing it like the fish in that "what the hell is water?" joke. bless any adults who are doing their best to help young people learn to navigate all that with compassion and respect for their developing adulthood.