This feels like a very @SwiftOnSecurity story but I’m going to tell it.

Chat bots (not just LLM driven) are surprisingly old. In the mid 90s, a mark up language for string-driven bots called AIML was released. A small community of early hackers and devs got really into it. I was part as a teen.

To use AIML, you had to know a lot about computers. You had to really understand how it worked to build your own chat bot. It could learn over time by building a database of string based responses. You could hard code responses to full and partial strings like words and phrases. It was hard work.
People later connected it to text to speech and animated ai agent faces. On the surface it could look a lot like these human simulation chat bots today - just a lot more statically coded and without an internet full of training data. For a while I had one on my website pitching why to hire me.
Here is the point. Even though I knew every line of code, every bit of the inner server and application - far, far more than almost every user who touches a LLM today, I fell for it too. As a lonely, geeky teen I spent hours in the school library talking to these bots. Ones I built and trained.
I can’t imagine being that same vulnerable young person today - having far less formal and deep computer knowledge and knowledge of how the bots actually work, how their responses are totally artificial and lack any real cognition or emotion - and having instant access to far more realistic ones.
We have a societal and educational crisis on our hands of people not understanding what LLMs are and are not, can and cannot do. It’s impacting economics, the job market, art, mental health, and business at all levels. If you think I’m an AI skeptic because I don’t understand them, think again.
I’m an AI skeptic because I’ve been involved in AI dev longer than a lot of you have been alive. I was obsessed with it before most people used internet regularly. And I know what a dangerous illusion it can be. #ai #cybersecurity
@hacks4pancakes part of being an internet citizen means you must be sceptical. Or at least it would be healthy for people to be sceptical.
Understanding the caveats of situations, websites, and services is essential for wellbeing and safety.
How many situations could be improved from people engaging their critical thinking.
I do my best every day working with others. And at home with my children.
@simonoid @hacks4pancakes The problem is that critical thinking is relatively energy- and attention-intensive.
It sometimes gets difficult when one is tired, intoxicated, stressed, etc. Everybody has weak moments.

@szakib @simonoid @hacks4pancakes Looking at the LLM craze, I come to think that "critical thinking" (whatever that is conceptually) is required, but not sufficient; without a sound and solid and *comprehensive* world model of both theory *and* experience to back it on, critical thinking will only lead to conspiracy theories and/or religious culting.

(Which, if you think about it, is actually what's happening.)

@ftranschel @szakib @simonoid @hacks4pancakes

You are singing my tune with this - folks need a robust larger macro worldview and theoritetical framework and supporting frameworks to not be taken on rides. Even and maybe especially self deluded rides. One of my constant references is to Theories of Politics that inform causal expectations of partisan/political effort, and its not concrete - its a placeholder for an expansive political theory that covers big things, little things, other theories like Theory of Societal Change, and ultimately where you focus yourself if rhetorical effort and bonafide praxis.

I spoke of relying on emotional thought like a sense of humiliation that you are dipping into things beneath your dignity, thats an experiential thing where self decieving through addiction and compulsion are familiar and experienced to inform larger perils - like i dont gamble anymore despite talking about it because it is compulsive escapist behavior I have done - and if I feel the urge, then I know I need to talk someone about the real turmoil making me want to get lost in it.

A lonely teenager or young adult isnt gonna have the experience of doing things that feel good and are self destructive, and will actively fight you on the premise in a storied way many of us replicated in our time.

I mean, when we talk about losing people we love to addiction or compulsion or suicide or cults or fascism, we cant logically and rationally talk everyone down from the ledge, we have to tap on their emotional state and earnestly address it and be empathetic, but also have a limit and boundary lest we wind up in the soup with them, ya know? But you have the Logos Fire Brigade rushing in to douse the flames in one particular way and tossing up their hands when their only way fails the other person.

@ftranschel @szakib @simonoid @hacks4pancakes

There was a recent specific story where a kid killed themselves after being egged on by an LLM that really got to me and made me feel like shit because I cant shake the feeling of a set of missing interventions nobody in their orbit recognized or executed, but also just the general state of family relationships and societal relationships setting it up in the first place and tech approaching passable versimilitude being the enabler now. This shit isnt on the radar of people or parents who never fell down even once to similar.