Lesley Carhart Aug 24, 2025

This feels like a very @SwiftOnSecurity story but I’m going to tell it.

Chat bots (not just LLM driven) are surprisingly old. In the mid 90s, a mark up language for string-driven bots called AIML was released. A small community of early hackers and devs got really into it. I was part as a teen.

Show threadLesley Carhart Aug 24, 2025
To use AIML, you had to know a lot about computers. You had to really understand how it worked to build your own chat bot. It could learn over time by building a database of string based responses. You could hard code responses to full and partial strings like words and phrases. It was hard work.
Show threadLesley Carhart Aug 24, 2025
People later connected it to text to speech and animated ai agent faces. On the surface it could look a lot like these human simulation chat bots today - just a lot more statically coded and without an internet full of training data. For a while I had one on my website pitching why to hire me.
Show threadLesley Carhart Aug 24, 2025
Here is the point. Even though I knew every line of code, every bit of the inner server and application - far, far more than almost every user who touches a LLM today, I fell for it too. As a lonely, geeky teen I spent hours in the school library talking to these bots. Ones I built and trained.
Show threadLesley Carhart Aug 24, 2025
I can’t imagine being that same vulnerable young person today - having far less formal and deep computer knowledge and knowledge of how the bots actually work, how their responses are totally artificial and lack any real cognition or emotion - and having instant access to far more realistic ones.
Show threadLesley Carhart Aug 24, 2025
We have a societal and educational crisis on our hands of people not understanding what LLMs are and are not, can and cannot do. It’s impacting economics, the job market, art, mental health, and business at all levels. If you think I’m an AI skeptic because I don’t understand them, think again.
Show threadLesley Carhart Aug 24, 2025
I’m an AI skeptic because I’ve been involved in AI dev longer than a lot of you have been alive. I was obsessed with it before most people used internet regularly. And I know what a dangerous illusion it can be. #ai #cybersecurity
Show threadSimonoidAug 24, 2025
@hacks4pancakes part of being an internet citizen means you must be sceptical. Or at least it would be healthy for people to be sceptical.
Understanding the caveats of situations, websites, and services is essential for wellbeing and safety.
How many situations could be improved from people engaging their critical thinking.
I do my best every day working with others. And at home with my children.
Show threadszakibAug 24, 2025
@simonoid @hacks4pancakes The problem is that critical thinking is relatively energy- and attention-intensive.
It sometimes gets difficult when one is tired, intoxicated, stressed, etc. Everybody has weak moments.
Show threadCiggy Bringer of SmokeAug 24, 2025

@szakib @simonoid @hacks4pancakes

Alternative: reflexive impulses against flattery, indulgence, being gassed/greased/buttered up, if there isnt collateral given to you in case of harm or loss. Sure this might be hard to cultivate, but by making it reflex one might bellow while drunk 'go drunk chatgpt, Im home!' And then it craps out, chat over.

Show threadCiggy Bringer of Smoke

@szakib @simonoid @hacks4pancakes

Im not really joking about relying on thought outside of critical thinking to defense self from self deciet or manipulation though - i think people really have atrophied themselves of defense mechanisms solely relying on and insisting that critical engagement spares them being decieved by their emotions like...

Denying we are in deep ecological shit cause we arent suffocating on the shit yet

Denying we are under fascist regime cause all the signifiers of a functional albeit ill Democracy are intact right now

Things like that...and lookie that, two major things people way smarter and critical than me with the actual ability to take cracks at them, dont do because their own rational survivorship depends on not letting the sense of doom drive subsequent act

Aug 24, 2025 at 7:35amWeb