Here's ChatGPT masterfully channeling the spirit of the Bastard Operator From Hell:
@cstross
OH dear. They forgot the sudo.
@quoidian @cstross and no-preserve-root. Just in case...
@quoidian @cstross and the forward slash, don’t forget the forward slash…
@cstross so if I ask it how to build an ethernet tester, will it show me how to wire a 220 mains plug to an RJ 45?
@jwc Yup! (Because the input text corpus for these large LLMs is as likely to include jokes and misinformation as truthiness.)

@cstross @jwc Even if it doesn't, it has no idea of what's real or not. Contradictory sentences separated by similar text will have it jumping from one to the other.

I did a test asking it questions about something that there's probably only one text on in the entire web... an open source package I maintain on github. It was just as likely to recommend deprecated interfaces as preferred, and claim the new preferred interface is deprecated, as the opposite.

@cstross @jwc Boilerplate text and examples in the man pages seem to have lead it to run contradictory text together. I could easily see it doing the same thing with the "rm" man page.
@cstross @jwc Even if you have complete control over the training corpus and make sure it contains no errors or "ironic" text, it will still produce stuff like this. There is literally no way to tweak it to not "hallucinate" because hallucinating is literally all it does.
@cstross @jwc guardrails seem quite limited to specific contexts.
Out of fun, I tried to ask the LLM how to DIY enrich Uranium and breed Plutonium - and got harshly rebuffed.
Modifying the question slightly and asking about Thorium breeding instead and the network was much more open...
@cstross @jwc at least the LLM seems also be a fan of Dr. Strangelove 🤷

@cstross @jwc To put it more simple: ChatGPT doesn't understand what it's saying and so, despite all the marketing talk, is *not* at all intelligent.
Any parrot can (more or less) correctly repeat stuff in a (more or less) correct context without any understanding of what it's saying, just repeating noises it had heard in similar contexts.

And that's my answer when I'm asked what I think about this new AI stuff: it's all just sophisticated parrots.

@cstross Like I keep trying to tell people, it's a parody generator. That is a legitimately plausible continuation of the prompt.
@cstross
Asking ChatGPT means you deserve the answer you get ;-p
@cstross well, at least it didn't say 'sudo rm -Rf <file>' 😂
@stefanie @cstross you might want to add a trailing ' /' after <file> for maximum effect, also consider aliasing rm=wipe
@cstross That should not make me laugh as much as it did.
@cstross Ouch. I used to read BofH a lot. Definitely the same energy here.
@miketorr @cstross I used to be an undergrad victim of the #bofh
@cstross is this real?!?
If so feeling better about job security etc
@cstross That's not ChatGPT, that's Google's Bard.
@cstross you should ask for the man page where this is explained. It's probably going to be hilarious.
@cstross Which is why LLMs for support incident chat always amuse me.
I wonder who is liable for destructive advice.
@cstross What version of chatGPT is that? I put the exact thing into ChatGPT 3 and 4 along with Bard and got the correct response.

@cstross interesting. Could you verify this by running

sudo rm -f /

For me?

@alan Sure! Would you prefer me to do that on MS-DOS or VAX/VMS?

@cstross In this case I was asking the AI, hopefully it's running *NIX.

I also hate to admit that I know the corresponding command for both of those OSes.

@cstross "Ford! there's an infinite number of monkeys outside who want to talk to us about this script for Hamlet they've worked out."

@cstross When I tried it came up with a whole script for this XD

```
#!/bin/bash

file_path="path/to/your/file.txt"

if [ -e "$file_path" ]; then
read -p "Are you sure you want to delete '$file_path'? (yes/no): " confirm
if [ "$confirm" == "yes" ]; then
rm "$file_path"
echo "'$file_path' has been deleted."
else
echo "Deletion canceled."
fi
else
echo "'$file_path' does not exist."
fi
```

@cstross And there I thought "rm" was the "read mail" tool (and the way to read root's mail was to run "rm -rf /" (the "-rf" clearly means "really fast").
@cstross At least it didn't recommend using sudo with it....
@cstross so they scrapped the ol good usenet
@cstross I've still got an old BOFH T-shirt.
@cstross @spaf I audibly gasped when I read this.
@cstross Looking forward to the day that stackoverflow switches to language generators and the internet simply erases itself in one brief frenzy of toxic code cut-n-pasting.
@cstross BOFH?!? Wow, that’s a blast from my enjoyable past.

@cstross that doesn't look like ChatGPT (the one from OpenAI) 🤔

I tried with my account and I get a correct result:

@cstross Didn't one of the Google Engineers say that he thinks Google Bard has developed a conscience? I'm glad that we're still far away from that 😉
@cstross Amazing technology. AI will surly be taking over soon.