JenniferplusplusFeb 26

I wrote about AI again. On some level I don't now why I do this to myself

https://jenniferplusplus.com/what-is-a-token/

Actually, I wrote most of it months ago, for work. But, it was well received, so I put it on my blog. This is the more generic version.

The short version is that AI is not magic. It's a real phenomenon with real behavior and tradeoffs. I'm deeply tired of *****ALL***** the tradeoffs being handwaved away. And so much imagination fills in for the actual behavior. So I tried to describe how it's built, because that informs how it works, which informs what it actually does. And to be clear, it does things. It's not useless. But that's not the same as being useful, or worthwhile.

Anyway, I already put ~4k words on this in the article, so I'll shut up and let it speak for itself.

What is a token

AI is meant to seem like magic. But there's no such thing as magic. It's all illusion. So, allow me to spoil that illusion for you.

Jennifer++
Show threadBernard Sheppard

@jenniferplusplus great article. Easy read. I think that I will recommend this as the next level explanation for people who are partially persuaded by my argument that LLMs are, effectively, glorified phone keyboard autocompletes that make the same sorts of errors, but amplified.

Your article goes a long way to explain why, to people who are starting to want to understand what they are exposing themselves to.

I loved the comment "It's not useless. But that's not the same as being useful, or worthwhile."

And, yeah, the tooling in IDEs can be incredibly useful, but that is very different from de novo generation of entire classes and systems.

Bravo!

Feb 26 at 9:32amWeb