JenniferplusplusFeb 26

I wrote about AI again. On some level I don't now why I do this to myself

https://jenniferplusplus.com/what-is-a-token/

Actually, I wrote most of it months ago, for work. But, it was well received, so I put it on my blog. This is the more generic version.

The short version is that AI is not magic. It's a real phenomenon with real behavior and tradeoffs. I'm deeply tired of *****ALL***** the tradeoffs being handwaved away. And so much imagination fills in for the actual behavior. So I tried to describe how it's built, because that informs how it works, which informs what it actually does. And to be clear, it does things. It's not useless. But that's not the same as being useful, or worthwhile.

Anyway, I already put ~4k words on this in the article, so I'll shut up and let it speak for itself.

What is a token

AI is meant to seem like magic. But there's no such thing as magic. It's all illusion. So, allow me to spoil that illusion for you.

Jennifer++
Show threadMark Reid

@jenniferplusplus @eniko This is a really great overview. I’ll definitely show people this if they want to understand the nuts and bolts of what a transformer-based system is.

What is still a little “magical” to me having programmed for 40 years and worked in ML for 25 is how even small working programs can be generated from repeated token prediction, precisely for the reasons you gave (ie, correctness isn’t explicitly modelled and they model non-coding language too).

Feb 26 at 8:26amToot!