I think a thing most people do not understand about chatbots is HOW SIMPLE THE UNDERLYING MODELS are.
It's really just statistics of how close words are to each other in a huge ass amount of written text, with a sprinkle of classification and labeling (done by humans).
And then it autocompletes your prompt. That's it, that's really all there is to it.
@thomasfuchs yes it has limitations, but even the proximity is governed these days by very interesting and sophisticated systems - it is now very far from “what’s the most likely word given the words behind me”
Like video games that are just a pile of hacks combined, but are able to deliver an immersive experience - these piles of hacks amount to very useful tools.
Yeah, there are tons of things to complain about the AI industry for, but it’s been a long time since LLMs operated as simply as that. There is a lot going on there.
And regardless of what’s happening behind the scenes, the capabilities and usefulness for specific work has gone up exponentially.
I get having serious issues with LLMs, but this specific type of criticism always strikes me as a reflection of unfamiliarity with the current state of the harnesses and models.
@sandorspruit @scottwillsey @Migueldeicaza How exactly is it not inferring the next token based on the previous tokens by using a statistical model?
I’d like to add that I have not made any statement about the usefulness, veracity or applicability of LLMs in my OP.
@sandorspruit @Migueldeicaza @thomasfuchs
Just using the tools will tell you people are speaking of that which they know not. 😄🤷
It’s not unknowable at all.
Human Brain Enthusiast
Miguel de Icaza ᯅ🍉
Scott Willsey
Sandor Spruit 🇪🇺🇳🇱🇺🇦🇨🇦