Hi, I work with generative machines. Everything from Markov chain generators to GPT-3. I’ve trained and tuned many models with GPT2 and 3, all with the intent of simulating human interaction.
I know a fair bit about generative machines, both how they work, and how to tune and interact with them to get particular results.
I need you to hear this: they do not know or understand anything. They are complex probability tables.
1/
Thanks to very clever programming and increasingly powerful GPUs, they are deeply complex and precise probability tables, but they are still only inferring based on what they’ve seen in the past.
We don’t understand how organic brains learn. Building an electronic brain remains science fiction.
LLMs produce an impressive simulacra of human language, but there is no there there.
I jokingly say that they “make shit up,” but that implies intent and ability to conceptualize.
2/
These machines can do neither. They produce strings of tokens that statistically appear like they were produced by a human.
To an LLM the statements “Neil Armstrong was the first man to walk on the moon” and “Neil Armstrong was the first man to walk on Mars” are differentiated only in that more people have written the former than the latter.
That’s it.
AI doesn’t exist and companies letting what we DO have make decisions is just a way to avoid culpability for the results.
3/3
@ieatkillerbees I studied AI briefly in college 30 years ago, and have been watching all the hype around "AI" lately with amusement. It's exactly what you say - just fancy lookup tables and decision trees. Sometimes it puts fun things together, but it's accidental.
(The reason I left CS was because I got interested in what it means to be human... due to studying AI. Debates about ethically turning off an AI felt ... Hugely premature. There's no being with a sense of self to turn off...)
I took one AI course in university in 1996. At the time, it was *entirely* focused on tree searching (i.e. map out the possible decisions and try to find an optimal path to the best outcome) and expert systems (assign meaning to symbols, write logical rules about them, then try to infer stuff.)
Neither of these is "thinking", but both are able to let a human audit the process. That is, they are debuggable
(TBF, my school wasn't really big into AI at the time. Which is not a slight on my instructors, but the course ended up being an introduction and survey of the field.)
@ieatkillerbees
I agree with this. But it also shows that no "intelligence" is needed for nearly every under grad essay. You just need some nice structure and regurgitate what other people have said (with sources)
I still think the more we learn about "AI" the narrower whatever true intelligence is becomes.
I think the fact that so many teachers are worried about auto generated submissions says a lot more about their submission expectations than about the new technology.
If you think of these systems as BS generators and cross-reference with the things they seem to be good at generating, it mostly serves to highlight the amount of BS that we create.
@gatesvp @ieatkillerbees @jfrench
If you're a teacher and not so good at identifying BS-no matter who generated it- you might want to value written submissions at 5% of a student's mark and set 95% of the mark coming from an oral examination which critiques what is in the written piece. Tell the students about this - pre-assignment.
@SnookyArdness1204 @ieatkillerbees @jfrench
Snooky, that's a reasonable idea, but not feasible Uber current teaching curricula. Teachers don't necessarily have that option. And where they do have some options, they don't have the time to grade material in this way.
We've optimized the grading of written material to be as mechanical and consistent as possible. Changing grading methods means stretching already thin teachers even more.
Not "may be", it is the case that most administrators are locked in to a utility focus. And they are locked in in a way that they don't get to change.
The changes you're asking for require a curriculum change. And they're not unreasonable requests. They're just not requests that administrators can make. The requests that the public has to ask for.
@gatesvp @ieatkillerbees @jfrench That is my biggest optimism with new AI: It will become much easier to call out superficial bullshit education, formulaic "creative" work, and corporate bullshit speak.
"You are only testing students for parroting stuff, not deep understanding" has only resulted in long arguments leading nowhere.
"A rock with a 12V battery attached could get an A in your class" results in much more actusl debate.
"But it also shows that no "intelligence" is needed for nearly every under grad essay."
By that token no intelligence is needed for many math problems. (though GPT like system seem to suck at word problems)
But this goes back to "what is the purpose of homework?"
Do I give homework because it's work I want someone else to do? Do I like homework and want people to give me stack of it. Heavens no.
The value of an essay for a student is just as a column of sums for a 3rd grader: exercise.
No one has ever suggested that no one needs to learn to add because there are calculators. (though it's true being fast and accurate are less important than they once were, especially with very large numbers)
Students will still need to practice writing and get feedback and then write more.
I'm not sure chatGPT is causing any crisis among intelligence theorists & researchers, who probably understand the points @ieatkillerbees is making. Undergrads (at least my students) don't write papers as an IQ test; there are other goals.
This brings up another point, though: p(A|B) ≠ p(B|A) or, put another way, a diagnosis/signal-noise problem: intelligence isn't needed to create an UG essay but that doesn't mean it wasn't involved or maybe even maxed out.
Some folks in my family have become concerned by ChatGPT and other tools, and as the family mathematician and computer gal, I am frequently asked to explain it.
My response? "ChatGPT, midjourney, all of those things-- they're efficient, sophisticated programs for throwing word matrices at the wall and seeing what sticks."
As a software developer, it deeply worries me how few computer programmers and software engineers seem to grasp this basic fact.
I had a very experienced coworker tell me the other day "I wish we had a ChatGPT front-end for searching our Jira bugs database." Uhh... what? ChatGPT knows nothing about our software or the contents of our cases; how could it produce accurate information out of that? The level of misconception there baffles me.
Well Blockchain mania gave a clue about the gullibility of our brethren ( and some sistren ).
@CliftonR @ieatkillerbees Does it not?
It's not clear that the bigger models actually don't
https://borretti.me/article/and-yet-it-understands
The bigger problem is that due to biases in training data you build biases into the AI results, and unlike actual code nobody understands how it works and then cannot fix it.
Yes - I can be fairly sure that our bug reports, feature requests, and resolutions are not in its training corpus, as they've never been on the public web.
Therefore ChatGPT "knows" nothing specific about them. One could try to train a partially-trained LLVM on them, at great effort and cost, but that seems much more costly than indexing the content with an associative DB.
The latter would likely provide most or all of its potential value - but isn't glamorous.
@CliftonR @ieatkillerbees So you haven't looked at the article.
While training on the actual data may increase accuracy large enough model may work without ever seeing the data in question, that's the point of machine learning, and the results show it has been achieved in some cases to some extent.
@ieatkillerbees Reminds me of this post from @bretdevereaux who talks about it in the context of history students using it as a substitute for doing their assignments.
@Infrapink @ieatkillerbees @bretdevereaux
Yes, I was going to mention that article too but ran out of space.
@matthew_d_green @ieatkillerbees IMO the impressive bit is the extraction of relational structure from the training corpus, and also outputting very good English grammar.
But if we accept that the model as learned the equivalent of
- astronaut(Neil Armstrong)
- walked_on(moon, Neil Armstrong)
- walked_on(mars, _nobody)
- ...
...from text analysis, the output comes easily from symbol manipulation without needing understanding of what walked_on actually signifies.
@matthew_d_green @ieatkillerbees To be fair, I expect that the model also understands relationships between walked_on, ran_on, flew_in but again without having a grounding in what that means, beyond statistical relationships from a large corpus.
If you prompted "describe Neil Armstrong's first walk on the surface of Mars" I expect it would assemble a credible report from facts about the moon walk, and Mars.
@meltedcheese @ieatkillerbees @LyallMorrison I just don’t know what those terms mean and how you empirically measure them. It sure seems like from the early transformers to ChatGPT, the models have become vastly better at understanding and reasoning about context and even new situations that can’t have been in their training set.
Obviously ChatGPT doesn’t really know what walking looks like, but that would be the same as a human who was never able to see someone physically walk but just had to read about it.
Datendengler
psychopomp disaster
Matthew Weier O'Phinney
Nora Chan
Eva Chanda
Geoff 🏴
Chris [list of emoji]
Jfrench
Gaëtan Perrault
Scott
Pēteris Krišjānis
Helge
Carwil Bjork-James
myrmepropagandist
Felicitas Macgilchrist
LucyKemnitzer
Riley S. Faelan
Darrin L Rogers
Martin Vermeer FCD
fmc01
Sophie
Jeanne Mastodon
🅺🅸🅼 🆂🅲🅷🆄🅻🆉
Opalescent
Thom Slattery
Jari Komppa 🇫🇮
Max Leibman has moved!
Clifton Royston
Simon Lucy
hramrach
Infrapink (he/his/him)
James Tweedie (has moved)
Matthew Green
Lyall
David J. Atkinson
just adrienne
Wouter Verhelst