mhoyeNov 22, 2023
Microsoft paid money for this. A lot of money. And they gave it to us for free.
Show threadmhoyeNov 22, 2023

I'm looking at a demo of this paper right now, which is kind of interesting - https://arxiv.org/pdf/2005.11401.pdf - but... it relies, the same way most AI models do, on a tectonic amount of human curation effort that's gone on behind the scenes to make it work.

I mean, it's nice I guess, and there's some nice features in a low-K-threshold, high-quality-training-data situation, but it sure looks like this will all fall apart if you point it at large, unvetted or adversarial data sets.

Show threadBenjamin SmedbergNov 22, 2023
@mhoye I'm curious whether the problem is not the AI, but the expectation of "scaling"... that is, the way we'd need to train AIs is roughly the same way we need to train baby humans: "Here honey, this is a good book, read this one." "I liked this article but I'm not sure how I feel about X." "No no, don't lick the wall socket."
Show threadBenjamin SmedbergNov 22, 2023

@mhoye also... it seems like most AI people have given up on...

1. Letting the AI ask questions to test its understanding (toddler)
2. Accepting corrections as input (elementary school).
3. Being able to research & cite sources (high school)
4. Being able to say "here's what I don't know" (college)

Show threadmhoyeNov 22, 2023
@bsmedberg My incomplete understanding of the state of the art is that there isn't really a path from where we are to self-interrogation or -correction in the models themselves, but that human feedback from ml-based chat services - "I don't think that's right..." - is driving those improvements. The biggest insights of the current AI cycle aren't coming from the ML tech, but from the questions being asked of it (which is, in yet another sense, just another mechanism for capturing free labor...)
Show threadBenjamin SmedbergNov 22, 2023
@mhoye exactly. I'm not in the generative-AI business, but I am in the trained-model business, and I'm astounded by how little there is around "learning"... it's all train-from-scratch and models that cannot explain themselves or be corrected at all.
Show threadCassandrichNov 22, 2023
@bsmedberg @mhoye Same as how, from time to time, folks fascinated by the engineering side of cryptocurrencies would come up with some ideas that would actually do useful things... and everyone in the domain promptly ignored them because the point was never to do useful things, it was to make a facade of solving grandiose-sounding made-up problems as an excuse for scamming people.
Show threadmhoye
@dalias @bsmedberg So, if you think of ML tools as "mechanical pattern recognition and repetition" - which is, fortunately _what they are_ - rather than falling into the trap of anthropomorphizing them even subtly (e.g. saying "learning" rather than "encoding") then their real utility becomes clear. I think there is a real tool here, somewhat-useful in itself but better as a dowsing rod for future improvement of tools & training, e.g. "Write this code" as hints for future language improvement.
Nov 22, 2023 at 9:45pmWeb