lainling
Dan LevensteinThe funny thing about LLMs is they’re not good for knowledge work because they sometimes make up stuff that doesn’t exist and they’re not good for creative work because they sometimes make up stuff that does exist.
Turns out this was a banger of a toot. Do we do soundclouds on #mastodon?
jonny (nonvenomous)@dlevenstein
Legally obligated to link to the funkwhale instance with one track uploaded
Legally obligated to link to the funkwhale instance with one track uploaded
clacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛@dlevenstein Yeah, what's your ... castapod, funkwhale, peertube or something? Patreon? Favorite charitable cause?
@dlevenstein and then LLM ppl will say "RAG fixes this," but then the problem becomes RAG-LLMs are not good for knowledge work because it's impossible to tell when they are making stuff up, and they're not good for creative work because it's impossible to tell when they are making stuff up.
Steven Bodzin@dlevenstein that is such a smart encapsulation of the problem that it implies a useful solution. It might be possible to have a dial or switch to tell these programs whether you want more facts or more fiction, more accurate portrayals of prior work or a novel creation.
Andrew@guacamayan I mean that would solve it, sure, but there's no reason to think it can possibly be built. "Fact" and "fiction" aren't concepts that LLMs understand — all they know is which words in which order are shaped most like the text in their training corpus. That's it. It's like asking a lottery machine to preferentially choose odd or even numbers — it can't be done, because the balls are mechanically identical.
Jennifer@andrewt @guacamayan I think it's absolutely crazy that people are using AI for things like writing contracts.
Urban Hermit@guacamayan @dlevenstein maybe what we need is masters level knowledge of how to write better queries? Assuming we could get accurate or creative answers based on asking the talking data base more rigorously defined questions?
Vijay PremaIt's usually good for coming up with marketing copy. That's what my wife uses it for....
...as even human written marketing copy is rarely accurate.
Nikhil 🐧@dlevenstein AI can't even draw a correct Rubik's cube in a solved state. A regular 3x3x3 Rubik's cube, where each side has a different colour. It doesn't seem to be capable. I gave it a few tries:
Nikhil 🐧 (@[email protected])
Attached: 1 image I didn't expect AI to have such difficulties drawing a Rubik's Cube. This picture is the position that I want AI to draw the cube: Basically a (solved) cube with one side rotated by 45°. So I tried a couple of times but noticed that the AI doesn't even manage to draw a regular Rubik's Cube where there is no rotation but with correct colors. I'll be uploading some pics next. #ai #rubikscube #chatgpt (1/x)
Riley S. Faelan@dlevenstein Just like Trofim Lysenko, who published many original ideas and many true ideas, the only two minor flaws being that his original ideas weren't true and his true ideas weren't original.
mybarkingdogs@dlevenstein I honestly think the only real uses for creative work LLMs will essentially be "inspiration generators," e.g. providing a place to get started, similar to doing a mood board or using an old text prompt list or looking through stock photos etc.
And IMO that would be defensible, because (as an example) you don't have to credit a photo of say New York or Tokyo for inspiring a sci-fi story set in a large metropolis on a distant star many millennia from now
mnemonicoverload@dlevenstein @mybarkingdogs Most expensive mood board, ever.
mtnrbq65@mnemonicoverload @dlevenstein @mybarkingdogs
My thought exactly!
Troed Sångberg@dlevenstein It's however a perfect recipe for innovating.
SturmUndDranger@dlevenstein This toot is incredibly funny if you’re more familiar with LLM as denoting a person who has a Master’s Degree in Law than anything to do with AI.
Tom Boutell@dlevenstein Every single thing they produce has to be reviewed by someone competent to review it, yup. That still leaves a large amount of usefulness, but also creates an endless series of fails.
alastair87 (old account)@dlevenstein And as we all know, humans never do either of those things . . .
@dlevenstein But more seriously, you should not be treating these things as semi-autonomous agents working alone. It's more like asking someone to do the research for you, and it's also inherent in that process that the findings may be faulty or biased, and that's exactly how it should be treated.
Clearly LLMs are not on par with competent human experts but nor is there any human who can parse, collate and even to some extent synthesise from such vast amounts of data at such speed.
Clearly LLMs are not on par with competent human experts but nor is there any human who can parse, collate and even to some extent synthesise from such vast amounts of data at such speed.
StarkRG@dlevenstein The problem with LLMs is that they never provide the source. (because they literally can't)
mmby@dlevenstein that's of no interest to users or investors who have already confused the map with the territory
this is like the rise of movies, but instead of the illusion of motion it's the illusion of mind
Utility Nerd@dlevenstein
Same with my brain.
Same with my brain.
Very Human Robot@dlevenstein to be fair, human workers have the same problem, but have the ability to plan ahead and double check before declaring success.
Xeno the CaveSpider 🕷🏳️🌈@dlevenstein But they're very helpfull when learning yourself a foreign language as they will answer you any grammar related question in great detail because THAT is their core strength!
Also never getting tired in exercising dialogues with virtual barkeepers and such 😍
Kip@fxk8y I wouldn't trust grammar explanations-- it's likely making up things that arent true even there 😬 being able to use grammar doesnt mean understanding it even for humans, and llms just create statistically probable stences
@kip The problem is that googling such stuff is nearly impossible in the first place, but after asking how the concept at play is called I was able to quickly verify that the answers were true.
Did this some dozen times until I gained some trust but I do the checks now and then when the answer seems odd or Duolingo rejects the sentence (happened only once so far).
Sadly Duolingo doesn't explain any grammar (at least not in their norwegian course) so LLMs are really helpfull in for example checking if something is a general rule or just valid for the few words I saw
Nicholas Nethercote@dlevenstein Nice!
Another one I like: "why would I bother reading something that nobody could be bothered to write?"
mypalmike@dlevenstein "My colleagues are not good at knowledge work because they are *sometimes* wrong."
Reinis Lazda@mypalmike @dlevenstein Yeah, it is no better than a colleague who can sometimes be brilliant, but sometimes a disaster with the same level of certainty, and you have to dig deep to find out which is which.
If you don't have such a colleague and feel like you should, then LLM is your option!
If you don't have such a colleague and feel like you should, then LLM is your option!
@ReinisLazda @dlevenstein I largely agree.
Used as a tool (as I do every day), the brilliant stuff is very much worth it. Very few people are able to distill expertise as clearly as an LLM. I'm ok with having to fact check, although it's not often that I ask anything purely factual.
Asking an LLM something like, "Summarize (some fairly deep subtopic of a subject)" tends to yield results fast superior to any other method, including asking an expert.
@dlevenstein In short, they behave just like humans?
Markus F. Frisch@dlevenstein Makes me think they might be a natural replacement for some kinds of business consulting …. 🤔
THIS ACCOUNT HAS MOVED