Kevin FoxBefore trusting an AI to tell you about stuff you don’t know, ask it to tell you about things you’re an expert in.
JoshData 🐢@kfury I always start by asking 'em about things that I'm the only person to have written about. If it gives a correct answer, it's copyright laundering. If it gives an incorrect answer, then I know the AI is garbage.
Resuna
M.S. Bellows, Jr.
Tito Swineflu@msbellows @resuna @josh @kfury I have a name that I know belongs to only one other person in the world. ChatGPT thinks I'm a late 19th-century composer and it attributed an opera to me that a quick google search could find the actual composer of.
Generally Salty@tito_swineflu @msbellows @resuna @josh @kfury Good to know that I'm still a ghost:
Me: Tell me about <me>. He was born in California in <year>.
ChatGPT: I couldn't find any notable public figures or individuals of significance named <me> born in California in <year> in the available information.
I'm not an individual of significance? FU!
@david @msbellows @resuna @josh @kfury some of the bots do this to me too, which is the correct answer.
@kfury @tito_swineflu @david @resuna @josh I've gotten the same "no significance" response, which is fine except that I've filed briefs with the U.S. Supreme Court, am named as counsel of record in published appellate opinions (both state and federal), was a regular columnist for Huffington Post in 2008, and have published in several other outlets including The Guardian. It's not an ego thing, it's a "this dumb computer doesn't even steal data well" thing.
@msbellows @kfury @tito_swineflu @resuna @josh
Right, I've had several technical books published and wrote some magazine articles, but that was all over 20 years ago. Still, I'm out there, stupid bot.
Jeff Grigg@david @msbellows @kfury @tito_swineflu @resuna @josh
I said
"Tell me about Jeff Grigg, the software developer."
and it gave me the whole "not notable" thing.
That Hurts My Feelings!!!
Same for
"Find online postings of Jeff Grigg, the software developer."
.
On the other hand, a Google search of the same could easily find me (and another person or two with the same name).
I've been posting online for well over twenty years. I'm *not* that hard to find.
Scott Jenson@kfury
When I asked about me I got nearly the identical answer. There is apparently a fairly generic answer for "industry designers"
When I asked about me I got nearly the identical answer. There is apparently a fairly generic answer for "industry designers"
@scottjenson @kfury Contrarian take on this whole thread: we're all immune from hackers and thieves: we're broke and boring,.
(Except for Kevin, who inevitably will become the victim of his own fame.)
@kfury @scottjenson (Light my pipe, settle back in chair) Sounds like you have an interesting story to tell.
@msbellows @resuna @josh @kfury Ironically, I am modestly famous. Luckily, it's in a way that doesn't make it on to the internet.
William Gunn@josh @kfury How about this: Ask it questions where learning is asymmetrical. Where it's hard to learn but easy to validate. Questions about tax regulations, for example, or historical facts, or about any complex subject where there's a lot written, it's hard to know where to look to find the answer, but you know it when you see it
zeno's paradox speedrunneractually. i ask it stuff that sounds factual but isn't and see if it hallucinates a factual answer
std::polar@pixelnull @kfury But that's leading. This will even get you far with humans. The claim was never that AI was good at facts. So don't ask them for facts.
okanogen VerminEnemyFromWithin
FeralRobots@sehe @pixelnull @kfury I would argue that it's very much claimed that genAI is good with facts. That's very clearly implicit if it's being marketed as a search engine.
Mike Macgirvin 🖥️I did, and that's why I won't use it.
jay@kfury this sadly won't help my Gell-Mann amnesia
Alexander Schmidt-Lebuhn@kfury Ah yes. As a botanist, I asked ChatGPT for the native range of one of the worst weeds on the planet, where copious information is available online that they would have used as training data. In response, it listed most of the native range as part of the invaded range. Not even a complex question of understanding, just a fact look-up, still botched the answer.
Next step: not falling for Gell-Mann amnesia, or a test like this will be for nought.
MarkSThat's because LLMs do not "look up facts". Rather, they construct plausible sentences using the statistical relationships between words. If that sentence is not factual, tough.
fmc01@markstahl @anschmidtlebuhn @kfury Precisely - why on earth would you expect an AI/LLM to give you anything other than a seemingly plausible response based solely on the statistical relationship between words - you’d have more likelihood of a reliable reply from cadavers resurrected with lightning bolts.
@frankcat @anschmidtlebuhn @kfury
This is where people confuse LLMs with intelligence.
The human brain makes a model of the world, which it is constantly testing against experience. For humans, language is merely the interface we use to communicate our internal model to other human beings. It's a lossy translation of a hidden model of reality that is itself non-verbal.
But in LLMs, words are all there is. There is no underlying model of reality behind them. It's just words strung together in ways that imitate human communication.
The phase "stochastic parrot" is extremely accurate.
@frankcat @anschmidtlebuhn @kfury
An LLM will literally "believe" anything you tell it. Take a look at what they are asking the Gab AI to believe.
VessOnSecurity (@[email protected])
Attached: 1 image Somebody managed to coax the Gab AI chatbot to reveal its prompt:
Pepper1700 
@kfury Failure rate: 100%
Larry@kfury
Back when (circa Vietnam Conflict,), one of my teachers used the same test against Time Magazine. If it was wrong about horse breeding and racing (of which she was knowing,) then the topics she was knowless of, they were likely to be as inaccurate.
Back when (circa Vietnam Conflict,), one of my teachers used the same test against Time Magazine. If it was wrong about horse breeding and racing (of which she was knowing,) then the topics she was knowless of, they were likely to be as inaccurate.
Colin Watson
Western Exposure@kfury I did. Not only did it make stuff up, it also neglected to tell me about the most important bits...
Cykonot@kfury also, listen to your favorite podcast (or science journalist or whatever) talk about something you really know 😬
Jo - pièce de résistance@kfury or don’t use AI at all. Even better. Not worth the burn.
MOVED to @@kfury
This concept also applies to (local) news outlets like the evening news programs on broadcast TV.
This concept also applies to (local) news outlets like the evening news programs on broadcast TV.
Ampelios@kfury I liked a remark by a guy working for Musk who was like, "I don't know anything about cars, but I know a lot about software, and the things this guy says about software tell me I should never get in one of his cars."
Adlangx@kfury Lie to me AI baby.
Michael Labowicz@kfury this is the approach I use when talking about the limitations of AI with those that seem to be enamored with the tech
zilla@kfury Kevin that is a great strategy. It works well and one can spot the mistakes it makes.
@kfury last year I ask it for information on Nino 3.4 temperature variations and it would obviously give me the wrong data, and when I would correct it (Bing) it would be like "You are correct, I am sorry for the confusion"). 1) Don't trust AI and 2) don't ask it to proof read your work. It is a thief.
Jessamyn@kfury This is what I do "Give me five contemporary books about time travel" and then see what it says. They tend to suggest H.G. Wells.
@jessamyn Oh I would to compare contemporary time travel book lists with you! That’s my jam.
@kfury If my site search is working (and it may not be) you can see the ones I've read.
https://jessamyn.info/booklist?s=%22time+travel%22
If you see a major one you've loved that I've missed (no nazis, no sexual assault, those are my only "nopes") let me know.
Wrong Place Wrong Time was the one I read most recently which I enjoyed, not sci fi at all except for, y'know, the time travel bit.
@jessamyn Thanks for this! Several I would recommend are in there, and others I’m adding to my list.
Two I would add are “Marooned in Realtime” by Vernor Vinge and “The Accidental Time Machine” by Joe Haldeman, both of which are entirely (or almost entirely) forward-only stories.
Q. Edwards@kfury What if you aren't an expert in anything?
Cluster Fcku@kfury I have friends who talk like LLMs
elgregor@kfury A lot of media will fail this test and you don't even have to be much of an expert. Sometimes it even happens when the incorrect stuff is the main focus of the material.
@kfury The same rule applies to papers published in Science and Nature, unfortunately
brandon@kfury the problem is, many of the managers and executives making decisions to implement AI are "experts" primarily in meaningless business prattle, which is precisely the sort of thing AI excels at extruding.
Rob Neppell@kfury Fun approach to take with journalists, too.
Ecte@kfury This! I do this often. It is actually rather informative
Gene Cowan 🏳️🌈@kfury @lisamelton This works for news outlets, too.
@kfury my first set of questions was about migrating Drupal 7 sites to a new server. IIRC I gave it 6-7 tries; they all got it mostly right - but each time they got something critical wrong that would have doomed the whole effort
David Nash@kfury @ShaulaEvans That was literally the first thing I did with an LLM: my prompt to ChatGPT v3 was “Describe the extrasolar planets around the star <<totally fake star identifier>>.”
I verified the fake star ID wasn’t referenced in Google or other major search engines as an error or possibly an unfortunate coincidental usage in fiction somewhere (results contained some near-miss real IDs at most).
Result. Paragraphs and paragraphs of bullshit, all referencing the fake star ID and sourced from God only knows where.
Some Kind Of Garf@kfury But then you'll have no reason to trustOoooohhhh...
@martlund 🌲🐶🇸🇪🇨🇱🌈@kfury I enjoy disputing AI all the time and it tickles me when it apologizes for not answering me correctly or completely.
Teri Yaki 🖌️🎹🚲💻@kfury That’s how you find out, but why ask in the first place?