Charlie StrossMay 30

A map of Northern Ireland, as horked up by ChatGPT. (See alt text for the true horror.)

Do not trust ChatGPT to do your geography homework!

That is all.

Show threadMark LynchMay 30
@cstross that monstrosity was shown in the middle of a segment during BBC NI’s “The View” last night, that consisted of local tech sector execs talking breathlessly about the inevitability of AI-driven productivity gains. I had to turn the TV off in disgust and actually went to bed feeling depressed at how utterly fucking unhinged is the sector in which I work. Why can’t these people see the blindingly obvious?
Show threadDavid Penfold May 30
@mark_f_lynch It's a grift that's keeping the economic plates spinning.
@cstross
Show threadMark Lynch
@davep @cstross Sam Altman’s only observable talent is marketing this shite to people who are simultaneously too dumb to realise they’re being scammed and in charge of the budgets he needs to line his pockets
May 30 at 7:47pmWeb
×
Charlie StrossMay 30

A map of Northern Ireland, as horked up by ChatGPT. (See alt text for the true horror.)

Do not trust ChatGPT to do your geography homework!

That is all.

Show threadJosh May 30
@cstross at least County Down is properly placed at the bottom!
Show threadMark KraftMay 30
@cstross Unster famously being where they play loud "oonz, oonz, oonz" electronic dance music until 3:00 a.m. every night.
Show threadFrank SkorniaMay 30
@cstross I can't wait for the first war to be fought over a border decided on by an AI-generated map </sarcasm>
Show threadTed MielczarekMay 30
@fskornia @cstross the US government is already using this garbage to write policy (like the tariff nonsense), so it's only a matter of time.
Show threadCharlie StrossMay 30

@tedmielczarek @fskornia Latest elsenet is that RFK Jr. used an LLM to write a whole chunk of healthcare policy claiming vaccines are poisonous and there's a childrens' health crisis caused by (waves hands).

We're all going to die to pad out Sam Altman's bank account.

Show threadgabriele renziMay 30
@fskornia @cstross to be fair, there's a chance the US will start a war over Greenland cause Trump.does not understand map projections, no need for ai slop
Show threadFrank SkorniaMay 30
@riffraff @cstross He doesn't need to understand anything about maps. He has a Sharpie and can make the map show whatever he wants.
Show threadAngus McIntyreMay 30
@fskornia @cstross In fairness, those six-fingered freaks in Trnsmoldistria brought it on themselves by attempting to annex our province of NaN, which -- as this map clearly demonstrates -- has been part of Bratwurstislava since Attila the Hen defeated Genghis Khan Noonien Singh at the bottle of Costco Polje.
Show threadAdonnenMay 30
@angusm @fskornia @cstross KHAAAAAAAAN
Show threadΣ(i³) = (Σi)²May 31
@fskornia @cstross Eh, most of the wars fought in the last 50 years in the middle east, southeast Asia or Africa were due to some border drawn by some British bureaucrat in some office. It really doesn't take AI to mess up this stuff. 🤷‍♂️
Show threadWalter van HolstMay 30
@cstross Seen a similar botched map of Africa....
Show threadPaul DryeMay 30
@cstross Arrmagh is presumably where pirates come from.
Show threadJeremy KahnMay 30

@cstross

I have never seen a more clear example of how ChatGPT & co are, at root, a distillation of

a short-attention-span American teenager bullshitting his way through a college undergrad presentation

Philomena Cunk would *never*

Show threadBeans_pleaseMay 30
@cstross Londogrl is beautiful this time of year though.
Show threadGraham CMay 31
@beans_please @cstross the Londogrl movement did wonders for DIY punk
Show threadPsittacus cafeinaMay 30

@cstross

Are you telling me ARRMAGH isn't a real place? Do better Ireland smh

Show threadThe Queen of Weltschmerz🏳️‍⚧️May 30
@caffetiel @cstross I assume that’s where Irish pirates come from
Show threadCait the Proud Trans WomanMay 30

@cstross

I just didn't know Londongrl was in Unster, is all.
https://www.youtube.com/watch?v=PSi4CDANuUY

London Girl - The Pogues

YouTube
Show threadJames WellsMay 30
@cstross
I see down, but where is up, and you mis-spelled Conan.
Show threadAlgaemanMay 30
@cstross I've been planning a trip to Londogrl for years.
Show threadgjmMay 30

@cstross Looks perfectly accurate to me.

Here's what Claude (Sonnet 4) produced for me. It's both much better and much worse.

Show threadCadbury MooseMay 30

@gjm @cstross

At least it got rid of the pirates in Arrmagh.

Show threadJeff CraigMay 30
@gjm @cstross The inclusion of the map scale is true 🧑‍🍳 💋 territory.
Show threadtautologyMay 30
@gjm @cstross I wanted to have a go and asked GPT to draw Yorkshire annotating the cities. It looks about right, until you see the cities are all over the place.
Show threadWinwaedMay 31

@tautology @gjm @cstross no idea what those rivers are doing, but at least "Here be dragons" is correct!

(I was born in one of those cities and grew up in another - they're the right cities but in the wrong order , to paraphrase someone from aforementioned dragon territory)

Show threadOliMay 30
@gjm @cstross
Belfast looks watery, like a new venice
Show threadonnobMay 31

@gjm @cstross Oh look, Claude generated the map of the Netherlands.

The map after Antarctica and Greenland melt, by the look of it…

Show threadMark LynchMay 30
@cstross that monstrosity was shown in the middle of a segment during BBC NI’s “The View” last night, that consisted of local tech sector execs talking breathlessly about the inevitability of AI-driven productivity gains. I had to turn the TV off in disgust and actually went to bed feeling depressed at how utterly fucking unhinged is the sector in which I work. Why can’t these people see the blindingly obvious?
Show threadDavid Penfold May 30
@mark_f_lynch It's a grift that's keeping the economic plates spinning.
@cstross
Show threadMark LynchMay 30
@davep @cstross Sam Altman’s only observable talent is marketing this shite to people who are simultaneously too dumb to realise they’re being scammed and in charge of the budgets he needs to line his pockets
Show threadThomas Lobig🐔🐔May 30
@mark_f_lynch I guess because Money (capital m) @cstross
Show threadSteve HerseyMay 30

@mark_f_lynch @cstross

Credibly attributed top Upton Sinclair:

“It is difficult to get a man to understand something, when his salary depends upon his not understanding it!”

Show threadblackcoatMay 30
@mark_f_lynch @cstross "It is difficult to get a man to understand something, when his salary depends on his not understanding it."
Show threadLimey_TankMay 30
@cstross That is beyond the pale…
Show threadThomas Lobig🐔🐔May 30
@cstross saw basically the same thread in German a few days ago. But it had dudes yelling at the OP they would be using the wrong tool for the job etc. Was as depressing as the shitty map from the LLM.
Show threadMagnus AhltorpMay 30
@Tom_ofB @cstross Yes, the whole selling point of using LLMs in the first place is to have a tool that interacts with the user in natural language. If the user has to know anything about how to use the tool or what the tool is good for, the tool has failed completely.
Show threadGLCMay 30

@cstross

My first impression was that Unster wasn't bad but then I found Northern Ireland which is indeed somewhat doubtful.

Points for Londogrl. Neither one thing nor the other as Churchill remarked on a very different occasion. Which is more or less ChatGPT's superpower.

Not bad for a fantasy novel frontispiece, needs more mountains. Or cowbell, I don't know.

Show threadAntiqueightMay 30
@cstross I kinda like LondoGrl though. Sounds like a fic based login...
Show threadChris BeattieMay 30
@cstross “logic dictates that Down must be at the bottom” 😆
Show threadSteve HerseyMay 30
@cstross
You can shorten that to simply "Do not trust ChatGPT" with no loss of accuracy.
Show threadDeterioratedStuccoMay 30
@n1xnx @cstross
"Do not use ChatGPT" uses fewer letters.
Show threadSteve HerseyMay 31
@SoftwareTheron @cstross
Brevity is truly the soul of wit!
Show threadOrion (he/him)May 30

@cstross When I show stuff like this to "AI" boosters, their response is, "Sure, right, that's terrible, but you're asking it to do something it's not good at! Let's focus on what it *is* good at!"

To which I respond: The thing it's "good" at is lying with confidence. Which isn't, like, actually *good*.

Show threadCharlie StrossMay 30

@orionkidder But it's not "lying". To lie you need a model of truth and falsehood. LLMs don't have an internal model of the external world to provide context, all they've got is a probability distribution of one fragment of text being followed by another, entirely divorced from context. They have no means to determine whether the text they're producing in response to a prompt is a valid answer or just an answer-shaped string.

There is no intellect here, just text.

Show threadrobMay 30
@cstross @orionkidder "There is no intellect here, just text." Correct - and so many people don't understand this.
Show threadOrion (he/him)May 30
@cstross True! Falsehood but not lies.
Show threadAaron In MinnesotaMay 30
@orionkidder @cstross Like fanfiction based on reality. It might seem to be in the same universe, so it can feel "correct" or plausible, but it can't be "right" in any cononical sense
Show threadJosh RiversMay 30
@cstross @orionkidder There is a thread in Stephenson’s Baroque Cycle where what we call tokenization was being explored and researched (in the times of Newton). I’m fascinated by how much value LLMs get out of lossy compression of language, but the leap from language to truth is too much. We are asking an MP3 decoder to be Beethoven…and wrestling with that difference.
Show threadyuhasz01May 30

@cstross @orionkidder

LLM are at best agnostic, but in real terms reflect biases and ideology of their creators.

Show threadllewellyMay 30
@cstross @orionkidder
technically, the lying is done by those who created it.
Show threadAtomic OrbitalsMay 31
@cstross @orionkidder In _On Bullshit_ Harry Frankfurt distinguished between truth tellers, who present the truth, liars, who conceal it, and bullshitters, who don't care about it. He concluded that bullshitting was more insidious than lying. Now we have automated bullshitters, which would not look out of place in a Stanislaw Lem tale.
Show threadLanda May 31

@geoglyphentropy to be frank I did not expect Lem to be the Sf author I read when young who‘d be the so helpful to understand the world we built.

I probably should have.

@cstross @orionkidder

Show threadokanogen VerminEnemyFromWithinMay 31
@cstross @orionkidder
It just creates a model of an authoritative sentence or paragraph or answer in response to an input question.
That is what people don't understand.
It isn't even an answer, it is a *model* of an answer.
Show threadHighlandLawyerMay 31
@Okanogen @cstross @orionkidder
An answeroid
Show threadPete / SyllopsiumMay 31

@cstross @orionkidder

Yes, of course you are technically correct, the 'best kind' of correctness - but this is unhelpful. As you well know language uses shortcuts and 'lying' is close enough.

To be particularly useful an LLM should be able to cite sources for each section of its responses, but then I suppose it wouldn't be an LLM and it'd be even more obvious there's plenty of A, and no I.

Show threadOrion (he/him)May 31
@syllopsium @cstross He's not just technically correct, IMO. He's making a point: attributing a conscious mind to these systems--eg, by calling their falsehoods "hallucinations--plays into the misperception that they think, have consciousness, are alive, etc.
Show threadCB WrightMay 31
@orionkidder @syllopsium @cstross I don't disagree with this, but we need a short pithy word that will describe the same thing without ascribing consciousness to AI, or hallucination will win, because it's evocative of the problem and easy to remember and despite all the unwanted connotations that makes it _useful_.
Show threadOrion (he/him)May 31

@ubersoft @syllopsium @cstross No, I hear you. The word "lie" has a *punch* to it that very few other words do. That's why I initially used it. It's why journalists are so unwilling to use it to describe what powerful people say.

I don't have a good rhetorical solution for this situation.

Show threadjimmynohandsMay 31
@orionkidder @ubersoft @syllopsium @cstross
As @geoglyphentropy points out, "bullshit" in the Frankfurt sense - "speech without regard to truth" - may work?
Show threadCB WrightMay 31
@jimmynohands @orionkidder @syllopsium @cstross @geoglyphentropy pithy, but probably won't get taken up in mainstream conversation I'm afraid. :-D