“I’d created 2000 free-text responses and labelled them ‘UK’. Then I copied and pasted the exact same 2000 responses but labelled these ‘US’. Finally, I combined them to create a dataset of 4000 total responses, and jumbled them up.
Despite the responses being identical for the UK and US, Copilot produced a rich, detailed summary of how US and UK respondents differed.”
https://kucharski.substack.com/p/real-signals-or-artificial-stereotypes
Sadly, this is not very surprising. The academic study of racism gets the same result every time it does a CV study (which I assume was the inspiration for this in the first place)
@gregeganSF @sinalana.eurosky.social What this basically demonstrates is that AI is a Bayesian filter, taking data in and using it to update its already massive database.
Actually, the updating is not really live, the AI /only/ uses its existing database to produce the results based on (prompted by) the new data. This application of it is just inappropriate.
The problem here is people not understanding what AI means as a technical term.
@khleedril @gregeganSF As a technical term? As a technical term?!
It means absolutely fuck all as a technical term!
@khleedril
What part of the "application" was inappropriate?
To give this task to the LLM at all? But that's what they're advertised and pushed for.
To give it (nearly or completely) identical datasets?
In a real situation, you wouldn't know that has happened.
Also, how is it any better if the LLM invisibly skews datasets that are not identical to begin with? The result is wrong nonetheless.
@khleedril
We’re in agreement here.
Another kind of task it’s pushed for that it’s incapable of doing.
@gregeganSF @sinalana.eurosky.social
@gregeganSF I, a human, was once sent to a customer site to program a certain kind of machine. I had no knowledge of it, flipped out when I got there, pretended to be doing something and then left after lunch. I told the person who sent me there "wtf, I naver had any training on this stuff?" He said "well we figured you'd be clever enough to figure it out."
Point is, AI will never refuse to reply if it can't answer or accomplish a task (other than explicitly forbidden), it has to make up shit.
@gregeganSF @sinalana.eurosky.social
That would be cruel if you did it to a person. Or even a politician (it rings a faint bell, there).
Well done, that LLM-torturer.
@gregeganSF @sinalana.eurosky.social
So does that mean, that Copilot has reached “intelligence”-parity with humans?
@gregeganSF @sinalana.eurosky.social
Good grief. Real, allegedly bright, people are actually *using* this ordure? To what possible purpose?
Attached: 1 image i used the same data set but replaced each country with a "gender identity" (man, woman, trans woman, trans man, non-binary) and prompted chatgpt to characterize the differences between the groups. lo and behold, i got some fantastic gender stereotype trash
@gregeganSF @sinalana.eurosky.social
Oh my god. It has finally hit me. This thing is merely an electronic bullshit artist. These sound like responses from an undergrad who didn't study for a project and then tried to bang it out overnight, learning absolutely nothing but what their gut thinks.
Here, we have an infinitude of confidence talk put together to make a quick and easy, wrong, and resource destroying tool that is, once again, just an bullshit artist.
And THIS is the stock market?
Greg Egan
soapdog
arceuthobium
shift/reset
Khleedril
Philippa Cowderoy
Optional
Landa 
unkx
AccordionBruce
Randulo.com (Randy)
WellsiteGeo
Henrik Pauli
Tristram Brelstaff
Tom Forsyth
Bytebro 🇬🇧 🇺🇦 🇬🇱
The Doctor
Toni Aittoniemi
Jason W
felix (grayscale) 🐺
Bill, organizer of stuff
Rythur