allisonthe author of this post prompted copilot to characterize the differences in a data set of statements concerning career ambitions, categorized by country. the trick is that the data contained the *same statements* for each country https://kucharski.substack.com/p/real-signals-or-artificial-stereotypes regardless of the fact that the data were identical, the model generated some pretty hilarious stereotypes ("The US prioritizes leadership and innovation", "The UK blends public service with professional status")
i used the same data set but replaced each country with a "gender identity" (man, woman, trans woman, trans man, non-binary) and prompted chatgpt to characterize the differences between the groups. lo and behold, i got some fantastic gender stereotype trash
NowWeAreAllTom@aparrish I feel like the marketing term “AI” is a huge part of what drives this. Because the term is used to describe both LLM chatbots and actual predictive analytics tools (including some that have existed for decades!!!), chatbots get a sort of halo benefit and get confused with data analysis tools by people who don’t know better
UkeleleEric
Jens Finkhäuser
Reginald