allisonthe author of this post prompted copilot to characterize the differences in a data set of statements concerning career ambitions, categorized by country. the trick is that the data contained the *same statements* for each country https://kucharski.substack.com/p/real-signals-or-artificial-stereotypes regardless of the fact that the data were identical, the model generated some pretty hilarious stereotypes ("The US prioritizes leadership and innovation", "The UK blends public service with professional status")
i used the same data set but replaced each country with a "gender identity" (man, woman, trans woman, trans man, non-binary) and prompted chatgpt to characterize the differences between the groups. lo and behold, i got some fantastic gender stereotype trash
ratatouille as a service@aparrish someone was telling me they use this stuff to do all their data cleaning and analysis at work and i asked how they knew it was giving them the right answers and they seemed confused by the question
S. Lott
s00ner🌈@slott56 @hannah @aparrish I've discovered that there are a shocking number of people who simply don't consider accuracy to be important. Their worldview doesn't care about what is true and what isn't true. It's about marking the task complete and moving on to the next for them and correctness isn't part of the equation.
ermo | Rune Morling@jeffers00n It could also be that the incentive structure (read: "Keeping my Job") is misaligned, in that it relies on the rate of completed tasks, not their correctness?
People whose livelihoods rely on not getting fired are not incentivized to engage in the finer points of whether LLM output is actually trustworthy, if the people dictating the incentives also do not care.
The question is who gets to carry the can once this shit show collapses under its own weight.
Bryan Wright@jeffers00n @slott56 @hannah @aparrish
I think we teach kids this in school. It's about turning in a 5-page essay, properly formatted, by the assignment's due date. Content is secondary to form and schedule.