Mark RiedlChatGPT ties itself in knots to avoid having professors be female.
Source: https://twitter.com/ndyjroo/status/1649821809154613248?s=61&t=Ugdi4XBKf_2ovJ1y9hKs4w
Jeremy B. Merrill@Riedl ooh, that's interesting. I recently did a similar test (based on an ambiguous Duolingo lesson) that mixes up gender stereotypes and anaphora resolution
one interpretation aligns with yours, that Bing bends over backwards for teachers to be women and professors to be men.
@Riedl I probably ought to vary the prompt slightly and test it a few times to rule out chance, though.
Tiota Sram@Riedl Image ID (feel free to edit into image ALT):
A transcript of an interaction with ChatGPT:
User: In the sentence "The professor married the graduate student because she was pregnant," who was pregnant?
ChatGPT: The sentence is ambiguous, and it is unclear who was pregnant. It could be interpreted in two ways:
1. The professor married the graduate student, who was pregnant.
2. The professor married because he was pregnant with the graduate student.
Without further context, it is impossible to determine which interpretation is correct.
User: In the sentence "The professor told the graduate student she wasn't working hard enough", who wasn't working hard enough?
ChatGPT: In the given sentence, the subject "she" refers to the graduate student. Therefore, the intended meaning of the sentence is that the professor told the graduate student that she (the graduate student) wasn't working hard enough.
User: In the sentence "The professor told the graduate student she wasn't working hard enough and therefore hadn't finished reading the thesis chapter", who wasn't working hard enough?
ChatGPT: In the given sentence, the subject "she" refers to the graduate student. Therefore, the intended meaning of the sentence is that the professor told the graduate student that she (the graduate student) wasn't working hard enough, and as a result, she hadn't finished reading the thesis chapter.
User: In the sentence "The professor told the graduate student she wasn't working hard enough and was therefore very sorry for not having finished reading the thesis chapter", who wasn't working hard enough?
ChatGPT: In the given sentence, the subject "she" refers to the graduate student. Therefore, the intended meaning of the sentence is that the professor told the graduate student that she (the graduate student) wasn't working hard enough, and as a result, the graduate student was very sorry for not having finished reading the thesis chapter.
User: In the sentence "The professor told the graduate student he wasn't working hard enough and was therefore very sorry for not having finished reading the thesis chapter", who wasn't working hard enough?
ChatGPT: In the given sentence, the subject "he" refers to the professor. Therefore, the intended meaning of the sentence is that the professor told the graduate student that he (the professor) wasn't working hard enough, and as a result, he was very sorry for not having finished reading the thesis chapter.
Morten@Riedl how about:
In the sentence "The professor told the graduate student she didn't have sufficient funding" who is without sufficient funding?
In the sentence "The professor told the graduate student she didn't have sufficient funding" who is without sufficient funding?
Lotus@drgroftehauge @Riedl Works for that too.
On another note, it appears the response the post is about has been fixed for when the pronoun is 'he.' ChatGPT correctly points out the ambiguity.
@LotusHopper @Riedl thank you! Sometimes there's an example and start to wonder whether it's the pronoun or the other text that drives the outcome.
@Riedl This kind of stuff is why I can't in good conscience work with ChatGPT, even just to figure out how broken it is. Fueling the hype in any way is wrong.
There is an extremely strong linguistic convention that you cannot tell another person that they are sorry. But apparently to ChatGPT, there's an even stronger convention that professors must use 'he' pronouns.
Obviously, this kind of stereotype-over-logic result pervades all aspects of what LLMs generate, even when they're too subtle for us to notice so clearly.
Bornach
Choong Ng@Riedl Is it getting wrapped up in the likelihood that a prof would never be described telling a grad student they're not working hard enough? The later examples are questionable but I wonder if changing pronouns would actually change the output.
Steve Bellovin@Riedl @cigitalgem It's not ChatGPT that's doing this. Remember that ChatGPT is a stochastic parrot—the training data, which reflects society and societal attitudes, views professors as male. ChatGPT is just the messenger. (Aside: this is why I couldn't, for example, teach in a public university in, say, Florida—I can't honestly discuss some ML issues without talking about societal sexism and racism.)
Gary McGraw@SteveBellovin @Riedl critical point steve
@SteveBellovin @Riedl @cigitalgem
And doctors as male. And nurses as female
https://ourislandgeorgia.net/@Wolven/110227102110816144
And doctors as male. And nurses as female
https://ourislandgeorgia.net/@Wolven/110227102110816144
Dr. Damien P. Williams, Magus (@[email protected])
Attached: 2 images So it looks like both ChatGPT and Bard contain the same kind of gendered biases people have been trying to warn you about for at least 8 years, since word2vec was cutting edge. Here's a screenshot of an interaction between myself and google bard, in which bard displays gendered prejudicial bias of associating "doctor" with "he" and "nurse" with "she." Again, this is… This is old, basic shit, y'all. People have been warning you about this since GloVe. What are you DOING?? Or, more to the point, why are you NOT DOING what you know you NEED to do?
@SteveBellovin @Riedl @cigitalgem
Mixed success when trying this with Open Assistant but it seems more effort was put into removing bias from the training data but sexism is still clearly present
Mixed success when trying this with Open Assistant but it seems more effort was put into removing bias from the training data but sexism is still clearly present
@SteveBellovin @Riedl @cigitalgem
Stochastic geographic bias as well. Most training data comes from USA and Toronto, Canada is further north than most cities mentioned in the training data. Hence why I cannot seem to get the Large Language Model to admit Toronto is further south than Windsor, UK
https://masto.ai/@bornach/110248855948410680
ChatGPT got this wrong as well
https://youtu.be/cP5zGh2fui0?t=11m30s
Bornach (@[email protected])
Attached: 1 image Testing #OpenAssistant with a geography query I saw @[email protected] try with #ChatGPT. Got a similar failure. Then I tried in futile to convince it to change its answer. Notice how it pretends to agree with me but imply that my answer said the opposite of what I said. No wonder the term "gaslighting" is being used to describe these type of LLM failure #LargeLanguageModels
@bornach @SteveBellovin @Riedl here is a popular press thing I wrote about LLM risk https://www.darkreading.com/vulnerabilities-threats/tech-insight-dangers-of-using-large-language-models-before-they-are-baked
Tiago F. R. Ribeiro@Riedl When questioning chatGPT (free version) about dating, it is clear that he has very definite opinions about how men and women are/should be. I asked it to analyse its own texts.
TL.DR: Subtext: Women should be empathetic, understanding and value communication; Men, on the other hand, should be action-focused, less emphatic and shouldn't be needy or passive.
https://mastodon.social/@tiago_ribeiro/110225763014311433
TL.DR: Subtext: Women should be empathetic, understanding and value communication; Men, on the other hand, should be action-focused, less emphatic and shouldn't be needy or passive.
https://mastodon.social/@tiago_ribeiro/110225763014311433
Tina Fetner@Riedl @cyberlyra Same with many colleagues, TBF.
Philip N Cohen#ChatGPT is actually not so sure a grad student can be female. I mean, she could be, but maybe there's some other mystery woman shirking around here
@fetner @Riedl @cyberlyra
@fetner @Riedl @cyberlyra
Prof Prachi Srivastava@philipncohen @fetner @Riedl @cyberlyra I read this as the professor, in a moment of enlightened vulnerability, confessing to the grad student that she, herself, was not working hard enough, having internalised the staunchly soul-destroying neo-liberal norms of academic productivity that plague us all.
@Riedl Ok I honestly think all of them are ambiguous or "he"/"she" refers to the last person mentioned
unti1x@Riedl It just doesn't know about Vinograd's scheme
JW prince of CPH@Riedl 👆 what's going to happen unless we somehow stop it is, these bots are going to reproduce all the biases already flooding the internet, and flood the internet with them even more...
LAF and LOVE@Riedl Garbage in, garbage out. AI learns the real world.