Dr. Damien P. Williams, MagusApr 19, 2023

So it looks like both ChatGPT and Bard contain the same kind of gendered biases people have been trying to warn you about for at least 8 years, since word2vec was cutting edge.

Here's a screenshot of an interaction between myself and google bard, in which bard displays gendered prejudicial bias of associating "doctor" with "he" and "nurse" with "she."

Again, this is… This is old, basic shit, y'all. People have been warning you about this since GloVe. What are you DOING??

Or, more to the point, why are you NOT DOING what you know you NEED to do?

Show threadsynlogicApr 19, 2023

@Wolven AFAIK when these trained neural nets answer a question theyre making an inference based on a kind of probabilistic bet on what the best answer might be. based on the input data set it trained on

if one researchs the gender percentage breakdown of doctors and nurses certain trends are seen in the real world. one source I saw said that around 60% of doctors are male, and 86% of nurses are women. which also conforms to my own firsthand (anecdotal) experience dealing with medical folks

Show threadDr. Damien P. Williams, MagusApr 19, 2023
@synlogic Yep, and there is a LOT OF LITERATURE about why that's bad.
Show threadBryan S. Weber
@Wolven @synlogic right, but do you know of any techniques that prevent it from happening? I am fairly confident the answer to your question is that... there isn't a solution to this problem that is implementable.
Apr 19, 2023 at 8:38pmWeb
Show threadsynlogicApr 19, 2023

@bsweber @Wolven I'm a programmer, and in broadstrokes know its possible to write code to augment the outputs of a NN inference such that it ensures all gender pronouns in the displayed text are "neutral" or "safe" or "ideologically correct" per any arbitrary set of rules

So it is possible, even if only via "duct tape". however, I dont think its necessary. And I dont think that exposing users to fact-aligned information is harmful. Ideology-aligned/censored information is bad as a general rule.

Show threadDr. Damien P. Williams, MagusApr 19, 2023

@synlogic @bsweber so you're cool with ideologically aligned information as long as it's been reinforced and systemically encoded into our societies, vut not ideologically aligned information that tried to undo it?

Or do you genuinely somehow think that the fact that fewer women are doctors is somehow "neutral?"

Show threadJobApr 22, 2023

@Wolven @synlogic @bsweber Funny enough in many countries the trend is strongly toward women either already outnumbering men, or outnumbering men soon because it's the older generation of doctors that is more male-biased, and they will eventually retire.

Wanna bet that the dudes who want "objective" information suddenly will have no problems with gender neutral descriptions of doctors in a decade?

https://www.oecd-ilibrary.org//sites/87e18004-en/index.html?itemId=/content/component/87e18004-en#figure-d1e24254

Home

OECD's dissemination platform for all published content - books, podcasts, serials and statistics

Show threadJohn V🏳️‍🌈Apr 25, 2023
@vanderZwan @Wolven @synlogic @bsweber that's very likely the case... it makes me think of the history of women in sports and suddenly a women's division would be set up when a woman or all women team beat a man or all men team. I have stopped being surprised by the lengths people will go to maintain white males as the group in power.
Show threadJoshua BarrettoApr 24, 2023
@Wolven @synlogic @bsweber Beautifully put. Lack of action is not lack of ideology.
Show threadAndrew FApr 20, 2023

@synlogic @bsweber @Wolven But in this case, the "fact-aligned information" isn't factual! There are contextual clues that the doctor is female and the nurse is male. The bot ignores this in favor of inferring something from a pronoun and ends up with nonsense like "the nurse apologized to the doctor because the doctor was late".

It's not ideology. Training an AI to rely less on statistical inferences about large populations with high variance (e.g. stereotyping) and more on explicit factual criteria is a good idea.

Show threadAndrew FApr 20, 2023

@synlogic @bsweber @Wolven Consider the counter example where I ask the bot to provide the boilerplate for a web app and I provide multiple contextual hints that I want a Ruby on Rails app (I mention Gems, ActiveRecord, DHH, etc.) And then the bot gives me a Python / Django app because that's statistically more common for this type of project. That'd be considered incorrect behavior, right? And the people who work on this probably spend a fair bit of time thinking of ways to train the model to avoid this situation.

So I don't think it's censorship to look into things like "can we get the bot to not assume a doctor is male if contextual clues indicate otherwise".

Show threadBryan S. WeberApr 20, 2023

@synlogic @Wolven

Interesting, I didn't think of that particular duct tape. Thanks!

Show threadAndrew FApr 19, 2023

@bsweber @Wolven @synlogic I'm not an expert but I think you can just do a bunch of reinforcement learning? Basically, just have humans keep providing these sort of prompts, downvote responses that make this mistake, explain the mistake. Do it enough times and it'll at least stop making the exact same mistake.

You can also just fill the model with lots of text explicitly designed to work around this (include more mentions in your corpus of female doctors and male nurses).

This is imperfect and time consuming but ... it's something.

Show threadAndrew FApr 19, 2023

@bsweber @Wolven @synlogic That said, OpenAI does a lot of this and it apparently doesn't work with ChatGPT yet.

So close ... and still so far.