Mastodawn

I think I have a genuine need for an #LLM. Can someone tell me if this is possible?

@openbenches contains ~40k text inscriptions.

Someone wants to know how many are dedicated to men, how many to women.

"To Grandma Sylvia" is obvious.
"To R Smith" is not.

Could an AI give a rough estimate of the gender of a subject?

Could it ignore text relating to who the inscription is from? "To Granny from Dave and Alice".

What would be the most accurate / cheapest / fastest / easiest tool to work with?

Not in any way I'd trust. LLMs tend to have huge gender biases that have, as of yet, gone unaddressed.

They are machines designed to make up plausible sounding responses that are difficult to concretely prove or falsify.

I don't recommend it.

@ajroach42 I guess that's part of the thing I'm trying to understand.

I can see how a biased system might insist that all "Sam"s are default male - or might not understand non-English languages.

But if I can see the assumptions that it makes, that could be very helpful.

Generally speaking, we don't have any insight into the specifics. These machines are essentially black boxes.

What we know about the things they get wrong come from people who are studying the outputs.

But the outputs are inconsistent.

@ajroach42 Yes, it's the outputs I'd like to study.

E.g.
Sylvia - Female 100% confidence
Joe - Male 100%
Jo - Female 70%
Sam - Male 80%
etc.

I appreciate it might be different each run, but would allow me to see how dodgy it was.

I haven't done that kind of testing personally, but I know there are some python utilities designed to do that kind of bias testing.

i haven't done much ML work since OpenAI hit the scene, so the specific analytical stuff I've done is very dated.

I've mostly stepped back and depended on studies others are posting.