Mastodawn

LLMs consistently pick resumes they generate over ones by humans or other models

#HackerNews #LLMs #Resumes #AIgenerated #HiringAlgorithms #MachineLearning #JobSelection

AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights

As artificial intelligence (AI) tools become widely adopted, large language models (LLMs) are increasingly involved on both sides of decision-making processes, ranging from hiring to content moderation. This dual adoption raises a critical question: do LLMs systematically favor content that resembles their own outputs? Prior research in computer science has identified self-preference bias -- the tendency of LLMs to favor their own generated content -- but its real-world implications have not been empirically evaluated. We focus on the hiring context, where job applicants often rely on LLMs to refine resumes, while employers deploy them to screen those same resumes. Using a large-scale controlled resume correspondence experiment, we find that LLMs consistently prefer resumes generated by themselves over those written by humans or produced by alternative models, even when content quality is controlled. The bias against human-written resumes is particularly substantial, with self-preference bias ranging from 67% to 82% across major commercial and open-source models. To assess labor market impact, we simulate realistic hiring pipelines across 24 occupations. These simulations show that candidates using the same LLM as the evaluator are 23% to 60% more likely to be shortlisted than equally qualified applicants submitting human-written resumes, with the largest disadvantages observed in business-related fields such as sales and accounting. We further demonstrate that this bias can be reduced by more than 50% through simple interventions targeting LLMs' self-recognition capabilities. These findings highlight an emerging but previously overlooked risk in AI-assisted decision making and call for expanded frameworks of AI fairness that address not only demographic-based disparities, but also biases in AI-AI interactions.

arXiv.org

Miguel Afonso Caetano Jan 23, 2024

#USA #NYC #AI #Algorithms #HiringAlgorithms #AIBias: "A study into the effectiveness of a New York City law targeting bias in AI hiring algorithms has found the legislation is largely ineffective.

New York City Local Law 144 (LL144) was passed in 2021, came into effect as on January 1 last year and has been enforced as of July 2023. The law requires employers using automated employment decision tools (AEDTs) to audit them annually for race and gender bias, publish those results on their websites, and include notice in job postings that they use such software to make employment decisions.

The study from researchers at Cornell University, nonprofit reviews service Consumer Reports, and the nonprofit Data & Society Research Institute, is as yet to be published but was shared with The Register. It found that of 391 employers sampled, only 18 had published audit reports required under the law. Just 13 employers (11 of whom also published audit reports) included the necessary transparency notices."

https://www.theregister.com/2024/01/23/nyc_ai_hiring_law_ineffective/

Law designed to stop AI bias in hiring decisions is so ineffective it's slowing similar initiatives

New York’s LL144 rated too broad, but researchers hope others can learn from that mistake

The Register