In which Santa's elves learn an important lesson about machine learning bias. 🎅🧵

'Twas the weeks before Christmas
And Santa was busy
His list was so long
It was making him dizzy

There are too many children
List checking can’t scale
If I do this by hand
I will certainly fail!

But one of his elves
Had a brilliant idea
What if we automate
Christmas this year!

[continued :) ]

Every child there’s been
From Aaron to Zeta
We can build an AI
Because we have the data!

Now, Santa is old
Didn’t quite understand
But he trusts his elves
So he okayed the plan

So the elves cleaned the data
They were very precise
To determine the patterns
In the naughty and nice

Gender and parents
And address and race
Grades and faves
Really, every trace

Billions of children
Made into numbers
AI makes predictions
While each child slumbers

[2/5]

The goal of the model
(Just to be clear)
Given these patterns
Who’s naughty this year?

They held back a sample
So that they could test
And the model? Not bad.
Ninety percent, at their best.

So they brought it to Santa
And said, never fear!
Predictive machine learning
Will save Christmas this year

Santa pulled out his list
For his own kind of test
Hours later he went
From impressed to distressed

[3/5]

We can’t use this, he said
You must realize the stakes
There also are patterns
In errors it makes

Like take little Aaron
Do you think it’s fair
That in numbers he’s like
Naughty kids over there?

We’re also assuming
Our own perfect past
Have we made mistakes?
We at least have to ask

Cause look at this insight
The data supplies us
Some elves on these shelves
Might have unconscious bias!

And you all made choices
Which features to add
Are you sure this was fair?
The elves looked quite sad

[4/5]

The elves had to admit
They’d bought into the hype
But those errors did matter
And Santa was right

For now, Santa said
I’ll go back to my list
Some things need a human
To ensure they’re not missed

I do think tech could help us
You didn’t mean to abuse it
But with any great power
You must be careful how you use it

So next Santa hired
An AI ethics team
We moved too fast, he said
We don’t want to break things.

[5/5 THE END]

... ok I'm pretty proud of this haha. I'm going to put it somewhere it can be more easily shared, but in the meantime I wanted Mastodon to get first look. :D
OK clearly "'Twas the AI Bias Before Christmas" needed a dramatic reading. Waiting for the phone call about the animated series! 🎅 https://www.tiktok.com/@professorcasey/video/7178174993212165418
Until Netflix buys the rights or I get a book deal, here's an easily shareable version. 🤣 https://cfiesler.medium.com/twas-the-ai-bias-before-christmas-3255cd3acf77
’Twas the AI bias before Christmas - Casey Fiesler - Medium

There always are kids Who are naughty or nice Every year for millennia You’ve checked that list twice First the elves cleaned the data They were very precise To determine the patterns In the naughty…

Medium
@cfiesler very well done, getting your break off to a good start!
@cfiesler this is AWESOME and I was just about to ask if you would do a TikTok... But you beat me to it. Gotta go watch now.

@cfiesler

Can't beat video! Great job!

And to all, Lesson of the day 👉 "with any great power, be careful how u use it!"🔥🔥🔥🔥

@cfiesler woohoo! Nicely done! Ho ho ho
@cfiesler I enjoyed seeing it come past my TT timeline after I’d seen it posted here this morning. Pretty awesome :)

@cfiesler

Fun. Can't share n TikTok but will "here". Fun times in both #socialmedia and #psychology.

Have a Happy. And regards to the elves on shelves. (Seen this one?speaking of #media and naughty (no tag). Happy Ho ho's &/or lights.

@cfiesler I enjoyed reading it 👏🏿👏🏿
@cfiesler I enjoyed it very much!

@cfiesler amazing!

"Some elves on these shelves
Might have unconscious bias!" 😂

@jtonline I was proud of finding a rhyme for bias lol
@cfiesler @jtonline "Liars" almost does it (if you pronounce it the UK way) - and it's fitting!
@cfiesler But, but, but why didn't they use a blockchain, huh?
@cfiesler that was great thank you. 😅
@cfiesler Brilliant. I wrote a short family tree to send to my kids with similar style. Humour helps to digest and appreciate things in our rather serious world.
@cfiesler really great. Fun and nails an important point.
@cfiesler love it! Thanks for not losing your head, Santa!
@cfiesler Brilliant! 🎅
@jeffjarvis thank you! a good use of my first morning of winter break haha
@cfiesler Thanks for the Christmas parable.
@cfiesler it's really good! Ty
@schock thaaaank you, I especially value your opinion on this 🥰

@cfiesler "So next Santa hired
An AI ethics team"

#proof Santa and his operations are headquartered in the North Pole and not Silicon Valley

@cfiesler wish IRL folks would handle issues with AI as well as Santa and his elves did.

@cfiesler Fun poem, but I might suggest a slightly modified ending..

"So next Santa hired
An AI ethics team from everywhere
made from people with backgrounds
and races that were sure to be fair. "

@AstroTman I certainly think that messages about diversity in tech are great, but (a) there's nothing in this poem to suggest that all the elves are of similar backgrounds/races; and (b) I think that there are a lot of excellent reasons for diverse hiring that aren't "please come fix all our problems for us."

@cfiesler

The past few years literally every week there is an example where AI has made race and gender biases that affect people. Just this week with Lensa!

People are noticing and so is Santa:

"We’re also assuming
Our own perfect past
Have we made mistakes?
We at least have to ask"

It's your poem, it's cute and you should be proud of it. No need to take my input at all. It's just random input after reading your poem!

@AstroTman Yep this is my research area! Actually one of the things that inspired this was someone commenting on my TikTok about Lensa telling me that I'm wrong and AI bias doesn't exist. :)

As I said, I'm totally on board with messages about diverse hiring! I'm just uncomfortable with the implication that the burden should be on the shoulders of marginalized folks to fix all our problems. I think these are two related but separate problems.

@cfiesler this was epic! 👌🏽😆🤓 made me smile.. and really drives the point home on #responsibleAI & #AIethics