Federal legislation may not always have a reputation for being super exciting stuff, but I think that it can be really interesting if you understand how some of the decisions that go into choosing the particular words that make it up shape the ways that legislation may be interpreted.

I want to share some of the things that make me excited about the approach taken in the Algorithmic Accountability Act of 2022 which might also inform other #techpolicy legislation to come, so… LET’S DIG IN! 🍽

Thing I’m excited about #1: Impact Assessment is an activity not an artifact 🏄

One thing you may notice in reading the bill is that it very rarely talks about impact assessment as a plural (“impact assessments”). This is because it treats impact assessment as a mass noun.

(Didn’t think you were gonna get a grammar lesson, did ya? 😜)

Mass nouns (also called noncount nouns) are words like diligence, management, information, feedback, hospitality, mail, poetry, software, training, or… bacon! 🥓✨

So, although one might sometimes talk about various softwares, trainings, feedbacks, or bacons, the typical use treats these more like continuous activities (or breakfast meats).

Similarly, when the Algorithmic Accountability Act talks about impact assessment, it doesn’t treat “assessments” as one-off events or as pieces of paperwork.

Rather, impact assessment is an ongoing series of activities, an integral part of responsible technology deployment.

I like to think of it like documentation of code.

When you’re developing software, there are sometimes discrete artifacts and resources that are produced (documents), but documentation is not just artifacts, it is a process. It is an ongoing activity throughout the lifecycle of software.

“Impact assessment” similarly refers to an ongoing activity that may involve a variety of artifacts and processes depending on the specific application of the term.

Big shout outs to Jingying Yang, Timnit Gebru, Meg Mitchell, Hanna Wallach, Jenn Wortman Vaughn, Deb Raji whose brilliant thinking on documentation for machine learning systems first exposed me to this concept through work like #ABOUTML
https://partnershiponai.org/workstream/about-ml/

So that’s thing #1: In the Algorithmic Accountability Act, impact assessment is a process, a set of actions, an ongoing activity that is integral to deploying automated decision systems to make critical decisions.

Thing I’m excited about #2: Focus on decisions, not data types 📊

A pretty big shift from the 2019 version of the bill (and most legislation in this space) is the move away from the definition of “high-risk” systems to the frame of “critical” decisions.

2019: https://www.congress.gov/bill/116th-congress/house-bill/2231/text

I know at first this may seem like such a nerdy (and some may say pedantic) point, but to me it’s important, and this is MY list of things I’m excited about, so… there!
😜

So here’s the thing about regulating “high-risk” systems, you kinda have to have an idea already about what is risky.

Most regulation for AI & automated decision systems (ADS) defines this by the:
1) severity of impact
2) number of people impacted
3) sensitivity of data involved

Going through the list, let's talk about why each of these things can be either difficult to use or even possibly antithetical to the typical goals of these types of legislation.

1) Severity of impact
For some regulation of automated decision systems (ADS), specifically regulation that seeks to provide recourse or to address harms of a certain degree, this can be useful. It does depend, however, on knowledge about the impacts of using a system.

In Algorithmic Accountability, the goal is to UNCOVER impacts from using automated systems, to identify (& address!) harms. Because all of the systems causing these bad impacts are not even known, that’s a less useful definition.

Privacy law is important and urgently needed, BUT privacy law and algorithmic accountability law are complimentary, not substitutes for one-another.

Not only does law and regulation for AI & ADS need to do different things, but sometimes the goals of privacy and algorithmic accountability are in tension!

This is why defining "high-risk" systems through data sensitivity is especially dangerous.

Regulating systems for making decisions based on the INPUTS to those systems rather than their outcomes creates perverse incentives to use less ~sensitive~ data, even if that is the data actually most pertinent information to the situation.

If a system is making critical decisions about a person's healthcare, it probably SHOULD be using sensitive health information! Using more benign data (through proxies or straight up irrelevant info) may not only be unhelpful, it may actually harm people.

So in Algorithmic Accountability, rather than defining the automated systems of interest by 1) impact, 2) number of people, or 3) data used, it uses the framing of "critical decisions.” Those are decisions relating to consumers’ access to or the cost, terms, or availability of education & vocational training, employment, essential utilities, family planning, financial services, healthcare, housing or lodging, or legal services)

Why is this useful?

Focusing the bill on a list of what are defined as "critical decisions" makes the targets of the bill more concrete. Concreteness helps to avoid self-referential definitions.

It also narrows the scope somewhat, which may important for potentially boring government-y reasons because, after all, this is an FTC bill.

Now, I don't have any inside knowledge on this, but I suspect this line of thinking was what prompted European legislators to add the Annex III to the EU AI Act, alluded to earlier.

I think I should expand upon the Federal Trade Commission (FTC) point here as well:
You may notice that the list of things that make up these "critical decisions" does exclude some things that many people might expect to see in AI/ADS laws (like some of what's in the EU AI Act).

This is related to that boring government jurisdictional stuff. Since the FTC is about consumer protection it doesn't cover government use like the criminal legal system, immigration, or govt benefits administration.

Reporting is an important element to accountability because it offers a level of transparency into processes and it introduces procedural check to ensure that impact assessment is, indeed, taking place. (This is different from some other approaches like audits, pre-approval, etc.)

I imagine this is one of the SPICIER elements of the bill, so let's discuss! 🌶✨

Before we get into why this is a Thing I'm Excited About, let's first talk about what many people want this bill to do (which is doesn't do), and then I'll tell you why I think that THIS is actually even better!
😈

⚠️ caricature incoming ⚠️
A lot of people want Algorithmic Accountability to be about catching bad actors red-handed. They want to expose and name-and-shame those who are allowing their automated systems to amplify and exacerbate harms to people.

This is righteous, and I empathize.

I also want there to be justice for those harmed, and I want there to be real consequences for causing harm that willful and feigned ignorance do not excuse. I do believe that this is a step in that direction, but this bill focuses on something slightly different.

It is less about helping the FTC catch wrongdoers (although there is that, and I’ll explain more) and more about making it easier and more clear on how to do the right thing.

One of the great challenges in addressing the impacts of automated decision systems is that there is not (yet!) an agreed upon definition of what "good" (or even "good enough") looks like.

We lack standards for evaluating decision systems' performance, fairness, etc. Worse still, it's all super contextual to the type of decision being made, the types of information/data available, etc.
😵‍💫

These standards may one day exist! But they don't yet. Algorithmic Accountability is about getting there.

And part of getting there, I believe, is facilitated through the three tiers of disclosure and reporting:
1) internal assessment
2) summary reports to the FTC
3) public info sharing FROM the FTC in the form of
- aggregate anonymous trend reports
- a searchable repository

Many folks may feel these reports should be made entirely public. I get where that's coming from, but here's why think this private reporting to the FTC is actually a kinda clever way to go about it...

For one, because we lack standards, it is premature to codify specific blanket requirements for which metrics for evaluating performance, for instance, all companies should use. As such, companies will likely choose whichever ones make them look "best” meaning people won’t put out damning info.

Okay, but here's the fun part! Because these reports are submitted privately to the FTC, companies are now in a position of information asymmetry. They do not know what OTHER companies are saying they did or how they performed on THEIR metrics. They may try to do the bare minimum, but they don’t actually know what the bare minimum is!

Gotta love it when collective action problems work on our side 😜

The FTC (+ some other agencies), however, get to see across the collection. And this is super useful! Not because companies are going to "tell on themselves," but because there are really interesting lessons to be learned from how different companies fulfill these requirements.

There is as much to be learned from what particular companies do say in their reports as what they don’t. The information asymmetry makes this more JUICY!

See, right now there's a dynamic where any company (or more like employee) that tries to really interrogate the impacts of these automated decision technologies gets called out for it.

It's a "tall poppy" situation. It's better to not know, to not try, than to find out the truth.

The companies that do the least don't make headlines. The automated decision systems that no one knows about don't feature in hashtags.

The current system rewards keeping your head down, not asking questions, & staying obscure. It often punishes those that try to ask, to measure, to identify and prevent harm. (Better to not test. Sound familiar?)

This private reporting dynamic shifts that calculus.

Now, companies aren't telling on themselves so much as they're telling on each other. The competition can reduce collusion pressures. There is an opportunity for a race to the top that doesn't exist in the current equilibrium.

Maybe you say "but still, there are things that The Public really does deserve to know!"

And it's true. Some things are really essential. Like knowing what decisions about you are being automated or knowing if there is a way to contest or correct one of these decisions.

And so... 🍰 Layer 3: information shared by the FTC to the public in the form of:
- aggregate anonymous trend reports
AND
- a searchable repository

The Algorithmic Accountability Act is a consumer protection bill (where consumer is defined as... any person. Turns out there's no official FTC definition of consumer 🤪)

Part of that protection comes from making key information available to the public in a place where individuals, but also awesome consumer-protection and advocacy organizations can access it.

Cool endorsing orgs: https://www.wyden.senate.gov/imo/media/doc/Support%20for%20the%20Algorithmic%20Accountability%20Act%20of%202022.pdf

This 3rd tier of disclosure consists of two types of information. One is an information-rich, qualitative report of the findings and learnings aggregated from the multitude of individual reports. This is where the FTC can highlight differences and patterns.

Personally, I'm really interested to learn about things like... do different critical decisions (health vs employment) gravitate toward different metrics for evaluating performance? What types of stakeholders are being consulted with? How?

The second half of tier three is the public repository. This has more limited information, but contains a record for every critical decision that has been reported and contains that key information we alluded to earlier.

The repository must "allow users to sort and search the repository by multiple characteristics (such as by covered entity, date reported, or category of critical decision) simultaneously," ensuring that it can be a powerful resource for both consumers+advocates and researchers.

There you have it, folks!

My big 3 Things I’m excited about in the Algorithmic Accountability Act of 2022:
#1: Impact Assessment is an activity not an artifact 🏄
#2: Focus on decisions, not data types 📊
#3: Three tiers of disclosure 🍰

There’s a lot there, but I hope this thread helps illustrate some of the clever ways that writing robust legal definitions about tech and red-teaming regulatory requirements can potentially produce better legislation for tech policy issues!