#ContentModeration is fundamentally about making social media work better, but there are two other considerations that determine how social media *fails*: #EndToEnd (#E2E), and #FreedomOfExit. These are much neglected, and that's a pity, because how a system fails is every bit as important as how it works.

If you'd like an essay-formatted version of this thread to read or share, here's a link to it on pluralistic.net, my surveillance-free/ad-free, tracker-free blog:

https://pluralistic.net/2022/12/19/better-failure/#let-my-tweeters-go

1/

Pluralistic: Better failure for social media (19 Dec 2022) – Pluralistic: Daily links from Cory Doctorow

Of course, commercial social media sites don't want to be *good*, they want to be *profitable*. The unique dynamics of social media allow the companies to uncouple quality from profit, and more's the pity.

Social media grows thanks to #NetworkEffects - you join Twitter to hang out with the people who are there, and then other people join to hang out with you. The more users Twitter accumulates, the more users it *can* accumulate.

2/

But social media sites stay big thanks to high #SwitchingCosts: the more you have to give up to leave a social media site, the harder it is to go:

https://www.eff.org/deeplinks/2021/08/facebooks-secret-war-switching-costs

Nature bequeaths some in-built switching costs on social media, primarily the #CoordinationProblem of reaching consensus on where you and the people in your community should go next.

3/

Facebook’s Secret War on Switching Costs

Update, October 1, 2021: The original version of this essay incorrectly stated that Metcalfe's Law dictated that the number of connections in a network doubled with each new user; that has been corrected, below.When the FTC filed its amended antitrust complaint against Facebook in mid-August, we...

Electronic Frontier Foundation

The more friends you share a social media platform with, the higher these costs are. If you've ever tried to get ten friends to agree on where to go for dinner, you know how this works. Now imagine trying to get *all* your friends to agree on where to go for dinner, for the rest of their lives!

But these costs aren't insurmountable. Network effects, after all, are a double-edged sword.

4/

Some users are above-average draws for others, and if a critical mass of these important nodes in the network map depart for a new service - like, say, #Mastodon - that service becomes the presumptive successor to the existing giants.

When that happens - when Mastodon becomes "the place we'll all go when Twitter finally becomes unbearable" - the downsides of network effects kick in and the double-edged sword begins to carve away at a service's user-base.

5/

It's one thing to argue about which restaurant we should go to tonight, it's another to ask whether we should join our friends at the new restaurant where they're already eating.

Social media sites who want to keep their users' business walk a fine line: they can simply treat those users well, showing them the things they ask to see, not spying on them, paying to police their service to reduce harassment, etc.

6/

But these are costly choices: if you show users the things they *ask* to see, you can't charge businesses to show them things they *don't* want to see. If you don't spy on users, you can't sell targeting services to people who want to force them to look at things they're uninterested in.

7/

Every moderator you pay to reduce harassment draws a salary at the expense of your shareholders, and every catastrophe that moderator prevents is a catastrophe you can't turn into monetizable attention as gawking users flock to it.

So social media sites are always trying to optimize their mistreatment of users, mistreating them (and thus profiting from them) right up to the point where they are ready to switch, but without actually pushing them over the edge.

8/

One way to keep dissatisfied users from leaving is by extracting a penalty from them for their disloyalty. You can lock in their data, their social relationships, or, if they're "creators" (and disproportionately likely to be key network nodes whose defection to a rival triggers mass departures from their fans), you can take their audiences hostage.

The dominant social media firms all practice a low-grade, tacit form of hostage-taking.

9/

Facebook downranks content that links to other sites on the internet. Instagram prohibits links in posts, limiting creators to "#LinksInBio." Tiktok doesn't even allow links. All of this serves as a brake on high-follower users who seek to migrate their audiences to better platforms.

10/

But these strategies are unstable. When a platform becomes worse for users (say, because it mandates nonconsensual surveillance and ramps up advertising), they may actively seek out other places on which to follow each other, and the creators they enjoy. When a rival platform emerges as the presumptive successor to an incumbent, users no longer face the friction of knowing which rival they should resettle to.

11/

When platforms' #enshittification strategies overshoot, users flee in droves, and then it's time for the desperate platform managers to abandon the pretense of providing a #PublicSquare. Yesterday, #ElonMusk's #Twitter rolled out a policy prohibiting users from posting links to rival platforms:

https://web.archive.org/web/20221218173806/https://help.twitter.com/en/rules-and-policies/social-platforms-policy

This policy was explicitly aimed at preventing users from telling each other where they could be found after they leave Twitter:

https://web.archive.org/web/20221219015355/https://twitter.com/TwitterSupport/status/1604531261791522817

12/

Promotion of Alternative Social Platforms Policy | Twitter Help

This, in turn, was a response to many users posting regular messages explaining why they were leaving Twitter and how they could be found on other platforms. In particular, Twitter management was concerned with departures by high-follower users like @taylorlorenz, who was *retroactively* punished for violating the policy, though it didn't exist when she violated it:

https://deadline.com/2022/12/washington-post-journalist-taylor-lorenz-suspended-twitter-1235202034/

13/

Washington Post Journalist Taylor Lorenz Suspended From Twitter

Controversial Washington Post reporter Taylor Lorenz says she has been suspended from Twitter upon order of its owner, Elon Musk. Lorenz made the claim in a Substack post, adding that she did not a…

Deadline

As Elon Musk wrote last spring: "The acid test for two competing socioeconomic systems is which side needs to build a wall to keep people from escaping? That's the bad one!"

https://twitter.com/elonmusk/status/1533616384747442176

This isn't particularly insightful. It's obvious that any system that requires high walls and punishments to stay in business isn't serving its users, whose presence is attributable to coercion, not fulfillment.

14/

Elon Musk on Twitter

“The acid test for any two competing socioeconomic systems is which side needs to build a wall to keep people from escaping? That’s the bad one!”

Twitter

@pluralistic

It was not any close to an original thought either.