Josh BraunI've seen plenty of discussions in recent days over the network structure of Mastodon and whether it can remain decentralized as large commercial players begin taking an interest in the fediverse. I am neither an engineer nor an economist, but I do research the way media and information get distributed and I have some thoughts on this. A thread…
Matthew Hindman, in his book "The Internet Trap" <http://assets.press.princeton.edu/chapters/s13236.pdf>, notes that most research on the internet has focused on its supposedly decentralized nature, leaving us with little language to really grapple with the concentrated, oligopolistic state of today's online economy, where the vast majority of attention and revenue accrue to a tiny number of companies. 1/
This is a really nice observation. While it's true that the research on concentration in new media is laggy, I'd suggest it's useful here to look to the political economy and material realities of previous communication systems. In particular, there are a number of important dynamics that have historically tended to shape the organization of telecommunication networks that have interesting implications for Mastodon. 2/
To start with, consider the physical infrastructure of older point-to-point communication networks like the telegraph and telephone. Economies of scale for these networks work in an entirely different fashion than in industries, like retail or manufacturing, that come most readily to mind when considering how markets work. 3/
To wit, if you run a traditional business where you make a product—ice cream, say—economies of scale work in your favor. As you produce more ice cream, you can drive down the production cost per unit by negotiating better rates with your suppliers, who'll charge you less for milk and sugar if they know you're going to buy in bulk. 4/
TheresaIf you run a telephone network, on the other hand, your costs go up faster as you sign up new subscribers. That's because the infrastructure necessary to connect all these people scales quadratically. To see what I mean, take a look at these stills from "The Far Sound," and old film about the Bell system: 5/
Edit: Fixed wrong word choice. Thanks to those who noticed.
That chart may look familiar, because it's largely identical to the one people commonly use to describe "Metcalfe's Law" (https://en.wikipedia.org/wiki/Metcalfe%27s_law). The difference is Metcalfe's Law is about the social benefits of a communication's network—how it's utility to end users increases as it scales. What I'm saying here—and what network operators have long realized—is that the *costs* of operating the network infrastructure also scale quadratically as it grows. 6/
(Word fixed.)
The Mastodon network doesn't rely on laying cable (that's someone else's scaling-infrastructure problem), but all the point-to-point connections between users in the network do create analogous scaling issues in terms of data storage, which piles up as those links are established—each connection a conduit for a bunch of new posts that have to be stored somewhere. 7/
You see this every time a novice admin posts with exclamation points about how quickly their new instance's hard drive is filling. And they're providing the service for free, so they're not recouping *any* costs apart from possible crowdfunding donations. 8/
Another issue has to do with rights of way. In the physical world, the government is the one with the power to allow you to trench up streets for your cabling or use a particular radio frequency, so they have lots of leverage over you. You can't move your Chicago telephone exchange to Philly, either, so historically governments driven hard bargains (not these days for reasons I'll touch on). Unsurprisingly, then, graft and logrolling have long been rampant in the telecom business. 9/
There's no government to negotiate with in the Mastodon network, of course. Servers connect with one another more or less by default to deliver messages. But it's also quite simple for one server to silence or defederate with another over offenses, heinous behavior, or even simple policy differences. That means some level of negotiation is often happening between admins to get servers un-suspended. 10/
The largest servers may take on the clout of city councils if smaller ones want access to their users, giving them the power to set the terms for rights of way. That said, the opposite dynamic seems also to be true on some sites, with admins automatically refusing to federate with instances over a certain size on account of the attendant moderation difficulties. 11/
Richard John <https://www.hup.harvard.edu/catalog.php?isbn=9780674088139> highlights how the growth costs of networks have often led to great incumbency advantages. With the telegraph system, for example, lots of startups died after creating small regional networks because they didn't understand the unique economies of scale going in. That enabled the giant monopoly of the day, Western Union, to grow even more mammoth in size by buying out these smaller networks and incorporating them into its huge one. 12/
Network Nation — Richard R. John
Network Nation places the history of telecommunications within the broader context of American politics, business, and discourse. This engrossing and provocative book persuades us of the critical role of political economy in the development of new technologies and their implementation.
Early on, well-intentioned regulators who aimed at breaking up big monopolies like Western Union didn't understand the weird economies of scale they were dealing with either. They actually created incentive programs that led to the creation of many of the ill-fated startups that Western Union later gobbled up to expand its network. After that, and some similar experiences with regulating the telephone network, they got trigger shy and began to treat telecom companies as natural monopolies. 13/
Similarly, it remains to be seen whether the flood of new instances starting right now on Mastodon will scale up smoothly, with users kicking in the necessary funding, or whether they'll be overrun by costs their creators didn't anticipate, leaving domains and/or user bases to be subsumed by (commercial?) instances with deeper pockets. 14/
Anyhow, all of this is to not to say that point-to-point information networks necessarily end up as monopolies, but that there are definitely economic forces that push in that direction that have to be considered in designing networks, crafting regulations, and — when commerce comes into play — structuring markets. Even email, everyone's favorite example of decentralization, is now facilitated by a small cartel of rich commercial providers. 15/
The problem, of course — the one everyone knows — is that once providers become big enough, they turn the tables on regulators and users. The Chicago city council may once have had the telephone exchange by the throat, but now massive telecomm and tech companies are dominant forces in federal government and strangle their users through lock-in, as @pluralistic details brilliantly. <https://web.archive.org/web/20210622050005/https://thereboot.com/unfair-use-anti-interoperability-and-our-dwindling-digital-freedom/> 16/
The future may well be a bunch of Mastodon instances managed by a small number of hosting providers. Or a commercial instance with so much of the user base that the majority of instances treat its federation requirements as de facto policy. The downsides of concentrated, commercial social media are well known. @ubiquity75's work provides a master class here. <https://www.behindthescreen-book.com/> 17/
If folks want to prevent these outcomes, for example to preserve a system where everyone logs on through small communities where they know the admin and moderators personally, the governing principles need to be put in place now. The technology won't do it by itself, and in fact it may create conditions that favor the opposite outcome. /fin
Sirpushalot@josh What would be an example governing principle?
ShantyMinister 
@sirpushalot @josh The orig engineer for Twitter has thoughts on this.
But it needs to include more about moderation, definitions around not tolerating hate, regulations around instances that grow too big, and input from a much more diverse cross section of people (that aren’t white, male). #TwitterMigration
A cool crab wearing shades@josh Thanks for a very thoughtful thread. I think the likely ideal outcome is going to be a mix of larger instances for specific communities and industries, for example the Animation Guild could start an instance for members as a benefit for paying dues. Twitter was an incredible tool for networking in animation and a replacement with real moderation would be a valuable service in that industry.
A mix of that and smaller social instances of hobbyists, fans, and general users.
wb x64@neckspike @josh the history of blogs, forums and mailing lists might be useful here. Sure there's a few big names in those tech spaces but the fact remains that you can spin up a web+email bulletin board (now with mobile apps!) without asking anyone's permission, and even though some people will say "why not use Facebook?" just as many will say "I'm not using Facebook."
And yes by the numbers Google and Microsoft basically own email, but I'm running my own. It's doable, and interoperable.
@neckspike @josh and in fact the enduring capability of email to remain interoperable despite monopoly this whole time speaks to methods for ensuring the same for ActivityPub: keep per-server admin requirements light but tools powerful, make components swappable (I host with a relatively smaller host), put pressure and blame on poorly behaved admins or protocol saboteurs (people get irritated when they can't get good stuff, or get too much bad stuff,) have a strong culture around the protocol.
@neckspike @josh *yes* "the fediverse" will expand 1000fold if Tumblr implements AP. But to me that's closer to AOL Keywords coming on the scene than anything. We just say "sorry friend if you wanna talk to us you gotta use something better" and it carries the stink of lameness and the cool kids sign up for "net cat 420 at leet dot biz" like the rest of us
Dr. Vibha Pinglé@josh i think users who can logon through small communities are doing this. Musk’s deranged shenanigans are leading institutions to create these communities.
@josh how relevant is this point? https://on.ft.com/3V1NIHL
Ben ✌️@josh This is a brilliant analysis.
I'm frustrated to an extent by governance in the software. There are a small group of users who prescribe what is and isn't a good fit for the culture of this massively expanding platform.
If monopolisation becomes seen as inevitable, there might be hope through partnership companies, rather than for-profit. e.g. JLP/Waitrose (for those in the U.K.). While this would also share problems re de-facto policy, it may be preferable.
Reef Whooligan@josh I also wonder if the nature of discovery will reinforce mega-instances. At an instance level each user is a crawler pulling "good" content into the local and federated feeds of their instance so the more users an instance has the more thoroughly and deeply they are searching the fediverse. A new user can take advantage of that by joining a large instance vs joining a small instance where that discovery is left as an exercise for them.
@reef @josh I opened an issue re searching and it's got more downvotes and upvotes (only two tbf). https://github.com/mastodon/mastodon/issues/21627
Just Russ@TracyRussellBlack @reef @josh tbh, I disagree it's helpful. The decision to prohibit content-disocovery was based on a hunch it might be abused, there's no actual data here.
The argument against allowing content-discovery searches isn't strong, the only references for it are to conversations people have had about it before. Even in #594 there was admission that the decision wasn't based on more than a 'I don't want it on my server'.
Matthew Malthouse@uc @TracyRussellBlack @reef @josh I've been on another social platform where strenuous efforts have been made to resist it and as far as I can lee the lack only discourages users left to the arbitrary choice of admins deciding which hashtags are "valuable".
Also search is either complex or resource hungry. Often both. Giving sysadmins other reasons to not want it. Federated systems only make that more difficult.
@TracyRussellBlack @reef @josh There's no evidence to show only allowing content-discovery searching via external tools or google reduces the risk of harm to vulnerable groups.
Content posted publicly is already freely searchable, and I argue mastodon gives users a false sense of security by making it feel hard to discover
tolortslubor
Chickenhat@reef sorry to be just an idiot responding out of nowhere, but I remember when I opened my original account at the first Twitter exodus a few years ago when the fediverse timeline was just everyone tooting TOOT! , I joined mastodon.social and at the time they told us to spend abt a week there and use it to find other instances where we shared interests. I just stayed there until I forgot my password and was locked out. Id prolly still be there. Am I killing us?
Dr. Juande Santander-Vela
Alexander Ruthol@josh
"everyone logs on through small communities where they know the admin and moderators personally"
Is that either feasible or desirable? Is that kind of BBS/forum style network what Mastodon users want? Or do they want the reach and depth of a Twitter v2?
But yes, Mastodon nodes obviously needs some way to recoup costs. Ads would be an obvious choice.
@Je5usaurus_rex @josh Some instances would rather be small for a dedicated community, and they're no longer accepting new users. The instances that can meet the demand of people looking for access will get those users, and once they do, many are captured by convenience inertia.
Lnklnx 
@Je5usaurus_rex @josh "Ads would be an obvious choice."
Hahahahahaha... No.
Albert Cardona@josh In addition to governing principles there are basic principles of graph organisation that can be brought into bear to tame the curse of all-to-all connections among federated instances, such as designing for small-world networks where a subset of the federated instances play the role of hubs that mediate communications, and data cacheing, for a select set of other instances. The internet itself is organised like this.
#SmallWorldNetwork #GraphTheory #mastodon #fediverse
#SmallWorldNetwork #GraphTheory #mastodon #fediverse
Peace Covey@josh really helpful . Thank you.
Guido Stevens HAS MOVED@josh Thanks for this insightful thread on scaling Mastodon.
The federated nature of the Fediverse may trigger "bell curve" intuitions that are dangerously wrong in a space that follows geometric, power law scaling and concentration dynamics.
desertrat@josh this is an excellent point. we’ve been experimenting with governance in DAO communities for a while now. Its messy, but there possible solutions to be explored from these decentralized communities.
Lee Fife@josh This seems like the crux of the problem. And we're likely to have multiple sets of governing prinicples that emerge along the way.
Can the fediverse create meaningful groundrules?
"Please be excellent to each other" isn't actionable. "We subscribe to this moderation guild" is.
Hans-Cees@josh would be nice to do a simulation what the optimal size of a #mastodon instance is given that each user follows X others and so on.
Nice #complexity and #datascience question.
BTW I don't think many instances will be run by a few and a monopoly will result at all.
Since the plumbing is already in place (tcp/ip and connections).
Nice #complexity and #datascience question.
BTW I don't think many instances will be run by a few and a monopoly will result at all.
Since the plumbing is already in place (tcp/ip and connections).
@josh I'm trying to think of a way that the economies of scale might be introduced for their resiliancy... or if they can be used without even more monopolization. It's a good question!
Even though the model of net currently doesn't use scalable resources , as you eloquently described.
Bee organization models? That may be a way to think about it at this juncture. For example; There are queens and drones and workers in honeybees vrs monk-like miner bees, vrs warlike paper wasps, vrs small family bumble bee cells....worth chewing on a bit.
agnesbookbinder@josh It seems one way to combat the increasing costs is to donate. That won’t take care of all of it, but it will address the increasing cost to providers.
Karen Carr@agnesbookbinder @josh It seems to me that much of this could be resolved by all the (smaller) servers resolving to defederate from any server over a certain size, as you suggest. The big servers might then grow, but they wouldn't be pushing the others around.
@josh
As Josh noted, Mastodon is scaling fast, overwhelming servers and mods. Entities with funds have an opening to step in, meet demand, and acquire a large user base they can exploit, with the potential for Mastodon to be dominated the way the email market is.
Those who care will educate and lobby new Mastodon users, crowdsource funds and volunteers, and support the individuals and orgs working to make sure the monopolization doesn't happen.
PhoenixDogLover@josh Thanks, Josh. Thought-provoking. I'm an admin on a small, new Mastodon instance, and also a recently retired professional with experience in IT and corporate governance. The Mastodon world seems like a blank slate; or else I'm just not tapped into the governance community. I'm looking.
Kevin McCurley@josh the dominant problem with mastodon is discovery. There is no centralised search and addresses are as opaque as email addresses. Decentralisation makes it much harder to do good search.
Thom Hartmann@josh This was truly useful information, as well as fascinating. Thank you!
@ThomHartmann Much appreciated! Thanks for reading.
Ramsi Woodcock@josh The governing principles need to be put in place. And a funding model needs to be identified. If the funding isn't there, and big commercial communities are governed out, then Mastodon won't scale.
Nick Couldry@josh very important and thoughtful intervention thanks!
mistershins@josh thank you for the interesting read. Seems like in the short term that communities should be checking in with their instance admins and providing support to make sure they are not overwhelmed already.
blaine@josh awesome thread. 🙏 is an excellent articulation of some core issues I tried to address in https://slocanstatement.org - would be keen to hear your thoughts.
I've had a lot of feedback (most importantly around the fact that I didn't explicitly call out moderation/safety) and will be iterating it, but glad to see the conversation continuing to evolve everywhere! Ultimately this is something that the whole fediverse needs to sort out together, not just us random white dudes. 😅 @ntnsndr
Nathan Schneider@blaine @josh Agreed—some important insights here. And they raise questions about what should and shouldn't be centralized.
E.g., a co-op of servers might centralize storage & version updates to reduce duplication, while still enabling individual servers to manage moderation decisions or other matters of local preference.
@blaine @josh And in terms of the nudges of the technology, whenever you rely on servers, you have a design nudge toward what I call "implicit feudalism"—a governance default based on unchecked administrator power. ntnsndr.in/ImplicifFeudalism
As we invest more in this space, we should be asking hard questions about what should really be decentralized, and how, and what could be organized better with "accountable centralization." ntnsndr.in/Decentral
There's some tech stuff where I think I know what I'm doing, but the "hey folks, we need to sort out the policy side" I'm trying to share as much as I can right now because I know that most people affected by it (esp BIPOC) don't feel safe here. I think step one is getting the wider fediverse community to agree that "we need to have these conversations, with a more diverse set of participants"
Hopefully soon. 😅
David Gerard@josh email is an excellent example.
* postfix: trivial to set up!
* having your mail accepted by GMail, Microsoft and Yahoo: complex technical invocations, then several weeks supplicating to unknowable wizards to deign to let you talk to them