Mastodawn

How are we going to pay for all this?

https://sh.itjust.works/post/39134

How are we going to pay for all this? - sh.itjust.works

I’m really enjoying lemmy. I think we’ve got some growing pains in UI/UX and we’re missing some key features (like community migration and actual redundancy). But how are we going to collectively pay for this? I saw an (unverified) post that Reddit received 400M dollars from ads last year. Lemmy isn’t going to be free. Can someone with actual server experience chime in with some back of the napkin math on how expensive it would be if everyone migrated from Reddit?

fratermus Jun 11, 2023

edit: misposted comment - see bizarre explanation below

westworld - lovely visuals
alias - excellent theme music
bojack horseman

michael Jun 11, 2023

What lol

fratermus Jun 11, 2023

I composed that comment in another thread, tabbed away, and when I tabbed back the pending comment was pasted into this thread that I wasn't even reading. Hit POST before I noticed it. 2nd time that's happened this morning.

What tv show intro do you never skip? - Lemmy

sharpiemarker Jun 11, 2023

You know you can delete comments right?

scharf_2x40 Jun 11, 2023

Lost lemming

fratermus Jun 11, 2023

Not lost, the interface loaded a comment I had written elsewhere into this thread, which I didn't even have open and wasn't reading. When I reloaded the page to try to stop the post it showed the original thread I intended to post to. I had to manually open this thread to see where it posted.

LootGoblin42 Jun 11, 2023

I think the idea is that many people can run lemmy servers so the load is split between everyone hosting them.

Krusty Jun 11, 2023

There already is a question similar to this. You can find lots of ideas there :)

zakk002 Jun 11, 2023

Unlike Reddit there will be many lemmy servers that exist, and I assume most will be supported by donations. A lot of servers have an Open Collective or Patreon option if you want to support them.

jgrim Jun 11, 2023

The cost will be spread out, and people can monetize how they see fit. I'm wondering if there will be additional benefits you can add to your instance for a charge that people might be willing to pay.

I'm considering offering an Element server and maybe email on mine with a shared username for each service. That's going to take time to setup, though.

We must prove that it's valued and let the monetization come later. I'm working on this in my spare time. Once I can grow, maybe I can put more effort into it. I think it'll be a lot of people like me for a while financing it out of pocket.

thestereobus Jun 11, 2023

Bank accounts and escrow and payment processors are complicated and require real companies with tax identifiers. Nobody is gonna set that up just to host an instance. IMO this is a perfect use case for crypto. Could be a monthly thing or micropayments. Maybe something a bit like the Basic Attention Token.

Seraph089 Jun 11, 2023

It's a perfect use case for Patreon, PayPal, and similar services that are easier to set up than any of the things you mentioned. And the stability of fiat currency is a benefit when we're looking at covering operating costs.

Crypto is fine as an option, but it doesn't seem like a great fit for the primary.

im stuff Jun 11, 2023

donations to my favorite instances, like wikipedia i hope :)

SubArcticTundra Jun 11, 2023

You can always have paid-access Lemmy servers

torknorggren Jun 11, 2023

Are ads forbidden?

salarua Jun 11, 2023

not forbidden, just difficult to make work. you'd have to create a fork of Lemmy with support for ads, and you'd have to convince people to come to your instance when there are so many ad-free instances out there

mrmanager Jun 11, 2023

People who hate ads.

So good luck.

LadyAutumn Jun 11, 2023

Lemmy is never going to handle an entire migration of reddit's whole user base. Most redditors use the official app, and the mainstream audience for the platform now represents the largest user group. They're not going to wholesale make the jump to Lemmy. That wouldn't even be possible without widespread coordination of resources. Each instance can only handle so many users, so new instances will have to continually be created to accommodate influx. Theres no profit incentive either, meaning whoever is running the instance server is purely doing it out of their passion for the platform. That doesn't scale linearly, there's only so many people out there with the resources to run a large instance.

Measuring the cost isn't possible. It depends on electricity and equipment costs which vary a lot. And the question doesn't make sense either.

Lemmington Jun 11, 2023

They're probably the sorts of people that would drive down the quality of content here, so no great loss anyway.

Anissem Jun 11, 2023

Donating $20 a month over here, hope it helps

Wander Jun 11, 2023

That's why I'm running my own server. Mastodon is much bigger than Lemmy and it does fine with community run servers.

Korgen Jun 11, 2023

Each instance funds itself. Some might try ads, but as of now, most are just funded by donations.

minorsecond Jun 11, 2023

I just signed up for the Lemmy.ml patreon. I wish they had a $5 per month option, but I can just not skip ordering doordash one extra time and help pay for this instance. I use the hell out of it so it's the least I can do.

Bumble Jun 11, 2023

It's certainly a challenge. I run three instances on the #fediverse. Two small ones and a larger one with 450 users. I have a donation page; initially people were enthusiastic and I covered costs. Now, I have regular monthly donators but not enough to cover costs so I am subsidising it. I took the decision when I launched that it could happen and it's my problem.

I think there will be many instances will fall away in the coming months due to costs. Especially if you are thousands of users and associated costs.

We need to come up with a new funding model, where people appreciate you get nothing for nothing. All the large corporates sell your data as advertising for revenue. The greater public do not appreciate they are selling their soul.

eekrano Jun 11, 2023

Is there an approximate specs per number of users guide to size a lemmy instance?

kinther Jun 12, 2023

I haven't seen one yet. Disk usage this morning on lemmy.world was reported at about 4GB over 11 days (probably low usage). The 100GB drive would probably fill up in 275 days or so if usage did not increment. If it's not redundant and dies, all that content is lost.

So storage will be a huge issue for lemmy unless I'm missing something.

nwithan8 Jun 12, 2023

100GB is practically nothing nowadays.

There are people (myself included, not to brag) running home servers with literally hundreds of terabytes of data. At that ~0.3 GB/day number, I alone could host 3,500 years worth of data. Get some of those r/DataHoarders and r/HomeLab guys on here and Lemmy would never run out of space.

gizmonicus Jun 12, 2023

Can content be stored somewhere like S3 instead of spinning disk? It would certainly be more robust and cheaper.

pre Jun 11, 2023

@psylancer If you got 100 million users so that it's costing 400 million dollars a year, then ideally you need one million servers with 100 users on each. They need to all pass around a hat between their 100 users to raise the 50 dollars in server costs a month.

bobaduk Jun 12, 2023

Seems like storage is the cost prohibitive part. Search is going to be difficult, too. I wonder whether the model there might be community supported shared services. A bunch of instances could jointly run an elasticsearch cluster or Algolia instance and charge for API access to cover costs.

CarbonIceDragon Jun 12, 2023

I wonder if the storage costs could be reduced, over the long term anyway, by having a setting to automatically delete anything older than a certain amount of time, configurable by the owners of a given instance. So an instance with ample funding or good hardware could disable it, and one that has enough drive space for about 3 years worth of posts could set that length of time for how long something lasts. Probably also would want a way for an instance admin to mark a posts as permanent and exempt from deletion. Would be a shame to lose that older content, but if reddit is any indication, anything people want to see again will likely get reposted anyway, and it would be preferable to losing an instance to rising storage costs. I imagine it might also make search a little better by having less to look through, though tbh I'm not sure how search algorithms work so I don't know how much that would help them if at all.

rektifier Jun 11, 2023

I think the biggest cost will be image/video storage. The text takes very little space in today's standards. The good thing is that symmetric fibre internet connections are becoming more common so it may be possible for members of the instance to contribute unused disk space to help with its image/video storage. This plus limiting the image/video sizes (and maybe forbidding video uploads altogether) will allow the instances to scale with user count.

PlasmaK Jun 12, 2023

Maybe coordinating with peertube or just using bittorrent to store video online would be good.

Pisck Jun 12, 2023

Sure, image/video is vastly more resource-intense, but it's not used for successful DDoS attacks and fiber connections don't stop them either. We need a better answer for horizontal scaling as servers currently aren't even closing registrations until they're already noticing performance issues and are vulnerable to relatively small increases in federated activity.

Salamander Jun 11, 2023

A small cloud server + a domain name costs less than a Netflix subscription. The developers have taken care to package lemmy in ways that are relatively straight forward to deploy, so a dedicated person with a small amount of experience can have an instance up and running in an evening. As long as a few percentage of users are willing to pay a netflix subscription to keep a server running, the financial burden would be spread.

Stumblinbear Jun 11, 2023

I think this underestimates how users will naturally gravitate towards more centralized instances, or they'll give up because the bigger instances are closed. Someone's gotta pay for it, and it's going to cost more than a Netflix subscription. Servers aren't cheap.

This also ignores that the system isn't horizontally scalable at all, so scaling up gets even more expensive

Salamander Jun 11, 2023

I think this underestimates how users will naturally gravitate towards more centralized instances, or they’ll give up because the bigger instances are closed.

(This is purely my personal opinion, of course!) In the scenario in which a few large instances dominate, the idea of the fediverse failed. One may estimate the likelyhood of success or failure given how they expect humans to behave, but in the end experiment beats theory. I think that for the fediverse to work a significant cultural shift has to occur, but I don't think that it is an impossible shift. I would like the fediverse to succeed, and so I choose to take part in the experiment.

This also ignores that the system isn’t horizontally scalable at all, so scaling up gets even more expensive

Yes, that might cause some serious issues. The project is still in an early-development phase, and I don't understand the technical aspects well enough yet to be able to identify whether there is obviously a fundamentally invincible barrier when it comes to scalability. My optimistic hope is that the developers are able to optimize horizontal scalability fast enough to meet the demand for scale. If it turns out to be impossible to scale, then only rich enough parties would be able to have viable instances, and that could be a reason for failure.

suspicious_dog Jun 11, 2023

What does ‘horizontally scalable’ mean here? I haven’t come across that before.

Salamander Jun 11, 2023

This is what I think, but if anyone understands it differently please correct me.

Vertical scalability refers to scaling within a single instance. More users join and they post more content, increasing the amount of disk space needed to hold that memory, network bandwidth to handle many users downloading comments and images at once, and processing power.

Horizontal scaling refers to the lemmyverse growing because of the addition of new instances. The problem in this form of scaling is due to the resources that an instance has to use due to its interactions with other instances. So, you may create a small instance without a lot of users, but the instance might still need a lot of resources if it attempts to retrieve a lot of information (posts, comments, user information, etc) from the other larger instances. For example, at some point a community in lemmy.ml might be so popular that subscribing to that community from a small instance would be too much of a burden on the smaller instance because of the amount of memory required to save the constant stream of new posts. The horizontal scaling is a problem when the lemmyverse becomes so large that a machine with only a small amount of resources is no longer able to be part of the lemmyverse because its memory gets filled up in a few hours or days.

honk Jun 12, 2023

I don't believe this is how it works though.

Let's say your tiny 3 person instance is connected to a big one. I believe it only pulls in content from the communities somebody from the small instance is subscribed too. Correct me if I'm wrong.

panoptic Jun 12, 2023

That's what they're saying.

Essentially - if someone from the small instance subscribes to a community that has a ton of data (huge post volume, images, whatever), the small instance needs to pull data over from the larger instance. At some point there may be communities that are so large small instances can't pull them in without tanking.

ShadowAether Jun 12, 2023

Could that be solved by caching? Can't the smaller instance avoid some duplication?

Jeremy [Iowa]Jun 12, 2023

You can summarize by thinking of vertical scaling as "make machine bigger / more powerful" with horizontal scaling as "make more machines".

bobaduk Jun 12, 2023

Some things can go faster if you add more workers, some things can only go faster if you make the workers bigger or faster .

If you're tidying a garden you can get it all done more quickly, and tackle bigger gardens, by getting your friends to help. That's horizontal scaling.

If you need to get a parcel from your house to Burkina Faso the only way to do that more quickly is to use a bigger, faster machine. That's vertical scaling.

The way Lemmy is designed right now (says the op, I don't know the detail) you can only support more users by making the server bigger and more expensive, not by using lots of smaller servers.

monobot Jun 12, 2023

I think that having few big instances is not failing, it is natural for social network (where lemmy is some representation of one) to be scale-free network, which has big hubs and buch of smaller nodes connected.

Most people would go to general instances, but artists will probably go to some art focused instabce, developer to proggraming.dev... But we will have bih hubs, there is no way around it.

Salamander Jun 12, 2023

Yes, you are right, I should have formulated that better.

I would expect that there would be a few big instances. What I should have said is having only a few big instances and no small instances would be a failure. It would be totally acceptable to have a few big instances and lots of smaller instances that can still interact with the fediverse. The failure would be if you have something like 20 very big instances that only interact with each other and that are inaccessible to the small instances - either because they close their federation, or because it is too resource-intensive for small instances to interact with them. In this case you end up with a centralized system again, not better and potentially worse than something like Reddit. As long as someone can spin-up their own instance if they want to and be part of the larger ecosystem, it would be a success.

manitcor Jun 12, 2023

this works on the same principals as fidonet, UseNet, email, etc. These protocols are more like fundamental services. The idea behind these was that instead of running a bunch if proprietary garbage you would run things that support A LOT of standard protocols. Why? Because NO ONE should be allowed to own our communications but ourselves.

The corps did not build these networks, we did. Software will improve over time, OSS shows the way.

jon Jun 12, 2023

The distributed nature of Lemmy should make things more manageable. Personally, I'm running an instance on a dedicated machine I already pay for, so it's not costing me anything unless storage skyrockets. Many other instance hosts are also hobbyists that don't mind covering the costs, and may take some form of donations locally on their sidebars.

There probably should be a built-in feature for instance admins to enable a local donation button to contribute to their costs, though. While Lemmy is fairly resource-efficient, larger instances are eventually going to require pretty beefy VMs to keep up with the traffic, image uploads, etc. I could see some instances randomly vanishing when their owners can't/don't keep up with their bills (which would force users over to other instances), but ideally if any instance owners can't afford to cover it, they hand control over to another community member to pick it up.

moreeni Jun 12, 2023

In general, Fedi admins simply close registrations when they can't keep up with an influx of new users, and point people to other, smaller instances

Pisck Jun 12, 2023

Closing registrations is all well and good, but can't activity / load still skyrocket as users from federated instances subscribe to, comment on, and post to their communities?

moreeni Jun 12, 2023

It will, but Lemmy is fairly light, so I doubt it will be critical. Fedi is about community, so in case of load problems people will crowd fund existing servers and host new ones

Tekhne Jun 12, 2023

I was thinking about setting up a Pi to host a small/personal Lemmy instance, do you think that's a reasonable plan? I have no clue how resource intensive Lemmy is. Was thinking it could be nice to store my and friends' personal data on our own server instead of some random remote server (to some extent obvi).

jon Jun 12, 2023

Yes it should run perfectly fine on a Pi, at least for a small instance. You will need to get ports forwarded or setup a reverse proxy if hosting at home, you will need to generate a valid SSL cert (i.e. Letsencrypt) to be able to connect to the federation

JasSmith Jun 12, 2023

Can I ask about the server load? I have a Plex server sitting around doing mostly nothing. I don't want to compromise that experience, but I've been thinking about starting an instance.

zikk_transport2 Jun 12, 2023

IMO it sounds like that some AI corporation should host their own instance(s). They only pay for server and maintenance costs, while community does the rest and they have their data.

Would be best of both worlds, isn't it? Once they become greedy, we are f*cked again, just like Reddit did...

Ziggurat Jun 12, 2023

Don't forget to make a donation to your instance if you love it. For most of us it's a bit early but I give my 10 EUR per year to my Mastodon admin. Also, if you can choose instance ran by a non profit rather than a person as it ease the whole donation mechanism and give you the right to check where your membership due go.

forgotaboutlaye Jun 12, 2023

Is there a list of non-profits that run an instance?

RiderExMachina Jun 12, 2023

So far the only one I know of is the Mastodon server Mozilla.social.

actually-a-cat Jun 12, 2023

Reddit has over 2,000 employees most of whom are doing bullshit nobody using the site actually needs or wants, it's possible to run a lot leaner than that. Like Reddit itself used to, before they started burning hundreds of millions trying to compete with every other social media site at once instead of being Reddit

RickyRigatoni Jun 12, 2023

bullshit nobody using the site actually needs or wants

Like being CEO.