@reiver …and it's not just the sheer amount of storage; prudent expiration and cleanup (e.g., #Enigmatick clears cache data each night that's greater than 30 days old) helps to manage that. It's the application load of pulling up the same records tens of thousands of times within a few seconds when new content is created (i.e., the fan-out of initial delivery and the resulting requests back for content).

I use Varnish and that helps a lot with the latter case. But a #FediCDN could be built on Varnish (like Fastly is) and provide decentralization in that service.

Enigmatick

3/

[Fediverse CDN]

Once you have Fediverse Caching servers — you have the basis to create a Fediverse content-distribution-network (CDN).

A (Fediverse-native) Fediverse content-distribution-network (CDN) could bring a new level of robustness and scalability to the Fediverse — while still maintaining the properties of decentralization, federation, and localization.

#CDN #ContentDeliveryNetwork #ContentDistributionNetwork #FediCDN #FediDev #FediDevs #Fediverse

2/

[Fediverse CDN]

One way to increase the robustness of the Fediverse is — to introduce the concept of a Fediverse Caching server.

It (the Fediverse Cachine server) caches stuff from the Fediverse rather than the regular Fediverse servers (that house local users and local content).

...

#CDN #ContentDeliveryNetwork #ContentDistributionNetwork #FediCDN #FediDev #FediDevs #Fediverse

RE: https://mastodon.social/@reiver/114009413139340907

1/

[Fediverse CDN]

One thing that crashes Fediverse servers (maybe the most) is — caching.

And, in particular, their storage drivers filling up (due to caching), which crashes the server.

Fediverse servers cache profiles, posts, images, and other servers on the Fediverse.

...

#CDN #ContentDeliveryNetwork #ContentDistributionNetwork #FediCDN #FediDev #FediDevs #Fediverse

User profiles, posts, images, and other content is cached across the various servers on the Fediverse.

Conceptually, software could access any of that from any server that cached it (and not just the origin server).

You could have a type of Fediverse content-distribution-network (CDN) if you did.

#CDN #ContentDeliveryNetwork #ContentDistributionNetwork #DeSo #FediCDN #FediDev #FediDevs #Fediverse