rexxit

Wouldn't the fediverse work better if it was like a drive array rather than independent communities on independent servers?

https://lemmy.world/post/1033231

Wouldn't the fediverse work better if it was like a drive array rather than independent communities on independent servers? - Lemmy.world

I get the impression that we’re headed for the same issues that pop up when we put all our eggs in one basket with Reddit/FB/whatever. People flock to the largest instance, and someday that instance could go down due to cost or the host losing interest. I’m wondering whether it would be technically achievable to have servers/instances and federation where the communities are essentially mirrored or have broadly distributed existence - maybe even with user storage a la torrents. If there’s a large [email protected] community and a small [email protected] community, all of the discussion, images, contributions to lemmy.here die if the server goes down for good. Yes, the users can relocate to lemmmy.there - even under the same community name - but it’s not the same as having full continuity of a completely mirrored community. I realize this concept has technical hurdles and would involve a reimagining of how the fediverse works, but I worry we’re just setting up for another blowup at some TBD date when individual sysadmins decide they’ve had enough.

Jul 4, 2023 at 6:17pmWeb
Show threadSinthoras39Jul 4, 2023

I don’t think it is that easy. But a good feature would be if you could make backups from communities on other instances in case they go down. It would not save the pics, but you should use a third party image service to keep the hosting costs of you instance lower anyway. In the same way, you, but only you, could download a backup of your account.

Also a good idea would be a new protocol which allows syncing feeds/communities between instance, which effectively is mirroring.

Show threadMaestroJul 4, 2023

This is what federation already does, mostly. Federation means that threads and comments are copied between instances and kept up-to-fate. You are reading this from a copy kept on discuss.tchncs.de. I am reading this from a copy on kbin.social. The original is kept at lemmy.world. If lemmy.world goes down then our copies remain. They are still readable. But you won't see my new comment and I won't see your new comment because lemmy.world was responsible for syncing our copies.

This applies to text posts, links, comments and votes. Images and videos would be gone because they are not copied, just linked to.

Show threadHamSwagwichJul 5, 2023

That's correct and that's the problem. If a given community server goes down, that community basically just becomes an archive. It really needs to be able to continue without the host instance, similar to how a mesh works. Each remaining server routes around the dead node.

There is also the problem of search engine indexing... If a given server goes down, that information is lost to the search engine, even though it's still on other nodes.

Which also leads to duplicate content problem for search engines, as ECU m each node of a given community contains the same information for a given post, making it crappy to index and search.

Show threadofcourseJul 4, 2023

I agree with OP that instances being closed any time is an issue that would need to be resolved fairly soon.

A new user currently has a choice for joining from a number of instances but there is no assurance to ongoing existence for them. Along with that, there is no way to transfer user accounts and data across instances, afaik. Having the option to transfer user accounts would help with the instance closing and eventually make the fediverse more stable.

If a user can transfer their accounts and data, there will be less hesitancy to join a new instance, and user accounts and data can be distributed across more instances. This can also work in such a way that if a subset of user data does not meet the criteria for another instance, then that subset of data is not migrated.

Another issue is with the presence of same community/magazine in multiple instances (let’s say [email protected] and [email protected]) which is frustrating for users since they need to track multiple communities for similar content and the same content is being copied to multiple communities. This should also be resolved by implementing account migration. We are already seeing that communities on certain instances are becoming the prevalent ones. This creates an incentive for the admin of those instances to not shut down. And if they did decide to shut down the instance, then the users can just migrate to another instance and the prevalent community will also get to keep all its data, just in the new instance.

Show threadofcourseJul 4, 2023

I agree with OP that instances being closed any time is an issue that would need to be resolved fairly soon.

A new user currently has a choice for joining from a number of instances but there is no assurance to ongoing existence for them. Along with that, there is no way to transfer user accounts and data across instances, afaik. Having the option to transfer user accounts would help with the instance closing and eventually make the fediverse more stable.

If a user can transfer their accounts and data, there will be less hesitancy to join a new instance, and user accounts and data can be distributed across more instances. This can also work in such a way that if a subset of user data does not meet the criteria for another instance, then that subset of data is not migrated.

Another issue is with the presence of same community/magazine in multiple instances (let’s say [email protected] and [email protected]) which is frustrating for users since they need to track multiple communities for similar content and the same content is being copied to multiple communities. This should also be resolved by implementing account migration. We are already seeing that communities on certain instances are becoming the prevalent ones. This creates an incentive for the admin of those instances to not shut down. And if they did decide to shut down the instance, then the users can just migrate to another instance and the prevalent community will also get to keep all its data, just in the new instance.

Show threadofcourseJul 4, 2023

I agree with OP that instances being closed any time is an issue that would need to be resolved fairly soon.

A new user currently has a choice for joining from a number of instances but there is no assurance to ongoing existence for them. Along with that, there is no way to transfer user accounts and data across instances, afaik. Having the option to transfer user accounts would help with the instance closing and eventually make the fediverse more stable.

If a user can transfer their accounts and data, there will be less hesitancy to join a new instance, and user accounts and data can be distributed across more instances. This can also work in such a way that if a subset of user data does not meet the criteria for another instance, then that subset of data is not migrated.

Another issue is with the presence of same community/magazine in multiple instances (let’s say [email protected] and [email protected]) which is frustrating for users since they need to track multiple communities for similar content and the same content is being copied to multiple communities. This should also be resolved by implementing account migration. We are already seeing that communities on certain instances are becoming the prevalent ones. This creates an incentive for the admin of those instances to not shut down. And if they did decide to shut down the instance, then the users can just migrate to another instance and the prevalent community will also get to keep all its data, just in the new instance.

Show threadPwnyJul 5, 2023
As a user, if the instance I originally joined is shutdown - I would want a feature that lets me easily transfer my account and data to another instance. But not sure how this would work considering right now I could go make an account with my username in a different instance…
Show threadWilfordGrimleyJul 4, 2023

That is how Aether works.

Aether is a reddit alternative that I prefer to Lemmy honestly, however it is not being maintained.

Show threadmyselfJul 4, 2023
God I’d love to implement a lemmy instance as a byzantine fault tolerant distributed system… Anyone wanna lend me a working brain?