Tony Yarusso
MistyThe more and more old servers I see close, the more intolerable it is that Mastodon still doesn't offer a way to migrate data like posts to new accounts. A user's post history is valuable, and it's more valuable the older the account is. Users *should* have the option to migrate those to a new server!
I think for many users, if they lose their posting history when moving to a new server, they have less reason to stay here instead of moving to a new network. If they're losing everything anyway, why not move to bluesky or something else?
I'm thinking about this because I saw a Mastodon server of eight and a half years shutting down, which is leaving its migrating users in a very awkward position if they have years' worth of posts that are being lost.
It's not, to be clear, the server admin's fault - they don't control Mastodon's feature set. It's squarely the responsibility of Mastodon to supply the server migration tools necessary to resolve these problems for its users.
João@misty 100%. I was very surprised it does not support that when I moved instances. I understand there’s need for reviewing posts in a new instance to make sure they comply to the new instance’s rules, but that is not a technical problem. Do you know if it’s “by design” that they don’t support migrating posts?
@magnavespa My understanding is that the migration tools were added pretty early in Mastodon’s life, covered the easier to handle stuff, and haven’t meaningfully changed since then. There are some technical challenges in importing posts but GoToSocial’s post importer did this; Mastodon has elected not to pursue this so far.
John Francis 🇨🇦🦫🍁💪⬆️@misty exporting posts is easy. Importing posts is intractable. The posts are backdated. The other accounts and instances mentioned in the backdated posts don't relate to the reposting account. Weird notification issues. Maybe tractable? Easy enough to just allow hanging a post archive searcher, viewer off the profile..."here's my old stuff", and allow the new account to use good old posts as a template.
EDIT: not intractable with appropriate limits
@johnefrancis It’s a solvable problem. GoToSocial can do this using a Mastodon archive export.
Scott VE3QBZ@misty @johnefrancis Does GTS's import process re-write the federated timeline somehow? If I move to a new server, with a new ID, and import my post that is a reply to a post on someone else's instance, does that other person see the migrated post as a reply properly? I don't think posts have UUID's across the fediverse, they use URI's, which means posts aren't unique, their "location" is unique, which would make this tricky in my mind, but probably I don't have the full picture.
post-Gundam cryptography@misty @scott @johnefrancis it can't, that's why we don't support importing conversations with other accounts. UUIDs wouldn't do it either since anyone can copy a UUID, but you're not wrong in understanding that the URL being the ID is a problem with current AP.
@vyr @misty @johnefrancis I mean we could go down endless rabbit holes to make it work, like UUID signing or something and some kind of tracking for UUID to URI mappings, but that's a veyr deep rabbit hole.
I do think a lightweight middle ground would be for Mastodon to allow browsing of a local post archive, selecting posts, and having them posted as new posts on the new instance, albeit at the expense of loss of relationship fidelity with respect to the broader federated timeline.
@scott @misty @johnefrancis fwiw, i did briefly look into what it would take to add the GTS-compatible backdating API extension and supporting internal changes to Mastodon. since i gave up on trying to work with Eugen and no longer run Mastodon myself, i'm not the girl to do it, but here's what you'd need to do to provide the API that the slurp import tool uses:
- accept backdated posts thru the standard posting API method, using the scheduled post time parameter set to some date in the past
- reject posts that are replies to or mentions of someone that isn't yourself
- do not push backdated posts to other servers: skip the federation worker
- do not generate local notifications for backdated posts (for example, if someone has new post notifications on for you, they probably still don't want to get ten thousand of them at once): skip the notification worker
- that's basically it
@johnefrancis @misty implementor here, it wasn't even that hard. the posts are backdated. conversations with other accounts aren't imported because of the context problem you mentioned. notifications and push federation are suppressed (boosting is still possible). compromises were necessary but it's still a massive improvement over "lol your Fedi posts are gone? put them on a static website outside Fedi".
the few people still working on AP are either uninterested in figuring out better post portability or don't have the time or clout to push any given scheme through, given that it'd need to be supported by a decent fraction of Fedi to work. that's unfortunate. but i'm mildly surprised nobody's patched Mastodon to support good-enough solutions, given that we don't have good ones yet.
@vyr @johnefrancis The existing follower/following account migration is itself a good-enough solution that's stood for many years; if that was good enough to ship, a good-enough compromise solution for post importing should be fine too.
Ember is so tired of people. 
@misty @johnefrancis also, the mastodon team definitely has the resources to implement or standardise some way of not breaking all old threads
filipa mv@misty ya, overall account migration needs work esp around adversarial admins or sudden shut offs.
this is not helpful but that said re post data i’m kinda leaning towards opposite in that i feel it’s maybe healthier that this is a somewhat ephemeral medium?
if your server dies repost your faves and let The Archive handle the rest kinda vibe
@misty this thought is informed by when i wanted to migrate off xitter and i got all of my post data and i thought of hosting it somewhere but
i came to the conclusion that i myself could not be arsed to read through thousands of dumb tweets to pick out the good ones & the risk of hosting a convo i would now regret seeing was not worth it
@misty (i’m not trying to contradict the feeling you have & rightfully point out that people care about this stuff! more a reflection of an epiphany i had where i realized that i didn’t care about it as much as i thought i did)
@phillmv Ywah, I think that’s a valid perspective. I have a bit of a knee jerk reaction to it mostly because the conversation thus far has been dominated by people with that perspective arguing that as a result the migration option *should not exist at all, for anyone, even as a choice*, and that’s led to the current situation where Mastodon doesn’t allow it. I think it’s totally fine for post migration to be opt-in but there’s enough people with post histories where it matters it should exist!
@phillmv I run my own server so I’m not vulnerable to unexpected shutdowns, thankfully. But my own post history does have plenty of things I actually would want accessible from the same server I’d continue posting to after moving.
I could do that on GoToSocial or, ironically, Bluesky. But it’s simply not an option on Mastodon.
jerIf you get too concerned about it, GoToSocial can import posts from Mastodon servers as well. This is not an argument against your point, just a potential solution to the problem for you someday.
I imported my history from mastodon.social, a Firefish instance, an Iceshrimp instance and a GoToSocial instance when I set up my own GTS server.
(It only gets posts you authored, though, nothing in reply to anyone else, so it's nowhere near as good as how Bluesky does it.)
D. Olifant Still Won't Shut Up@misty @phillmv My suggestion is to get the Mastodon archive regardless, and you can at least preserve your posts as a static HTML archive with Posty.
Bringing them over to a new instance, as others have mentioned, is currently only supported on GoToSocial via Slurp.
I wrote this piece on how I closed down Oliphant Social and maintained a static archive of the posts there.
@oli I think it’s great that exists! It provides an excellent solution for people with the technical skills to accomplish it, and looks very slick. I think it’s beyond the means of most of the users I’m thinking of though.
@misty Absolutely.
The 'static archive' thing should be either part of the Mastodon export, or Mastodon should facilitate it. (They created the export format, after all, just never gave us any way to use it in Mastodon itself.)
And I'm hoping GoToSocial will just build in some kind of functionality that lets you point it at your former account (after an account move, perhaps) and stream over your posts from the active server.
But Mastodon should also just build this feature, too.
@misty My theory is that Mastodon development is about 98% focused on use-cases of mastodon.social, and mastodon.social won't care about that feature, which is why I think it won't happen.
I'm hoping GoToSocial will just build in some kind of functionality that lets you point it at your former account (after an account move, perhaps) and stream over your posts from the active server.
This is almost halfway there already, as you don't even need a Mastodon export. After authenticating, slurp will pull all the posts from a live instance and dump them to a file. I bet being able to pipe the output of slurp downloading into slurp uploading wouldn't be a very big amount of work.
@nyquildotorg @misty It's a natural conclusion, and Slurp is also written in Go, and Vyr contributes to GoToSocial and even did the patch for backdating posts that made it all possible in the first place.
mcc@misty @mcc idk what i would do if i were god-empress but related vibes:
- humans are cognitively unequipped to handle every stray thought being permanent
- in a world with finite resources is it sustainable to archive every stray thought? maybe it’s a hoarding instinct
- we deal a lot with this at work where we spend a fair bit of resources to store a lot of data we know no one ever looks at it & it causes scaling issues
- bracketing historical research, *no one including myself* will ever read through my post archive. a lot of tweets grow stale. maybe 5% of my posts are worth preserving?
Kim
Samantha Rose@FizzyDaisies If the old server is still up you can, as your old account still exists and you can log in so you can delete posts as you like.
Of course if you deleted your account you can't do that, but then your posts will all be gone from the server anyway.
What happens to posts cached on other servers is a bit less clear, but I think (not sure!) that they will in most cases always be deleted after some time anyway.
@samantha42 That’s great. Thank you! I moved from Mastodon(dot)social and never tried to login after. I didn’t know I could delete that account. That’s even better.
@FizzyDaisies Yes! After you've migrated you're still able to log back into the old account to do a few things like post deletion.
@misty Excellent. Thank you!
Garrett Wollman@phillmv @misty @mcc Remember (or maybe you don't) the *enormous* consternation on Usenet when Google started archiving everything, forever? And people thought that posts lasted maybe a month on a "big" server, but it turned out that Henry Spencer had 9-track tapes full of archived posts from the 1980s? Whatever side you fall on in terms of preservation, when the social expectations change, someone is going to wind up unhappy.
@phillmv @misty @mcc (Usenet being kinda the inverse of activitypub, your articles are broadcast to everyone and stored locally on their servers so it doesn't matter if your server goes away but you have no formal control of how long your articles reside anywhere else and no ability to delete anything.)
argentum
Tushar Chauhan
damien 🥖🐈⬛🧣@misty gets a bit more embarrassing when GoToSocial essentially solved this (even by importing from a Mastodon archive)
@eramdam I'm really glad it offers it! I don't see many gotosocial servers available as general registration for the public though, so it's not really an option for most people.
Aisling "the bastard of earth" Fawn@misty yeah and the (by design it seems) focus to not have a built in client probably doesn’t help either :/
zbrown@eramdam @misty in fairness it's still fairly early in what the creators play for it, and creating a front-end can become a massive time/effort sink distracting from building solid foundations.
Frustrating certainly for those of us who might like to use it certainly, but I respect what the team is going for there and they'll probably produce a better server as a result.
joshuatj@misty indeed. Are there third-party tools that might help with this? Or is this very much can only be done with changes in core?
@joshuatj There aren't. It has to be supported by the server, so there's nothing third party tools can do. GoToSocial can do it, working from a Mastodon data export, but Mastodon itself can’t.
@misty sad. I really like Mastodon, but I've heard anecdotally one of the bigger hurdles for people joining Mastodon was deciding on an instance at the beginning. I can imagine being able to migrate posts across instances would remove this hurdle.
@joshuatj Yup. :/ I see people who say “it doesn’t matter, you can always change later”, but that’s only true if you think your posts are of zero value. That’s not true for most people!
The Tattooed Nonna 🧙♀️@misty @joshuatj I was so attached to my history on twitter that I downloaded it all when I left.
But here i was determined to not become so obsessed with the past. I like that mastodon has an option for auto delete of posts.
But for people that want to save them, if you move servers, they still exist and you can add a redirect to your new server
But here i was determined to not become so obsessed with the past. I like that mastodon has an option for auto delete of posts.
But for people that want to save them, if you move servers, they still exist and you can add a redirect to your new server
Rihards Olups
Morgan ⚧️@misty @richlv @joshuatj I think that'd get tricky because if I someone replies to me and I reply to them, and then I move servers... If my account kept those posts, it couldn't maintain the thread there unless the other person's posts were also edited 😬 to mention my new account instead of the deleted one
Editing other people's posts feels like something my account, my server, or my old server should not have the authority to do
Editing other people's posts feels like something my account, my server, or my old server should not have the authority to do
@raphaelmorgan @richlv @joshuatj In an ideal world we'd have some kind of DID setup so post and user IDs aren't irrevocably tied to a specific server…
JJ 
@misty @raphaelmorgan @richlv @joshuatj you don't need DIDs -- you just need a consistent mapping from an old user id to a new user id and an old post id to a new post id. and a server->server bulk import tool and functionality for servers to update their internal user mappings upon a 301 permanently moved.
problems arise though because 1) mastodon's post ids are only unique per-server and 2) a server's internal representation of external users is a username@domain tied to an internal identifier, which needs to update upon receiving a 301. these are both doable, as i understand it -- but only really doable by someone very familiar with the codebase since ids are used everywhere... so realistically one of the very few masto core developers :c
@misty fwiw i wrote abt this in more detail and why i think it's feasible some time ago, if you're interested: https://github.com/mastodon/mastodon/issues/12423#issuecomment-1956384303
(it got immediately misunderstood tho 🫤)
econads@raphaelmorgan @misty @richlv @joshuatj some kind of forwarding then? Like metadata of all the places it was posted in the past?
Paolo Amoroso@joshuatj This third-party Mastodon migration tool is a laudable effort but using it is well beyond the abilities of ordinary users.
Stephen Bannasch (316 ppm)