Edit: it's fixed now
If you follow anyone from the Mastodon instance oldfriends.live they seem to be having some major problems at the moment.
They're reaching out asking for help from anyone who feels like they're enough of a #PostgresQL expert to help them troubleshoot what sounds like major database issues.
My VPS had a hardware failure. Working hard to get it back up soon. Should be today or tomorrow. If that fails, I will revert a server snapshot over weekend.
The 500 error page is correct, though, that shows my image, the default error and the hashtag. That is what everyone that goes directly to the website URL should get...
Issue: missing chunk number 0 for toast value 4480499 in pg_toast_16547 in the accounts table.
@losttourist @revivalrecords
@SpaceLifeForm
https://masto.ai/@FeedingTubePaul/116038470673031508
2/ The database backup is a few days older than I would have liked so I learned my lesson. Since it is just two accounts, and I have this backup account on masto.ai and was procrastinating moving Hashtag games account to its own domain, which I have now done, I was just buying some time to try to resolve it.
3/ FYI, I pulled the trigger on the snapshot restore and #OldFriends is back up. Once accounts got cleared of the pg_toast chunks, it was failing in the status table and that would have taken another several days to clean and who knows how many other tables.
Working out a better database backup solution...
Flippin' 'eck, Tucker!
Oldfriends.live Paul backup
SpaceLifeForm