Okay, Dragon Style is down for upgrades and whatnot.
Step 1: take a snapshot in case Shit Goes Wrong. Digital Ocean says it may take more than an hour. Guess I'm getting some drawing done. Or something.
This is after like 12min so maybe more like 30-45 min? I'm just browsing the web and eating breakfast.
Backup done at about an hour. Drawing of myself waiting for this to happen: not finished.
oh god I have to update the ssh key on the server as well as on my local machine before it will talk to github
done with swearing at ssh and git, moving on to swearing at migrations
but first Docker must recompile the entire world from first principles

database migration part 1 done, part 2 in progress

*goes back to drawing*

shitbuckets, the site is running but just giving me an error screen
the mastodon docs say there is supposed to be a reference number in the html of the error screen that I can use to track it down in the logs but guess what? No reference number. Ugh.
rolling back to the snapshot, debating between trying again to get to 4.1.2 after figuring out how to do a suggestion from the mastodon project discord, or trying to just go to 3.5.9 which is the bugfixes backported to 3.x and a smaller change from the 3.4.6 that we've been running
anyway in lieu of your normal stream of dragon-related content on dragon.style, please enjoy this drawing I did while waiting for stuff to backup/compile/etc.

Oh good at least restoring from the backup worked, I am glad I took the time to make that happen!

I took it down again immediately because, well, major vuln out in the open.

new snapshot in progress so I don't have to fix SSH key stuff for a third time if I need to revert again, gonna take a walk in the park and Not Think About This for a while.

did you know that if you flush every single old docker image from your machine it is possible to get it into a state where it cannot seem to figure out how to get a new copy of the OS image when you ask it to rebuild everything from first principles? It's true! I just did it!

*restore from last snapshot* god this game takes so long to respawn

well that ain't right, I flushed a bunch of caches and shit and rebuilt everything and it's still saying it's on 3.4.6, pinging @anthracite to see if that goes through or if Safari is just loading up some sort of Potempkin village from cache

fucking weird: Safari on my phone and Mac both bring up the site and say it's on 3.4.6, firefox and chrome on the Mac bring up the Keybash Elephant. One person on the Mastodon discord had suggested cache problems and there is definitely some of that going on.

I am tired of hunching over the computer and I am going to go home and see what happens if I bring the site back up and ask the SO to load it up on their computers. It's down for now. Still.

I think this may be one of the highest-hassle updates I've ever had, usually once I get through Dependency Hell on my local dev copy it's pretty trivial to bring it up on the server!

HMMMM. Docker is definitely running some stuff that was built 17 months ago no matter what I do. Perhaps "docker-compose build --no-cache" will work?

*starts building the entire world from first principles, AGAIN*

nnnnnnooooope, it's still running images created 17 months ago even after I tried manually deleting everything except the Ubuntu and Docker images and doing 'docker compose build --no-cache'.

What the shitting fuck. Perhaps I need to just blow away all things Docker on the remote server and reinstall THAT from scratch. It's been a long day and I think maybe that's for tomorrow, my apologies for not having the site up tonight. 

hahaha omfg I can't even upgrade Docker because they have moved to a completely different URL in the time since this virtual machine was set up and this broke apt, this is becoming comedy

*cracks knuckles* *invokes hermes/mercury*

let's get hackin', may I quickly find the right cryptic invocations to make all these caches behave and get dragon.style running again today!

I basically spent all day uninstalling Docker and upgrading the server's OS. And drawing goofy porn while it compiled for hours on end.

@egypturnash there is a new batch of release as of about an hour after you started this specific update attempt, btw. they fix some admin crashes, media processing, and app icon generation failures, so if you're running into that, it might be relevant.

(this is @kistaro's vore alt)

@synbria @kistaro

Ooh, thanks for the heads-up! Once I get this one working it should be pretty easy to update my dev copy to that, then put it on the site and go live again.

@egypturnash Nah, thank you for all the work you're putting in on it!