Okay, Dragon Style is down for upgrades and whatnot.
Step 1: take a snapshot in case Shit Goes Wrong. Digital Ocean says it may take more than an hour. Guess I'm getting some drawing done. Or something.
This is after like 12min so maybe more like 30-45 min? I'm just browsing the web and eating breakfast.
Backup done at about an hour. Drawing of myself waiting for this to happen: not finished.
oh god I have to update the ssh key on the server as well as on my local machine before it will talk to github
done with swearing at ssh and git, moving on to swearing at migrations
but first Docker must recompile the entire world from first principles
database migration part 1 done, part 2 in progress
*goes back to drawing*
shitbuckets, the site is running but just giving me an error screen
the mastodon docs say there is supposed to be a reference number in the html of the error screen that I can use to track it down in the logs but guess what? No reference number. Ugh.
rolling back to the snapshot, debating between trying again to get to 4.1.2 after figuring out how to do a suggestion from the mastodon project discord, or trying to just go to 3.5.9 which is the bugfixes backported to 3.x and a smaller change from the 3.4.6 that we've been running
anyway in lieu of your normal stream of dragon-related content on dragon.style, please enjoy this drawing I did while waiting for stuff to backup/compile/etc.
Oh good at least restoring from the backup worked, I am glad I took the time to make that happen!
I took it down again immediately because, well, major vuln out in the open.
new snapshot in progress so I don't have to fix SSH key stuff for a third time if I need to revert again, gonna take a walk in the park and Not Think About This for a while.
did you know that if you flush every single old docker image from your machine it is possible to get it into a state where it cannot seem to figure out how to get a new copy of the OS image when you ask it to rebuild everything from first principles? It's true! I just did it!
*restore from last snapshot* god this game takes so long to respawn
well that ain't right, I flushed a bunch of caches and shit and rebuilt everything and it's still saying it's on 3.4.6, pinging @anthracite to see if that goes through or if Safari is just loading up some sort of Potempkin village from cache
fucking weird: Safari on my phone and Mac both bring up the site and say it's on 3.4.6, firefox and chrome on the Mac bring up the Keybash Elephant. One person on the Mastodon discord had suggested cache problems and there is definitely some of that going on.
I am tired of hunching over the computer and I am going to go home and see what happens if I bring the site back up and ask the SO to load it up on their computers. It's down for now. Still.
I think this may be one of the highest-hassle updates I've ever had, usually once I get through Dependency Hell on my local dev copy it's pretty trivial to bring it up on the server!
HMMMM. Docker is definitely running some stuff that was built 17 months ago no matter what I do. Perhaps "docker-compose build --no-cache" will work?
*starts building the entire world from first principles, AGAIN*
nnnnnnooooope, it's still running images created 17 months ago even after I tried manually deleting everything except the Ubuntu and Docker images and doing 'docker compose build --no-cache'.
What the shitting fuck. Perhaps I need to just blow away all things Docker on the remote server and reinstall THAT from scratch. It's been a long day and I think maybe that's for tomorrow, my apologies for not having the site up tonight. 
@egypturnash Nah, thank you for all the work you're putting in on it!
@Momentrabbit Thanks for the chill <3