We've been hit by a massive DDoS attack. The site may not work as expected. We're working on mitigating the attack. Fastly are helping us. If in doubt, check status.mastodon.social for information.
Everything is back to normal now. Yes, it was an attack, not legitimate traffic. No, we don't know who was behind it. I agree the timing with Twitter being down was unfortunate.
Fastly and Datadog who both sponsor us by providing us a free service were instrumental in analyzing and mitigating the attack. Shout out to our own team as well, glad I don't have to do this alone anymore!

@Gargron   

Kidding! Well done mate 

@Gargron Thanks to all! The spirit of this community is heart warming.
@Gargron πŸ‘ πŸ‘ πŸ‘ & thanks to all
@Gargron Time to start a fund drive for a Korbomite device (original Star Trek).
@Gargron This can be related to the down Ielo ISP in France
The timeline match.
@Gargron πŸ‘πŸΌ πŸ‘πŸΌ πŸ‘πŸΌ
@Gargron Will there be a blog post detailing what and how it all happened, and list any IOCs, etc.?
@Gargron πŸ‘ πŸ‘ πŸ‘
@Gargron well done! ❀️
@LaureM Not necessarily. Depends on the circumstance. The Post could easily host its own instance. A smaller newsroom would have trouble. @Gargron
@Gargron Are you not employing any sort of rate-limiting?
@Gargron
Thanks to you and everyone who helped avert the attack
@Gargron
Did it affect joinmastodon.org?
@Gargron Thank you and the team for all your hard work getting this handled!
@Gargron maybe the api team at twitter got reassigned to ddos attack mastodon

@timberwraith

It is a prelude to invasion. Those aliens whose UFOs we shot down.

@Gargron thanks for all the hard work!
@Gargron Weird way to spell suspicious

@Gargron

I still see this via US-CA-SJ:

Error 403 Forbidden
Forbidden

Error 54113
Details: cache-sjc10058-SJC 1678140947 872542668

Varnish cache server

@johnlogic @Gargron I'm still getting that Varnish cache error page too - but only on the screen I was using during the outage. On other screens things are working fine.

Deleting cookies doesn't fix it. Restarting the browser doesn't fix it.

@johnlogic @Gargron I suspected that with a cache problem, the fix was to just wait until it expired, and indeed it's all better now.
@Gargron Thank you so much 😊 πŸ™
@Gargron - Thanks for your attention!
@Gargron Feel your pain. Had 9 months of incessent DDoS last year. It's so bloody awful :(
@Gargron I'm getting 403 errors trying to access the site - am I blocked then? I didn't do anything apart from try to load my timeline while you were down!
@Gargron Can't even get on from mobile data - are you blocking the UK?
@Gargron My conspiracy theory of the moment: Elon rerouted all the API calls this way.