Donncha Ó CaoimhJust testing the load on my server. May delete later.
Also, enjoy the video of Gaybo on the Late Late Show in the early 80s.
https://odd.blog/2022/12/21/computers-are-the-big-thing-this-year/
261 embed requests since this was posted a few minutes ago.
359 requests for the post itself.
1 minute load average topped out at 1.34 for 5 seconds before going back to 0.34.
I had garbage collection TTL set to 60 seconds and I think it expired in the middle of that rush of requests for the page.
I had tail -f on my log in a screen session, which adds a bit to the load too.
My blog was still very responsive the whole time.
Now to test it again with a longer TTL. Not sure if Mastodon instances cache the preview.
https://odd.blog/2022/12/21/computers-are-the-big-thing-this-year/
Looks like they do cache the preview. That's good news at least. :)
Here's a very old blog post, which surprisingly has a link to a blog post that is still alive. Colour me surprised!
289 requests for the post
273 requests for the embed preview
1 minute load average topped out at 0.71 for 5 seconds.
The post was cached by WP Super Cache, with a garbage collection TTL of 600 seconds, and GC check every 120 seconds. Only using simple caching.
Apache:
KeepAlive off.
5 start servers.
Min 10 spare servers.
29 processes.
I currently have 1.8K followers, which might be more than the average account here, so I guess those followers are split over about 270 instances.
Gabriel Beecham
Tom Walker@donncha This is what I've been saying to people when they mention this embed-previews rush problem – it won't take down your blog if you have some caching set up. As you would know of course!
@tomw the only thing I will say, is the very initial rush might cause problems if they don't have the spare capacity to meet it.
That's why I have the "5 min spare servers". I'm going to change that to 1, restart Apache and test it again with this URL:
@tomw With start and min spare servers set to 1 it took much longer to service every request and Load went up to 1.24.
It definitely pays to have the capacity there in the first place, even if the web server is just idling.
Also, if it's a new post, there might not be a cache file there when the first few requests come in for it. This is a long standing problem though. Popular sites with RSS feeds have (or had) the same issue when feed readers were popular. Need to pre-load new posts...
@donncha Yes that makes sense. It would maybe be worth doing some kind of guide for people – the discussions I've seen tend to jump straight to talking about Cloudflare, CDNs etc when this can be solved at the WP level with cache (and yeah maybe some web server config as needed).
@tomw Good idea. Even "simple caching" with WP Super Cache works wonders.
I'll test with caching turned off and monitor the load. :)
Here's another old blog post. Not cached at all.
@tomw Interesting. Load average only spiked to 1.12 with no caching! Not bad!
267 requests for the embed of that page.
290 requests for the page itself.
No extra caching like memcached or anything.
This is with Apache reset to 5 start servers, min spare servers set to 10 again.
@donncha Great! I think the bit people don't get is that you just need to avoid hitting the database with every page load, at a minimum, and you'll get through it. That's what's been taking people's sites down.
Gidi Kroon@donncha What was the spread in time of the requests? I heard that Mastodon tried to spread the load by having a random delay before fetching, and your test seems ideal to know whether I heard correctly... Non-Mastodon servers may have been the initial peak of 5s.
(I know that Friendica caches the preview, because when I edited a blog post after testing the link it kept showing the old version also in new posts)
(I know that Friendica caches the preview, because when I edited a blog post after testing the link it kept showing the old version also in new posts)
@gidi Maybe there is a delay, but when there are so many servers it probably doesn't matter. I saw servers hit the post within a second or two of me posting it. mastodon.ie first of course, then others.
@donncha Sounds like the retrieval happens at least soon after the post has been delivered, with delivery delays giving some natural spread. The caching setups on people's blogs are going to have to do their job once we get posters with millions of followers...
@gidi Yeah, people like that will want sites running on WordPress.com or similar, even if they don't get the human eyeballs on it afterwards.
Then again, there is probably only one request per Mastodon instance, so as long as there are few instances it might be manageable.
A quick check shows 390 unique user agents hitting the "i-have-to-try-this" URL. Not bad for 1.8K followers.
Blog post summary written, but it's late here. I'll share it in the morning when I can keep an eye on the logs and load average.
Blog post summary is up! My server barely noticed the extra traffic which followed the usual pattern. wp-json request followed by page request. 619 requests in total.
https://odd.blog/2023/01/02/the-mastodon-onslaught-on-your-blog/
Ross McKayThat sounds wrong way around; is it true? I'd have guessed page request followed by wp-json oembed request, otherwise how does the fedi server know how to make the oembed request?
FWIW, whenever I publish a new post, Facebook is right there to load the post and cache it. My guess is that my SEO plugin does that (The SEO Framework), certainly isn't me doing that (no FB for me).
Even only caching pages with WP Super Cache will at least halve the load from the fediverse; worth it.
@webaware I did have that list of URLs reversed initially, but then I looked in my logs and saw the wp-json URL being hit at the same moment as the page URL by the bots I looked at.
Since you asked, I checked again and other bots visited the page first! I'm not sure what to make of that.
Jeremy Herve@donncha It's an interesting analysis, thanks for sharing! Definitely turn it into a blog post, I think lots of folks would be interested in it.
It sure highlights an interesting impact of the fediverse, bringing in some new challenges for everyone.
It sure highlights an interesting impact of the fediverse, bringing in some new challenges for everyone.
@jeremy I need to write a Supercache addon plugin that preloads the latest post when published. That would help a lot and should be simple enough to do.
The Android Police owner asked about that years ago (he was getting trashed by RSS feed readers) but I never got around to it. I think he moved to a different caching plugin after that.