Interesting. I publish an off-topic blog post without advertising it anywhere, 1.5 hours later my stats show 130 visits but only one request to the website’s CSS file. I guess I’ve inadvertently created myself a bot honeypot, regardless of what their user agent says. One bot actually made three identical requests except for the user agent: Safari, Axios and Firefox.

There is a certain increase in bots masquerading as regular browsers here, it’s almost 40% of them. I’m fairly certain it weren’t that many a few years ago. Some bots claim to have been referred by Google, an oddity that I’ve already noticed in the stats for new articles published in the past months.

So, is everybody and their dog scraping web content for AI training these days?

@WPalant does RSS get counted in the stats?
@brown That’s only about requests to the article itself, not RSS feeds.
@WPalant yea, even with my tiny website I'm now getting periodic bandwidth usage warnings from my host as AI scrapers "crawl" it. They obviously throw any kind of reasonable rate limiting out of the window 😐
@kzar I’m used to bots as such, but my website shows a factor 3 increase in “visits” compared to a year ago, and I suspect that it is all bots. And it’s really many different bots with different crawling strategies, often masquerading as actual users.