Jack Yan (甄爵恩)If #Google says you should not block it, then you should block it. Pays to do the opposite to what Big Tech says as they never have your best interests at heart.
https://www.improvemysearchranking.com/what-does-googleother-do/
Djoerd Hiemstra 🍉@jackyan I suspect they created #GoogleOther to break the crawling / robots.txt / nettiquette rules without getting too many repurcusions on #GoogleBot.
@djoerd I think you are right. Iʼve blocked it for a few weeks and there are no issues for our site: searches or traffic. It goes to a lot of unusual pages on our sites, e.g. it tries to dig out old Mediawiki revisions on one of them.
@jackyan Interesting. Sounds like some at Google must have said: We need "AI crawlers" too! https://c3.unu.edu/blog/the-rise-of-ai-crawlers-a-digital-menace-reshaping-the-internet-landscape
@djoerd They definitely do. We block one called Google-CloudVertexBot, and we also block anything coming from googleusercontent.com. You can see where we managed to deal with most of the #theftbots in the middle of the graph:
https://jackyan.com/blog/2025/10/the-trails-of-the-theftbots/
A lot were from Tencent and ByteDance. Tencent hits this site over 30,000 times a day (still does).
The trails of the theftbots
This was interesting: how the “AI” theftbots (yes, I’m coining that word) affected Autocade’s traffic. We had blocked a bunch of them already—notably the western ones and ByteDance—but there were many others that still got through. This is from Cloudflare, but even their default settings miss a bunch, especially Chinese ones, and they don’t