And that's another ban hammer for AI scrappers, this time forging their user agent as well as ignoring robots.txt
101.32.0.0/16
#tencent
Now it's also ignoring tens of HTTP 4xx errors a second too...
And that's another ban hammer for AI scrappers, this time forging their user agent as well as ignoring robots.txt
101.32.0.0/16
#tencent
Now it's also ignoring tens of HTTP 4xx errors a second too...