J'ai limité à 1 req/min certains bots en renvoyant un code 429 #ClaudeBot
Ca fait quand même 3 hits/s #ClaudeBot
FYI: Anthropic clarifies what its three web crawlers do - and how to block them: Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility. https://ppc.land/anthropic-clarifies-what-its-three-web-crawlers-do-and-how-to-block-them/ #WebCrawlers #SEO #DataPrivacy #ClaudeBot #SiteVisibility
Anthropic clarifies what its three web crawlers do - and how to block them

Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility.

PPC Land
ICYMI: Anthropic clarifies what its three web crawlers do - and how to block them: Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility. https://ppc.land/anthropic-clarifies-what-its-three-web-crawlers-do-and-how-to-block-them/ #Anthropic #WebCrawlers #SEO #ClaudeBot #DigitalMarketing
Anthropic clarifies what its three web crawlers do - and how to block them

Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility.

PPC Land
Anthropic clarifies what its three web crawlers do - and how to block them: Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility. https://ppc.land/anthropic-clarifies-what-its-three-web-crawlers-do-and-how-to-block-them/ #Anthropic #WebCrawlers #ClaudeBot #SearchEngine #DigitalMarketing
Anthropic clarifies what its three web crawlers do - and how to block them

Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility.

PPC Land

Comment bloquer les crawlers IA qui pillent votre site sans vous demander la permission ?

https://fed.brid.gy/r/https://korben.info/bloquer-crawlers-ia-robots-txt-htaccess-nginx.html

#Anthropic really tried to hit #Autocade today … but we had pre-blocked the buggers. #ClaudeBot
Ah, the digital pyramids have spoken: You really don't need 🐕‍🦺 Anubis 🐕‍🦺, unless, of course, you enjoy turning your visitors into cryptographic slaves. 🙃 Apparently, ClaudeBot's relentless pursuit of your sweet, sweet data isn't enough to warrant anything more than a raised eyebrow. Maybe just get a bigger 'robots.txt' and pray to the cyber gods! 🙏💻
https://fxgn.dev/blog/anubis/ #digitalpyramids #cryptographicdata #ClaudeBot #robots.txt #cybersafety #HackerNews #ngated
you don't need anubis

don't use anubis just to block llm scrapers

fxgn

ClaudeBot/1.0 via l’IP 216.73.216.160 m’a saturé mes serveurs

J’ai donc bloqué cette IP : 216.73.216.160 . En fait j’ai bloqué 216.73.216.0/24 et 3.141.17.0/24. 

C’est visible via : https://webanalyse.cyber-neurones.org/ 

1119 354 (27.47%)9 (00.09%)1.7 GiB (16.20%)216.73.216.160526 720 (06.15%)4 (00.04%)386.9 MiB (03.53%)216.73.216.136711 672 (02.69%)1 (00.01%)175.9 MiB (01.60%)216.73.216.11489 434 (02.17%)1 (00.01%)137.8 MiB (01.26%)216.73.216.20795 867 (01.35%)1 (00.01%)91.1 MiB (00.83%)216.73.216.78

Misère.

#ClaudeBot #firewall

My (tech)blog has been 'visited' by #ClaudeBot, should I be honored or concerned (and should I take measurements)