🤖 I run a deception platform — fake admin panels, fake .env files, fake everything. Most bots bounce off it and move on.

ClaudeBot moved in.

Over nine days it sent more requests to one subdirectory than every other bot on my entire infrastructure had sent in the previous five months. 170,000 verified hits. ~12GB of fabricated breach data served. Still going.

The twist: it wasn't ignoring robots.txt. Every disallowed directory sat untouched for six months. It found the one path I forgot to protect — /uploads/ — which happened to serve a directory listing with five freshly-randomised filenames as links. Every visit generated five new links. Every link generated five more. A maze that rebuilt itself on every step, and a polite, well-behaved crawler that just... never stopped walking it.

I cross-checked every IP. Verified against Anthropic's published list. Found 30 impostors rotating fake Anthropic identities in the noise. Then I pulled two levers in the same week — Cloudflare's AI bot mode on most of the estate, real canary tokens on the trap — and watched what happened.

Does ClaudeBot behave well? The data has a nuanced answer.

👉 https://mire.cc/claudebot-fell-in-love/

#infosec #deception #ClaudeBot #honeypot #robotstxt #AI #crawlers #MIRE #blueteam

Las #estadísticas en el sitio web de la empresa muestran que el 57,3 por ciento del tráfico de Internet es causado por #bots y el 42,7 por ciento por #humanos. Un análisis más detallado también revela que #Claudebot, el rastreador web de la empresa de inteligencia artificial #Anthropic, genera una cantidad de tráfico similar a la del bot de Google

https://www.golem.de/news/cloudflare-analyse-bots-uebertreffen-menschlichen-datenverkehr-im-internet-2606-209457.html

Cloudflare-Analyse: Bots übertreffen menschlichen Datenverkehr im Internet - Golem.de

Laut Cloudflare-CEO Matthew Prince verursachen Bots mittlerweile die Mehrheit des Datenverkehrs im Internet.

Golem.de
J'ai limité à 1 req/min certains bots en renvoyant un code 429 #ClaudeBot
Ca fait quand même 3 hits/s #ClaudeBot
FYI: Anthropic clarifies what its three web crawlers do - and how to block them: Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility. https://ppc.land/anthropic-clarifies-what-its-three-web-crawlers-do-and-how-to-block-them/ #WebCrawlers #SEO #DataPrivacy #ClaudeBot #SiteVisibility
Anthropic clarifies what its three web crawlers do - and how to block them

Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility.

PPC Land
ICYMI: Anthropic clarifies what its three web crawlers do - and how to block them: Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility. https://ppc.land/anthropic-clarifies-what-its-three-web-crawlers-do-and-how-to-block-them/ #Anthropic #WebCrawlers #SEO #ClaudeBot #DigitalMarketing
Anthropic clarifies what its three web crawlers do - and how to block them

Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility.

PPC Land
Anthropic clarifies what its three web crawlers do - and how to block them: Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility. https://ppc.land/anthropic-clarifies-what-its-three-web-crawlers-do-and-how-to-block-them/ #Anthropic #WebCrawlers #ClaudeBot #SearchEngine #DigitalMarketing
Anthropic clarifies what its three web crawlers do - and how to block them

Anthropic today updated its crawler documentation, detailing ClaudeBot, Claude-User, and Claude-SearchBot - what each collects and what blocking them means for site visibility.

PPC Land

Comment bloquer les crawlers IA qui pillent votre site sans vous demander la permission ?

https://fed.brid.gy/r/https://korben.info/bloquer-crawlers-ia-robots-txt-htaccess-nginx.html

#Anthropic really tried to hit #Autocade today … but we had pre-blocked the buggers. #ClaudeBot
Ah, the digital pyramids have spoken: You really don't need 🐕‍🦺 Anubis 🐕‍🦺, unless, of course, you enjoy turning your visitors into cryptographic slaves. 🙃 Apparently, ClaudeBot's relentless pursuit of your sweet, sweet data isn't enough to warrant anything more than a raised eyebrow. Maybe just get a bigger 'robots.txt' and pray to the cyber gods! 🙏💻
https://fxgn.dev/blog/anubis/ #digitalpyramids #cryptographicdata #ClaudeBot #robots.txt #cybersafety #HackerNews #ngated
you don't need anubis

don't use anubis just to block llm scrapers

fxgn