terry matulaKind of wild to see all the Mastodon/Fediverse instances that immediately ping my site after posting a link or even replying to a post with a link.
And then right after that, all the "exploit" pings.
I learned about Team Cymru, and their IP services. Pretty cool, and "free". Using PHP's dns_get_record, I'm able to make a super quick query for an ASN record, and block the ones I don't want.
I'm no security expert, but this seems to be working well.
This whole project is just pleasant to work on... no frameworks or database or external dependencies. Just pure PHP, with a smattering of CSS and JS.
bob.php 
@terry not enough context for me to latch on to what you're doing with it. for example... can i automate blocking the ai bots?
@bobmagicii it's a bit janky, but I pass the IP to the Team Cymru service to get the ASN. And I'm storing the asn to a blocklist.json file.
My biggest issue was with Tencent Chinese servers, with multiple IPs but they all had the same ASN. This way I can block ALL of them with a single value instead of hundreds/thousands of different IPs.
For AI bots that have "good" user agents, I have a separate system that parses the ua and blocks based on that. Most big AI companies are good about it.
@terry ok that is about what i was thinking was going on. i found user agent on the bots not trustworthy because after i made fun of them for sending clear user agents, a month later, i found their traffic sending normal browser agents from the same data centers.
like i don't see Anthropic Claude anymore but i still see traffic from their datacenter using a 2 year old iphone ua.
may be automated responses to my blocking.
but if i can get an authourity to just tell me "yeah, that's them"
@bobmagicii I could just be lucky, and looking at logs I don't see any "Claude" ua. I do see "amazonbot" and "GPTBot" among others. If Anthropic is hitting my site, they're not slamming it at least.
Looking at search referrers, it's interesting I don't see Google. Just DDG and Bing. Not sure if that's just because of the type of people who might search for obscure Houston bands, or Gemini screwing up search.
@terry the exploit pings with the ldap data and stuff stopped after a day. but the mastodons are still constantly hitting me too. one of the exploiters is run by some guy thinking he is doing community service for the fediverse.
this is just the past 60 seconds lol. instances that i know already hit me yesterday will hit today, so the servers don't seem to cache well.
i'd sure love for some conversions from this lol. oh wait its just servers talking to servers nobody is actually seeing it lol.
