Mastodawn

The @lwn web site is currently under the most intense scraper attack I have seen yet. 1.3M unique IP addresses within the last couple of hours, and it's not done yet. The work we have done on defenses appears to be paying off, though; the server is holding up reasonably well — so far.

...just in case anybody wonders why I have a rather dim view of the whole AI industry...

Show thread

Bart Veldhuizen 🚀1d ago

@corbet @lwn same on @blenderartists right now with up to 3.7M requests/hour. Looks more like a DDoS in our case than AI scraping though (which I suspected at first)

Show thread

Fazal Majid

@BartV @corbet @lwn @blenderartists
Who would have a motive to DDoS LWN or Blender? Other than Microsoft and Adobe, of course.

Most likely those IPs are from residential proxies so you can't do an easy filtering rule like "Block all IPs in AWS/GCP/Azure address spaces". There were revelations last week than half of all Smart TV apps include residential proxy SDKs.

Show thread

Bradley M. Kühn 🏳️‍🌈1d ago

@fazalmajid

At SFC, we've been seeing the primary culprit is .cn IP numbers and Zuckerberg.

& I can confirm User-Agent is fiction, at least from those parties. robots.txt of course ignored.

Cc: @BartV @corbet @lwn @blenderartists