mhoyeMar 17

"we caught one" god. Luxurious.

Once again, I strongly advise you to set up fail2ban so that anyone you serve a 404 catches at least a full day ban, and if you don't care about talking to other people's services, do your best to fully block the IP ranges associated with all the major hosting companies.

https://exple.tive.org/blarg/2025/10/21/raised-shields/

https://infosec.exchange/@foobardevs/116246141464905287

Show threadAndrew ZonenbergMar 17

@mhoye a single 404 is very aggressive, all it takes is one fatfingered link in a page pointing to your site and you're banning visitors en masse.

after 5-10 404s in a row for different URLs, different story.

Show threadmhoyeMar 17
@azonenberg Nobody is fat-fingering their way to .env or backdoor .asp files.
Show threadAndrew ZonenbergMar 17

@mhoye yes i'm all for having specific poison URLs that trigger an immediate ban.

But "immediate ban on any 404 whatsoever" seems heavy handed.

Show threadmhoyeMar 17
@azonenberg Read your logs, tell me what you see. Nobody's typing out URLs anymore.
Show threadIrenes (many)Mar 17

@mhoye @azonenberg .... we do personally do that

we're prepared to accept that we do not exist, in a statistical sense. that is true in SO many ways

but we hesitate to put into place a rule which would lock ourselves out. we'd at least apply a threshold of a few 404s over a window of time, not just a single one

Show threadIrenes (many)Mar 17
@mhoye @azonenberg another thing we've seen in logs recently is automated requests ie. for favicons that never existed, or for stylesheets that have moved, which happen as part of every successful pageload and which generate 404s. so there needs to be some way of dealing with that, too
Show threadDon MartiMar 18

@ireneista @mhoye @azonenberg I'm seeing bots guessing URLs that look like something I might have written, too. There is no `/articles/removing-trackers/floc-affinity` on my site. Maybe generating broken links for a user who asked for a summary?

And there are always a lot of 404s for stuff like `/page-title/favicon.ico` after my site gets on "Hacker News" -- so many crawler scripts (at varying levels of working) get links from there

Show threadmhoyeMar 18
@dmarti @ireneista @azonenberg I've created favicons despite never caring about that at all, because so many clients that appear to be legitimate humans doing legitimate human things reach for them automatically, Including a bunch of feed readers and aggregators.
Show threadDon Marti
@mhoye @ireneista @azonenberg Yes, I also made them. Found some good sources of freely licensed images and a handy favicon maker site. https://blog.zgp.org/favicon/
Hey kids, favicon!

How to use freely licensed graphics and a helpful free favicon generator to make all the icons needed for a web site.

Mar 18 at 11:50pmWeb