Jonathan B ✈️🪄👨🏻‍💻6d ago
My wife’s website keeps going down this evening. I’m not sure why (and typically am not in the easiest location to debug). Lots of connections in the logs from a user agent of “meta-external agent/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)” though… 🤔
Meta Web Crawlers - Sharing - Documentation - Meta for Developers

This page lists the User Agent (UA) strings that identify Meta’s most common web crawlers and what each of those crawlers are used for.

Meta for Developers
Show threadJonathan B ✈️🪄👨🏻‍💻6d ago

All of them are going to /shop/basket/?remove_item=…&_wpnonce=…&add-to-cart=…

Have turned on strict bot mode in WP Defender for now and they’re getting 403 responses.

Site hasn’t gone down since… 🤔

Show threadBen C
@jmb yeah, the AI crawlers are a constant nightmare and they love to hit up every possible combination of query parameters.
Apr 2 at 10:07pmWeb