Grant SlaterDear AI Companies, instead of sneakily scraping OpenStreetMap.org, how about a tiny $10,000 donation? We'll even throw in a shiny new download link to our entire planet's geo data! Who knew it was that easy? Start here: https://supporting.openstreetmap.org/donate/ #win #ai #bots #OpenStreetMap 🌍 🤖 🤑
Jes Moved to @@Firefishy how about OSM just blanket blocks AI crawlers
@Jessica Unfortunately not easy. User-Agents are often library-defaults (eg: python-requests/2.26.0) or faked (Browsers or "googlebot" or similar). Honouring robots.txt treated as optional. When blocked they change IP or User-Agent.
@Firefishy most AI crawlers do have their own user-agent so the big offenders can be blocked, like Bytespyder and such.
@Firefishy I'm not saying block every single IP that has ever datamined you for AI, I'm saying block the ones that truly cause damage, or you could set a really generous rate limit that only people who would cause trouble would go over.