Dear AI Companies, instead of sneakily scraping OpenStreetMap.org, how about a tiny $10,000 donation? We'll even throw in a shiny new download link to our entire planet's geo data! Who knew it was that easy? Start here: https://supporting.openstreetmap.org/donate/ #win #ai #bots #OpenStreetMap 🌍 🤖 🤑
Donate – OpenStreetMap Foundation

@[email protected] Scraping OpenStreetMap. 🤦 First time I see that combination of words.
@bart Unfortunately it is extremely common. Sometimes 100s of req/s hitting expensive API endpoints. Multiple IPs, faked UAs.
@Firefishy @bart why scrap when there is planet.osm available?
@wikiyu @bart Sssssh! Don't give away our secret s̶a̶u̶c̶e̶ source. Honestly, no idea. The full planet.osm data would be a lot easier to use than painfully slow scraped data.

@Firefishy @bart
Exactly, there are also increments with changes... and parts per continents and...
Oh god i cannot imagine ANY reason to scrap it from website.

Or maybe... it was shIT answer of ai for "download whole osm"

@wikiyu @Firefishy @[email protected] probably asked an ai for a program to get all the data...
@Firefishy @wikiyu @bart that's expecting people behind those companies to be, you know, actually competent
@SRAZKVT @Firefishy @bart you won that conversation ;-)
@wikiyu @Firefishy @bart Because that’s what CoPilot told their junior dev to do when they asked for boilerplate code.
Probably.
@Firefishy @bart Similar thing to what's happening to @readthedocs then https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse/ (although luckily they were not being subject to faked UAs, AFAIK...)
AI crawlers need to be more respectful

We talk a bit about the AI crawler abuse we are seeing at Read the Docs, and warn that this behavior is not sustainable.

Read the Docs
@Firefishy @bart Interesting challenge. Care to share how you overcome these nuisances? Do you block them by headers or just let them wreck havoc?