Among the many things Doctorow gets wrong in That Post is this:

"It's not 'unethical' to scrape the web in order to create and analyze data-sets. That's just 'a search engine.'"

Apart from the fact that AI companies are particularly malicious in the way they scrape the web, I'd say we accept search engine scraping mostly on the premise that it's done for the benefit of the scraped sites. There's no such principle of mutual benefit in AI scraping — the AI company gets the value of the data scraped and you get bupkis at best, and possibly DDoS'd

@lrhodes

"I'd say we accept search engine scraping mostly on the premise that it's done for the benefit of the scraped sites"

I would qualify this somewhat by pointing out how, independent of AI, this acceptance ultimately led to Google benefiting from scraping websites at the latter's expense. The value proposition of Google indexing your site is it draws more visitors to your site who may not have known about it otherwise.

@Video_Game_King @lrhodes Let's not forget that this notion that one obviously wants to "draw visitors" or attract more traffic to one's website is bonkers: the owner of the website has to pay for the corresponding resource usage on the server (additional network traffic, CPU load, ...) and often doesn't get any direct benefit.

IOW, this notion itself is predicated on turning such accesses into money, e.g. via advertisement.