Perhaps it would be useful to have some user-friendly generic DB client (is there some manner of schema description format spec that could be of use here?), such that the usual mostly-standard formats for dumps can be viewed by users that decide to torrent or otherwise download the dumps.

(No, the average user is not going to get much use out of even just SQLite, nevermind PostgreSQL dumps. I do not represent the majority.)

This would permit such sites to hard-limit the number of concurrent or total queries without a complete loss of availability.

Making said client compatible with updatable torrents (bep49 could be used for this too) would be even better (there is a dearth of such torrent clients as of currently).

#SearchEngines #Torrents #Crawling #Crawlers #DoS #WhyInlineTags