LOL this is not how you deal with "data scraping," this is how you deal with a catastrophic loss of system capacity.
You limit data scraping by blocking things a human user couldn't do, like access a thousand posts a minute. This is aimed directly at reducing normal activity across the whole system.
That's what I've been saying. Seems like they just can't afford as much bandwidth anymore. They weren't paying their hosting contracts, and their revenue stream shrunk a lot, so they probably just had to negotiate on those overdue contracts and negotiated out less bandwidth.
@Biohazardous @maxkennerly it’s not a question of bandwidth, it’s a question of “a tweet costs this much compute”
Much of Twitter was on their own hardware, and they shut down one of their data centers a while back. so there’s just less system capacity overall to handle the cascading failures
and tuning those systems such that they don’t fall over is no longer possible with their poor understanding of the systems they inherited from the people that management drove out (including many trans/queer SREs!)
(note: not one of said *ex-twitter* SREs; this is based on my understanding of conversations with ex-tw SREs, plus personal experience with how complex distributed systems fail)
Max Kennerly
Biohazard
kouhai, of the health issues