@wyatt8740 I too used to apply #HATEOAS: anything accessible through URLs in HTML documents was fair game. That was until I encountered a whole captchalogue of methods that websites use to deter scraping.

1. #RateLimit per IPv4 address and /24 and publish no AAAA record for IPv6, with 24-hour fail2ban penalties for exceeding a limit
2. #CAPTCHA
3. #LoginWall (what Twitter tried and still uses in a scaled-back form)