<rant>
Thanks so much to all the LLM companies out there who are hitting http endpoints all over the world!
Now my link-checkers fail because the http servers think I'm an LLM bot.
Why does nobody use these LLM models to answer the question
`Dear LLM, how can I harvest even more data for you without getting all server admins mad?`
Or perhaps the answer is
`I don't care how you do it, just give it to me...`
</rant>