An odd proposal: "llms.txt" web standard using plain text summaries of webpages to fix that LLMs are really bad at searching web pages. What happened to that thing we used previously to search the internet? That seemed to do a pretty good job of searching text.

https://llmstxt.org

The /llms.txt file – llms-txt

A proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time.

llms-txt