@Wolven the gist of it is Google and co. got greedy and started showing answers to things instead of the links, going so far as to show full recipes and summaries of wikipedia articles, driving traffic away from those sites (which are largely ad-supported).

In response huge swaths of the web started requiring authentication to see anything.

Then social media platforms realized it was better for their data harvesting to make their stuff not externally searchable, and newer ones made stuff inaccessible by default (discord, insta, etc).

It turned the web into more giant walled gardens.

Then Google seems to have stopped investing in search so the SEO brigade ran wild. And every other search engine tries to be Google in one way or other.