Baldur Bjarnason"LLMs are here to stay," is like saying that the modem-based internet of the 90s was forever. It was a largely unusable piece-of-shit tech that was manifestly incapable of delivering beyond a fraction of what the dot-com bubble promised, but it prompted investment in fixing the one thing that prevented that promise: bandwidth.
I can't emphasise strongly enough how this ISN'T happening with LLMs. They are throwing all their money into building ON the broken tech and next to nothing in developing a replacement that has an actual chance at fulfilling the promise LLMs hint at, because there genuinely aren't any plausible pathways to get there from here
The modern AI Bubble is 100% a "modem speeds were good enough and the dot-com companies only failed because they didn't have enough funding to get them over the hump created by the government's regulatory fight with Microsoft" delusion.
Modem speeds are very much not enough and if broadband had proven to be infeasible for some unexpected technical reason, then the web would definitely have faded away into a hobbyist-oriented subculture in the style of ham radio.
tamas@baldur I was with you up until this point, but I disagree here. We have largely used up the bandwidth increase with much larger assets on the web unnecessarily. Sure, maybe webapps will render a spinner and progressbar instead of looking at the browser load everything, but it's still not loading that fast. Everything did not get as fast as the difference between the 56 kbps connections of modems, compared to 1-2 GBit fiber connections today (and their improvements in latency). The user expectation is still that you have to wait a bit for websites to load, even though we could do much better. Similarly, if connections remained as slow as they were then, we could still have achieved many of the advancements that make the web what it is today, except we would still have a lot of pressure to deliver small file sizes. Many of the advancements in that area are flat out ignored because we have a lot of bandwidth.
@tamas I think you've forgotten just how incredibly slow the modem internet was. It barely did images, period. Let alone streaming video or audio. Also screen resolutions needed to improve match the experience of print to remain competitive and that meant that images had to follow. Images sizes are not the biggest issue with the web today. The modem experience was much slower than you remember.
@baldur My point is that we have ignored many software advancements that could make better use of the low bandwidth. For example, real-time voice chat was not possible back then, and yet, speex was developed in the early 2000s that made this possible in as narrow as 8kbps bandwidth. Similarly, we were limited in our use of graphics, but image compression codecs also vastly improved since then. All I'm saying is, I don't think the web would have faded away to a hobbyist subculture even without the bandwidth developments.
Jeff GriggAnd it's not as if high speed internet did not exist at the time: The backbone and well connected sites had high bandwidth. It's just that most homes did not.
And it's not as if no homes had high bandwidth feeds: Many people did have "cable TV," which is high speed distribution to homes. And some had satellite (with high latency issues).
It was practically always an issue of cost and building out infrastructure.