This feels like a much larger shift in the conversation than Crypto/NFT, and I think it's because the output of LLMs is specifically tuned to be compelling—and intelligence is sort of a side-effect, and an inconsistent one.

I know that the counter-argument to “but seventeen fingers!" or “a statistically significant proportion of ChatGPT's factual output is false" is “it will get more accurate with time”, but accuracy (speaking as somebody who works at a place that gets fairly caught up in its own obsessions with that notion) is a fairly squishy notion, and not just when you get to the margins of it.