@metarecursive Yes, I agree that they have little to do with AGI, except perhaps as a single component of a larger architecture, much like the role that brocca's area plays for us.
But the fact that they can offer a convincing simulacra of language *at all* is endlessly fascinating to me as a scientist. From the standpoint of philosophy of language, this gives us direct evidence that it's possible to separate language and reasoning.
What's funny is that many people conjectured that you could have reasoning without language, but pre-LLMs I don't think anyone was seriously considering that you could have language without reasoning. And yet here we are. I haven't followed academic philosophy for a while, but I so wish to see it start tackling what this means for our conception of language.
And I agree that as it stands, this technology is deeply tied to capital. That's why I think it's important to have these conversations now, before we get locked into this state of affairs for good.