This is precisely the right model to have for LLMs. They are optimizing for "is this a suitable reply to this entry text". And *the entire conversation* is fed back as the new entry text.

It is still remarkable that the result of that optimization can give something that it is occasionally useful, but it also provides a mental model that makes confabulations all the more likely.

https://mastodon.social/@mcnees/115172552726806174

#LLMs #StochasticParrots #SyntheticText #SyntheticTextExtruders