Are there useful or interesting ways to use LLMs other than prompting them? I feel like compressing all the text in the world via a hierarchically structured statistical model is probably useful, but that we're using it in a way that is unlikely to do what we'd hope.

Like everyone I'm impressed and amazed by what they can do, but also very frequently nonplussed by their stupidity. The amazing things convince me there's something important here, but the stupidity seems to have a consistent character that makes me think we're using them in a non-optimal way.

For example, I would expect them to be good at gathering text from their training data that is talking about the same thing in two different ways. This seems like it would be very helpful for synthesizing views on a complex question, but I think not.

I think that part of synthesizing multiple views is building a mental model of the underlying meaning, finding the points of disagreement, and putting that into a new framing. This feels like an inherently back and forth process that LLMs can't do by their very structure.

"Reasoning" models with chain of thought get a little way towards this but they feel like an overkill solution that also isn't enough to really address it. But I have to admit I don't know much about their internals and I've never really had the chance to use them myself.

Another aspect is that I'm not sure it's possible to train models to produce truth, in some sense. I feel like we learn this by living in the world and trying to use the imperfect pieces of knowledge and skills we have to achieve stuff. Without that connection, can it go beyond compression?

So that's why I'm wondering if there is another way to use LLMs that more clearly makes use of the fact that they're an incredible compression scheme? Search seems like one possibility but maintaining sources would undermine their compressing role I guess.

Maybe generating good keywords and alternative phrases that people use when talking about something, that could be the starting point of a literature search? Has anyone tried using them in this way just via prompting? Or maybe there's another way to use the core model without prompting?