@marick My mental model of how LLMs work, specifically chat LLMs, is imagine that they are really good at diagramming sentences. And they have a training corpus of state_(n-1) and state_n pairs of diagrammed sentences, then they can make correlations between parts of the diagrammed sentences in each state, and those are the LLM weights. In other words, LLMs are weighted pattern matching (the sentence diagramming part) and use correlation to predict the next pattern. With large attention windows, this correlation takes on a bit of a conditional probability chain dynamic that captures some sequencing, some priors, etc to make the final prediction. In other words, when one is using LLM chat, the entire conversation is resubmitted as a prompt to a dumb machine for the next response. There is no intermediate/dynamic memory that is being nudged or anything resident for the "session".
With that framing in mind, then I think if the prompt specifies a specific pattern (callback method/style), there is a reasonable chance that it will be applied. But if there is no specific request for a pattern, it will be based on whatever latent patterns that existed in the corpus. People are hell bent on adding more "smarts" to LLMs, they will change/improve over time, and that will be a different machine than the one we have today.
My personal opinion is that LLMs are another tool and one should really understand their current utility and that means that one has to spend some time with it, much like any other tool/compiler/language/etc.
I've found I use LLMs primarily as search, eg give me code to do a binary search or something...but the other way I use them is "I'm struggling with X, what are 5 questions I should be asking that I'm not thinking of?" And then use that to build off. Any often, starting a new session with where I've gotten too, rather than all the baggage to clutter up the attention window, can make the results much more applicable.