John RegehrI'm fundamentally a tool builder, and LLM coding agents work one million times better if you give them good tools, and I wrote a thing about this
kbeyls@regehr Thanks for writing this! It feels close to how I started thinking of LLMs as "Kahneman system 1" thinkers (stochastic, intuitive, ...) . So you need "Kahneman system 2" tools (deterministic, analytical, controlled, ...) to control, verify or shape their output. It sounds like "executable oracles" are exactly one such kind of "Kahneman system 2" tool...
@kbeyls I like this way of thinking about it. isn't it hilarious how we have 75 years of SF showing us coldly logical AI and then when we get them they're touchy-feely and lazy?