One of the unresolved nagging challenges of engineering analysis software is setting and verifying code accuracy at a gross, usable level. Something like stating the code will calculate temperature within ยฑ5% or some fixed band of uncertainty or error. Formal software quality assurance (SQA) asks that expected or guaranteed accuracy be documented and verified but I'm always at a loss at how to actually do that. Very often codes make no firm guarantees or claims and as a code author, making an estimate or guarantee seems an intractable task. Formal error analysis is difficult enough for simple analytical expressions; add sophistacated numerics along with empirical correlations and the whole effort seems impossible.
This isn't a new problem - it doesn't seem to have gotten any easier or better than when I took my first numerical analysis class in the late 1980s. Since then I've discovered interval arithmetic and symbolic execution systems like KLEE
https://klee-se.org/ but these are still off in ths weeds of research and basically unknown to those creating production code, either to practitioners writing Fortran or Python or in any of the commonly available software tooling (VSCode, etc). I don't know if usable tooling even exists or where to look for it if it does - the communication pipeline from software engineering research to practice seems small to nonexistent.
It's more thsn a little frustrating that such a vast amount of money and attention is sluiced at making non-deterministic unverifiable systems marginally more believable (and nowhere near as trustable or useful as our current traditional deterministic systems) while we still seem to have no better way of answering fundamental questions or making reasonable claims about the accuracy of the codes we trust every day for performing mission- and safety-critical engineering analysis. Looking at a year's worth of CACM articles, it doesn't seem that making numerical software any more trustworthy is on anyone's radar and whatever work is being done in this area (I'm assuming there's some) is completely unknown to those of us working at the coal face of engineering software development and QA.
Am I missing something or is the situation as bleak as it appears?
#researchsoftwareengineering #rse