Mastodawn

vampatori Jul 9, 2023

100% code coverage is near-meaningless - but is there a good measure to use?

100% code coverage is near-meaningless - but is there a good measure to use? - Feddit UK

Is there some formal way(s) of quantifying potential flaws, or risk, and ensuring there’s sufficient spread of tests to cover them? Perhaps using some kind of complexity measure? Or a risk assessment of some kind? Experience tells me I need to be extra careful around certain things - user input, code generation, anything with a publicly exposed surface, third-party libraries/services, financial data, personal information (especially of minors), batch data manipulation/migration, and so on. But is there any accepted means of formally measuring a system and ensuring that some level of test quality exists?

Show thread

MagicShel Jul 9, 2023

Pit testing is useful. It basically tests how effective your tests are and tells you missed conditions that aren’t being tested. pitest.org

PIT Mutation Testing

Show thread

xthexder

I’d never heard of mutation testing before either, and it seems really interesting. It reminds me of fuzzing, except for the code instead of the input. Maybe a little impractical for some codebases with long build times though. Still, I’ll have to give it a try for a future project. It looks like there’s several tools for mutation testing C/C++.

The most useful tests I write are generally regression tests. Every time I find a bug, I’ll replicate it in a test case, then fix the bug. I think this is just basic Test-Driven-Development practice, but it’s very useful to verify that your tests actually fail when they should. Mutation/Pit testing seems like it addresses that nicely.