Brian OkkenDo you think you should write more automated tests?
If so, why do you think that? What are your barriers to doing so? What would enough look like?
If not, what gives you that confidence?
I’d like to explore this topic more, so any RTs appreciated.
Blair@brianokken I always think I should have more tests. The key thing they do for me is give me confidence that I didn't break anything with my latest change.
Where I struggle the most is integration tests; I can get all my unit tests to pass, then I try to use my application and it's still broken.
I struggle to write tests for functions for things like plots and automated reports, since my success criteria keeps changing as I improve the outputs, and it's difficult to know what to test against
@brianokken made some progress over the last day: used the click.CliRunner to write some tests for several scripts that get packaged together. Make sure all plugins installed correctly, and make sure the scripts run and put their reports where I think they should.
1/2
@brianokken So I run this to make sure nothing is broken, then build the executable with pyinstaller. Lastly I run the same test cases as before, but this time using subprocess.Popen against the .exe I just built, just checking the exit code.
Saves a ton of time & headache in building and distributing my little tool 🤓
Antoine Alberti@blair @brianokken I stop using automated tests (i.e. checks) when I reach integration, i.e. infrastructure. Not only because they tend to be flaky. Because automated tests are good for known knowns, while infrastructure failures appear to mostly be unpredictable. In production you mostly face unpredicted errors, and that's a sign.
Instead, I prefer:
- observability and verifying invariants *in production*.
- domain segregation and bounded contexts in code design, to test less infrastructure
Instead, I prefer:
- observability and verifying invariants *in production*.
- domain segregation and bounded contexts in code design, to test less infrastructure