Jason GormanAsk 12 developers and you'll get 12 different answers, but what do you think? What are the "units" being tested in "unit tests"?
Sarah Peper 🏳️⚧️🇺🇦@jasongorman I seriously dislike the terms unit and integration test. I wish I could come up with better terms for the handful of different test scenarios I regularly encounter. I think the existing terms are highly misleading
RenéWiersma@jay_peper @jasongorman There are deterministic and non-deterministic tests, any other classifications are bound to be subjective.
@renewiersma I tend to differentiate tests by what kind of environment I feel I need to include (i.e. with SQL involved or a HTTP server/client)
@jay_peper Those would be non-deterministic tests.
@renewiersma they happen to be very deterministic (as in: succeeding 100% of the time)
@jay_peper Do they still pass when your internet connection is down?
@jay_peper How does your code make a connection to a remote server or database when you have no internet connection?
@renewiersma there is no remote server. There are local servers (stood up by the tests or some test support)
@jay_peper I see. I still would call those tests non-deterministic, though. You have reduced the ways in which they could fail by hosting the remote instances locally, but in a production environment they might still fail, and the code calling those instances will still have to address the ways in which they could fail.
@renewiersma no. I'll add those fail scenarios as well. The tests are deterministic
@jay_peper Let's say you want to test a remote service timing out on a request, how would you set that up?
@renewiersma with a server that just waits a bit longer than the timeout (ideally a very short timeout, so the test does not become slow). WireMock for example has direct support for this
@jay_peper Excellent. I actually realized we aren't disagreeing. My point was that there are deterministic tests and non-deterministic tests. If you use reliable mocks or stubs to test your code, then those are, like you said, deterministic tests.
@renewiersma ok, then my misunderstanding probably was caused because I avoid any non-deterministic tests. And only as a manually-run test (i.e. testing if a third party provider actually behaves as their docs say)
@jay_peper I think that that is a sensible strategy.
@renewiersma (which just brings me back to: how else do you test if your framework actually does what you think it should do?)