Jason GormanAsk 12 developers and you'll get 12 different answers, but what do you think? What are the "units" being tested in "unit tests"?
Sarah Peper 🏳️⚧️🇺🇦
Jez Higgins@jasongorman Pounds, shillings, _and_ pence?
@jasongorman I seriously dislike the terms unit and integration test. I wish I could come up with better terms for the handful of different test scenarios I regularly encounter. I think the existing terms are highly misleading
RenéWiersma@jay_peper @jasongorman There are deterministic and non-deterministic tests, any other classifications are bound to be subjective.
@renewiersma I tend to differentiate tests by what kind of environment I feel I need to include (i.e. with SQL involved or a HTTP server/client)
@jay_peper Those would be non-deterministic tests.
@renewiersma they happen to be very deterministic (as in: succeeding 100% of the time)
@jay_peper Do they still pass when your internet connection is down?
@jay_peper How does your code make a connection to a remote server or database when you have no internet connection?
@renewiersma there is no remote server. There are local servers (stood up by the tests or some test support)
@jay_peper I see. I still would call those tests non-deterministic, though. You have reduced the ways in which they could fail by hosting the remote instances locally, but in a production environment they might still fail, and the code calling those instances will still have to address the ways in which they could fail.
@renewiersma no. I'll add those fail scenarios as well. The tests are deterministic
@jay_peper Let's say you want to test a remote service timing out on a request, how would you set that up?
@renewiersma with a server that just waits a bit longer than the timeout (ideally a very short timeout, so the test does not become slow). WireMock for example has direct support for this
@jay_peper Excellent. I actually realized we aren't disagreeing. My point was that there are deterministic tests and non-deterministic tests. If you use reliable mocks or stubs to test your code, then those are, like you said, deterministic tests.
@renewiersma ok, then my misunderstanding probably was caused because I avoid any non-deterministic tests. And only as a manually-run test (i.e. testing if a third party provider actually behaves as their docs say)
@jay_peper I think that that is a sensible strategy.
@renewiersma (which just brings me back to: how else do you test if your framework actually does what you think it should do?)
Ed Blackburn@jay_peper @jasongorman YMMV but I find Jay Fields' **Solitary or Sociable **distinctions helpful but I'm a classicist who substitutes non-deterministic collaborators for deterministic fakes. Fowler's Unit Testing page covers the well I think. TL;DR I agree with you. Terms like unit and integration are misleading.
Jay Bazuzi@jasongorman this confusion is why GeePaw Hill started staying "microtests".
Jon Ayre@jasongorman I prefer "component test" because that's the key difference. You're testing individual components to ensure they behave as expected.
@JonHAyre Component as in "unit of software release"?
@jasongorman Component as in a self-contained piece of code that performs a generic task. E.g. a reusable class in object oriented development. Also components can contain other components. An analogy: a spark plug is a component, as is the engine into which it is fitted. Each can be tested in isolation to ensure it behaves as required. The car is the product into which the components are installed. Component tests make sense independently for the component as well as for the system as a whole.
@jasongorman hence the unit/component test is itself a system into which you fit the component to be tested, akin to a test rig in engineering terms.
@JonHAyre I thought maybe you meant "component" in the sense of "Component-Based Development"
@jasongorman My original background is in electronic engineering so I tend to use terms and design systems in the same way. I find that software development spends a lot of time reinventing concepts and evolving them into the approaches that engineers have practiced for centuries (agile is an example). There's no harm in that as it leads to a deeper understanding, but it would be better if it was more clearly recognised. Hence "unit" to me is "component".