On Unit Tests

Published 03 June 2018

In software development it is very common to see two following extreme approaches to unit tests:

1. Tests are not needed at all

A lot of software has no tests at all, and it is not a secret. Some (sometimes quite popular) libraries, products, features, etc – it might lack tests, but still work. Why is it so?

Possible theories:

2. Everything should be covered with tests on 100%

Another extreme is to cover everything, so that you have 100% coverage (yes, even on getters/setters), and any smaller number means that build won’t even run. Any change in the codebase requires to rewrite some chunk of tests as well, which adds cost to maintainability.

Build Statuses

Let’s talk about build statuses and what does it actually mean:

So, build colour gives us only probability – green build is likely to work properly, and red build probably has some problems, but it is not really guaranteed: there are situations when you can decide to check something after having green build and actually deploying to production a red build.

Business Value

Very often you might hear that tests bring value to business, that they always pay off. This is a pretty arguable statement, which generalizes all possible cases.

In reality, we need to:

To summarize, even 100% coverage won’t give you full confidence, but will definitely require a lot of effort to write and maintain – because of that, rule “X%” coverage usually is not the best.

What Tests Give

Tests can’t prove that the program has no bugs, only that it has bugs (which also can be bugs in tests itself). They prove that some contract is correct, that certain function/modules/programs with some input will output certain result. It does not guarantee it for all input, and almost always we have no way to verify for all possible inputs: we don’t know what user will send us with POST request, or what we get from a REST service.

Does it mean that tests are not needed? Not really – more often they are needed than not. If there are a lot of people working on the same codebase, or you plan to do some refactoring, it’d be hard to understand that nothing has broken. Otherwise, if you write a small service which will stay untouched for years, probably tests are not so crucial. Or you work on something alone and don’t plan to have contributors – how often do you write unit tests for small utility functions or bash scripts?