To mock or not to mock? Test doubles and how to use them

As software developers, we are taught to test our code to prevent bugs from reaching production. There are a wide variety of types of tests, like integration, end-to-end, and acceptance, but unit testing is the most used tool in our arsenal. As a result, we spend most of our time writing these kinds of tests which check each piece of our codebase.

Sometimes, however, our unit tests depend on internal or external dependencies. These dependencies might need a lot of time to run or have unwanted effects. Test doubles are used to replace some of those services with lightweight objects to help remove unnecessary dependencies or reproduce specific scenarios.

In this article, we’ll discuss test doubles for unit testing and how to implement them in your codebase. We’ll also briefly go over the effect mocking can have on our code coverage.

What are test doubles?

A test double is an object or procedure that represents simplified versions of their originals. In other words, they act as an empty shell of an object that has only a few properties or behaviors defined. A unit test usually focuses on a very specific function. But that function might need a complex input or data from an API call. A test double fills that need without actually running other code.

There are multiple types of test doubles that we discuss below, but it’s important to note that these definitions are often blend into each other depending on the framework. In my experience as a developer, it hasn’t yet been necessary to know the differences between these terms. However, having a common vocabulary to talk about these objects helps to articulate a test’s needs.

• Dummy
  A dummy acts as a placeholder object with no real attributes or behavior. You will likely find this double in a parameter list, where the parameter isn’t necessary. An example might be passing in null for an external service that is not used in that test.
• Stub
  A stub is an object that has some attributes set. These properties are often hard-coded and do not represent real data. They should not be able to respond to behavior requests (i.e. it should not respond to functions)
• Spy
  A spy is a more advanced version of a stub where the properties are often hard-coded but can also collect indirect output. It can, for example, capture information on how frequently an attribute has been called. It can also provide information about how that attribute may have changed.
• Mock
  A mock is a more sophisticated stub that will also respond to behavior requests. They can be really powerful by controlling the internals of a complex function that may not be easily testable. One example is that a mock might control the output of a database transaction or an API call.
• Fake
  A fake is a slightly different kind of object that fulfills all the qualities of the original object but has greatly diminished capabilities unsuitable for production. An example would be for a fake to always produce the same result for a given attribute. You might see these more frequently in test factories.

When should you use test doubles?

Let’s go over a few more scenarios where you might want to use a test double. This will help you see both when you have separate dependencies and when your unit tests are doing more work than they should.

1. Functions called use significant resources
   Functions can take a lot of memory or time to complete (especially if not written in an async manner). This makes it hard to test if your test suite takes too long to run. Using test doubles here can save you time in your CI/CD by replacing long-running functions with pre-determined behaviors.
2. Data is checked against a live server
   When hitting a real server whether owned internally or externally, there can be a lot of side effects. The server could be down, the API could have changed, or you could be rate-limited for making too many API calls. These situations should not prevent a unit test from passing. Using test doubles can ensure consistent test conditions.
3. Specific behaviors in your test environment are not possible
   Sometimes you need to create specific situations that might not make sense in your test runner. If certain preconditions need to be faked or a command already exists, test doubles can be used to trigger that situation. We’ll see an example of the latter in the following section.
4. Tests are written before implementation
   If your team is using test-driven development (TDD), you will frequently be writing tests before the actual implementation. Because a function hasn’t been written yet, you can use a stub or mock to stand in as placeholders with expected inputs and outputs.

Most programming languages have a test double framework. It’s worth noting again that different frameworks will bend the definitions of the test double types, or not use them at all.

Real Example of Test Doubles

Test doubles are extremely useful and quite prominent and good unit tests. I wanted to show a real-life example of their use.

At Codecov, we open-source our uploader which is used to upload code coverage reports. The uploader will run outside executable commands if provided with the right arguments. In this example, we’ll take a look at the part of the codebase that deals with running gcov.

import { isProgramInstalled, runExternalProgram } from "./util"

export async function generateGcovCoverageFiles(... gcovExecutable = 'gcov'): Promise {
    if (!isProgramInstalled(gcovExecutable)) {
        throw new Error(`${gcovExecutable} is not installed, cannot process files`)
    }
    …
}

gcov.ts

The snippet above has been simplified for the sake of simplicity. You’ll notice that in the function, we call isProgramInstalled which will check to see if gcov exists on the running system. On most of our test runners, however, gcov is already pre-installed. This makes it difficult to test the case where gcov is not installed.

it('should return an error when gcov is not installed', async () => {
    const spawnSync = td.replace(childProcess, 'spawnSync')
    td.when(spawnSync('gcov')).thenReturn({ error: "Command 'gcov' not found" })

    const projectRoot = process.cwd()
    await expect(generateGcovCoverageFiles(...).rejects.toThrowError(/gcov is not installed/)
})

gcov.test.ts

You can see in the above test, that we are mocking out the response from trying to run gcov. Now when our test runner tries to see if gcov exists, it will throw the error "Command 'gcov' not found".

Mocking and code coverage

It’s worth noting how test doubles can affect code coverage. It should be true that if you are mocking out a function, the function will not run. And if that is true, then if no other tests call the function, then it will not be covered. As a result, your overall code coverage will be lower.

This can be useful for two reasons. One, it helps to identify parts of your codebase that are still not covered with tests. Two, it should give you extra certainty that your test doubles are working as expected. I would recommend collecting coverage when setting up mocks and stubs for the first time to ensure that no unexpected lines are being run.

For more information about test doubles, I recommend Mocks Aren’t Stubs and Testing With Test Doubles?