Code coverage is a measure of how much of your code is run with a suite of automated tests. High code coverage, therefore, equates to more lines of code being tested.
However, there is a pitfall in equating high code coverage with a lack of bugs. Many developers and teams strive to achieve 100% code coverage, believing it to be the ultimate indicator of code quality and reliability. However, reaching this milestone can be time-consuming. There are many reasons to support the case against 100% coverage.
100% coverage considers only the overall codebase. What if an alternative approach can offer more practical benefits: focusing on achieving 100% patch coverage.
What are Project and Patch Coverage?
Before delving into the merits of patch coverage, let’s clarify what project and patch coverage mean in the context of code testing:
Project coverage refers to the total amount of code coverage across your entire codebase. It provides a comprehensive view of how well your codebase is covered by tests. Achieving 100% project coverage means that every line of code in your application is tested by your test suite.
Patch coverage is a more granular measure. It focuses specifically on the code changes introduced in a particular patch, pull request, or commit. Patch coverage calculates the percentage of your new code that is covered by tests. This metric is crucial when assessing the quality of your changes and ensuring that they don’t introduce regressions or bugs.
Striving for High Patch Coverage
As you can see, aiming for high patch coverage is a more effective strategy than demanding high project coverage. It is actionable and relevant to a developer’s workflow and can help to identify bugs before they merge changes.
It is important to recognize that demanding 100% project coverage on a team puts an undue burden and stress on a team. It can also shift valuable engineering resources to the wrong tasks. What it comes down to is that a developer should test their own code versus test all of the code.
Instead of the often elusive goal of achieving 100% project coverage, consider shifting your focus to achieving 100% patch coverage. This means that every line of code you introduce in your changes is being tested, reducing the chances of introducing defects and improving the overall quality of your codebase.
How to Measure Patch Coverage
Measuring patch coverage is easy.. Try using Codecov, which automatically measures and reports patch coverage as part of your CI workflow. You can integrate it into your development process and receive patch coverage information as a status check, making it easier to track your progress.
To get started, sign up for an account on Codecov and sync your repositories.
Using Patch Coverage in Your Workflow
Here’s how you can incorporate patch coverage into your development workflow:
- Write Code and Run Tests: As you write code for a new feature or fix a bug, accompany it with unit tests, integration tests, or other relevant testing techniques.
- Check Patch Coverage: Run your tests in your CI/CD, review the patch coverage report provided by tools like Codecov. This report will highlight any portions of your changes that lack test coverage. Notice
line 55is added, but no coverage was found for it.
- Address Coverage Gaps: Go back and write tests for the code that remains uncovered. This step ensures that your changes are thoroughly tested and less prone to issues.
While 100% project coverage may be a noble aspiration, it’s often impractical and resource-intensive. Shifting your focus to achieving 100% patch coverage provides a more pragmatic approach to improving code quality and reducing the risk of defects. By emphasizing the parts of the code that developers have direct control over, you can ensure that your changes are thoroughly tested, leading to more reliable and maintainable software.
If you have ideas or questions let us know what you think. And if you’re new to Codecov you can try it for free today or request a demo to get started.
