The success of your application depends largely on how reliable your product is. With source code always changing, developers have to constantly make sure no bugs are being introduced to the application. As a result, a variety of testing tools have arisen over the years, to ensure stable software for products that need to be dependable at any time.
What Is Code Coverage?
Code coverage refers to how much of your code is being executed while your automated tests are running. This metric is calculated by special tools that add tracing calls inside the binaries of your code. This insight into your applications can inform future development, for example:
- Finding what parts of your code are covered by your tests. You can decide if a chunk of code needs more testing or not.
- Finding what parts of your code are not covered by your tests. You can create tests for parts of your app that might be hiding bugs.
- Remove dead code. You can easily decide if unexecuted parts of your app should stay or go.
Code coverage tools use one or more criteria to evaluate applications, but common metrics include:
- Function coverage: how many of your defined functions have been called
- Statement coverage: the number of statements that have been successfully executed in your program
- Path coverage: how many paths of the control structures in your program (if statements or loops) have been executed
- Condition coverage: how many Boolean expressions have been validated inside your program
- Line coverage: how many lines of your program have been executed
The overall value of code coverage in your app will be expressed as a percentage. Best practice is to establish what percentage of coverage is acceptable for your project before releasing any new changes in production.
So…What Is Codecov?
Speaking of code coverage, Codecov is an online tool that provides coverage metrics to companies across the world. As long as you have tests in your app and your coverage tool of choice prints data in a format compatible with Codecov, you can plug Codecov into your projects and see coverage metrics for whatever workflow you’re using.
Since there are so many code coverage tools you could choose for your projects and plug in to Codecov, I’ve compiled a quick reference guide for fifteen of the most-used coding languages and their respective coverage tools. For each tool mentioned, I’ve included a short brief, including pricing information, so you can start improving the quality of your product today.
Let’s begin.
Java
- JaCoCo: A totally free code coverage library for Java, with a great team behind it still maintaining the product after many years. A true must for all the Java developers out there.
- OpenClover: A sophisticated, free Java code coverage suite, with more than 20 metrics to describe the reliability of your product.
- JCov: A no-cost, open-source code coverage tool to plug into your Java environment and release the true power of coverage metrics.
JavaScript
- Istanbul: The most famous JS tool for code coverage. Supporting unit tests, server-side functional tests, and browser tests. And it’s all for free!
- Blanket: A simple, free-to-use JS library designed for both the web and the server-side of JavaScript.
- jscoverage: Written purely in JavaScript, this free tool is an ideal companion for verifying code coverage both on the browser and server-side of your application.
Ruby
- SimpleCov: A free code coverage tool for Ruby, providing a clean API to filter, group, merge, format, and display the results of every analysis you will perform.
- undercover: A free tool inspecting your Ruby files, warning you about changed methods, classes, and blocks that need tested.
- DeepCover: A free Ruby library that aims to be the best code coverage tool for the language, providing deep feedback on your entire codebase.
Python
- Coverage.py: A free tool for monitoring the coverage of your Python apps, monitoring every bit of your code to find what was executed and what was not.
- pytest-cov: A free language plug-in to produce a coverage report of your app.
- PyCharm’s integrated coverage tool: With the professional version of the PyCharm IDE, you have built-in support for performing coverage checks on your code with low runtime overhead. The tool runs $199 per year for every user.
Golang
- Golang’s built-in coverage reports: Since Go was designed with its tools in mind, you don’t need to look for any packages. Code coverage is built into the language with the test -cover command.
Native Mobile Development
- Swift: When it comes to this language, you can either go for a built-in option like the Xcode 12 coverage tool, or you can pick a free library like Slather, which will generate test coverage reports for Xcode projects and hook them into your CI.
- Kotlin: Similarly, you’ve got a few different options for a language like Kotlin. Its most used tools are Cobertura and the already mentioned JaCoCo.
C
- Gcov: A free, open-source code coverage analysis and statement-by-statement profiling tool.
- BullseyeCoverage: An advanced tool to improve the software in vital systems such as medical aerospace and defense. Today’s starting price is $800 per license with a yearly subscription.
- Coco: A complete suite of tools to analyze, report, and integrate code coverage in your application. Today’s starting price is 1350€ per license, with US dollar prices calculated for the current exchange rate.
C#
- Coverlet: A cross-platform code coverage framework for .NET, supporting line, branch, and method coverage, all for free.
- NCover: An incredible suite of coverage testing tools for .NET teams, offering innumerable features, coming in at a minimum of $658 per year for every user.
- dotCover: A .NET unit test runner and code coverage tool, purchasable as part of the dotUltimate package, starting at $399 per year per user.
C++
- Parasoft Jtest: A suite of tools for various testing purposes, providing support for code coverage. Pricing isn’t disclosed on their site; you’ll have to contact the team directly for a quote.
- Coco code coverage: the already mentioned Coco coverage test suite also offers great support for C++, without any modification on the price or features’ offer.
- Testwell CTC++: a product active for the last 25 years, with extremely reliable support for code coverage in all its shades. Pricing isn’t disclosed on their site; you’ll have to contact the team directly for a quote.
Perl
- Devel::Cover: The way to go when you need to implement code coverage in Perl, with a free, reliable, and all-in-one product.
Lua
- LuaCov: A simple and free coverage analyzer for all your Lua scripts.
Rust
- Tarpaulin: A free Rust library providing source code line coverage functionalities. The product is still in an early stage of development, yet it’s already proving a good choice for testing Rust applications.
- grcov: A free library collecting and aggregating code coverage information for all the Rust files in your project.
- kcov: A free BSD/Linux/OS X code coverage tester for compiled languages, as well as Python and Bash.
Dart
- test-coverage: A simple command line tool to collect code coverage information from your Dart tests.
- coverage: Another free command line tool providing collections of data related to your project’s code coverage.
- Built-in Flutter coverage command: If you’re using Flutter in your project, you can also generate a code coverage analysis by running
flutter test --coverage.
Scala
- scoverage: A free, Apache-licensed code coverage tool for Scala, offering statement and branch coverage.
- scct: A free Scala code coverage plug-in you can add into your workflow to get all the information about code coverage in your project.
- JaCoCo: After some tweaks, you can apply this tool for use in a Scala environment.
Clojure
- Cloverage: A simple and free plug-in you can integrate into all your Clojure projects to get amazing code coverage reports.
- Radagast: A simplistic free library for Clojure, reporting if you have any holes in your code coverage and where exactly they are.
- lein-cloverage: A lein plug-in for the Cloverage testing tool.
Conclusion
Testing tools exist for a specific reason: to offer developers full control over the quality of the products they release. Code coverage is a testing metric that can absolutely make a difference in your workflow, providing reliable information for verifying, scaling, and improving your application over time.
Don’t forget to check out Codecov to ensure your team enjoys a more straightforward path to success. All you need is a project with a suite of tests and a coverage tool of your choice (you can take inspiration from the list above). If your coverage tool outputs data in a format compatible with Codecov, you can start improving your product immediately, no matter your workflow, CI provider, or cloud platform!
Know of any cool code coverage tools that aren’t a part of this list? Leave a comment to help your peers discover them.
