How to Set Up Codecov with C++ and GitHub Actions

Tests are an important component of software development: they help prevent bugs and can act as documentation. One of the best ways to ensure your code quality stays high over time is to monitor your test coverage, through a measure known as code coverage. Monitoring code coverage is essential to maintaining code quality, and there are many tools out there that can assist with this.

OpenCppCoverage is an open-source C++ code coverage tool for Windows that can generate coverage reports locally, and then leverage Codecov GitHub Actions to integrate coverage reports in your CI pipeline.

In this tutorial, you will learn the basics of OpenCppCoverage and how to integrate it into your CI pipeline. If you’d like to skip ahead to the full code for the demo, it is available here.

Running Tests and Coverage Locally

To follow this tutorial, you need Windows Vista or higher and Microsoft Visual Studio 2008 or higher. You can clone the demo repository with the following command:

git clone https://github.com/imaculate/HelloCovDemo.git

Alternatively, you can proceed with the step-by-step instructions below.

1. Creating a Library

First, you will create a static library project from Visual Studio. The library will house computations below that can be performed with bit operations.

// HelloCov.h
bool is_power_of_two(int n);
int multiply_by_two(int n);
int divide_by_two(int n);

//HelloCov.cpp
#include "HelloCov.h"

bool is_power_of_two(int n)
{
    if (n < 0)
        return false;

    return (n & (n - 1)) == 0;
}

int multiply_by_two(int n)
{
    return n << 1;
}

int divide_by_two(int n)
{
    return n >> 1;
}

Ensure the library compiles successfully, then proceed to add tests.

2. Add Tests

In the same solution, add a Native Unit Test Project with reference to the HelloCov library. In a source file, add one test method for each function as follows:

#include "CppUnitTest.h"
#include "../HelloCov/HelloCov.h"

using namespace Microsoft::VisualStudio::CppUnitTestFramework;

namespace HelloCovTest
{
  TEST_CLASS(Test1)
  {
  public:

    TEST_METHOD(IsPowerOfTwo)
    {
      Assert::IsTrue(is_power_of_two(8));
    }

    TEST_METHOD(MultiplyByTwo)
    {
      Assert::AreEqual(14, multiply_by_two(7));
    }

    TEST_METHOD(DivideByTwo)
    {
      Assert::AreEqual(-4, divide_by_two(-7));
    }
  };
}

Verify that all the tests pass.

3. Install OpenCppCoverage

There are two ways to install it—using Chocolatey:

choco install opencppcoverage

Or using the interactive installer. Go to releases and download the latest release of the installer.

4. Generate Code Coverage Report

From the root directory of the solution, run the following command:

OpenCppCoverage --sources HelloCov -- "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" debug/HelloTest.dll

The path of vstest.console.exe depends on your version of Visual Studio, so update it accordingly. The report will be generated in the folder CodeCoverage-<Timestamp>. You can view it by opening index.html in a browser. On navigating to the report for HelloTest.dll > HelloCov.cpp, you will see the line coverage like below:

Configuring GitHub Actions

Coverage reports are most useful when making changes to your code. In this section, you will integrate Codecov into your workflow so that you are able to view reports on pull requests or pushes to your repo.

1. Repo

Create a public GitHub and push to it the solution above.

2. Sign Up for Codecov

If you don’t already have an account, head over to Codecov and sign up with your GitHub account. Select the repo you will be integrating. This tutorial uses a public repo for simplicity but if your repo is private you will be provided a token. Alternatively, you can add a repo by navigating to https://app.codecov.io/gh/<gh-username>/<repo-name>.

3. Setup GitHub Actions

GitHub Actions make it easy to automate software workflows. You’ll set up a workflow to essentially run OpenCppCoverage on a Windows machine, then upload the report to CodeCov. These actions are specified in a YAML file in the .github/workflows directory in your repo. Add the YAML file below to your workflows folder:

name: Upload CodeCov Report
on: [push, pull_request]
jobs:
  run:
    runs-on: windows-latest
    name: Build, Test , Upload Code Coverage Report
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
        with:
          fetch-depth: ‘2’
        id: checkout_code
      - name: Setup MSBuild and add to PATH
        uses: microsoft/setup-msbuild@v1.0.2
        id: setup_msbuild
      - name: Run MSBuild
        id: run_msbuild
        run: msbuild /p:Configuration=Debug /p:Platform=x86 HelloCovDemo.sln
      - name: Setup VSTest and add to PATH
        uses: darenm/Setup-VSTest@v1
        id: setup_vstest
      - name: Setup OpenCppCoverage and add to PATh
        id: setup_opencppcoverage
        run: |
          choco install OpenCppCoverage -y
          echo "C:\Program Files\OpenCppCoverage" >> $env:GITHUB_PATH
      - name: Generate Report
        id: generate_test_report
        shell: cmd
        run: OpenCppCoverage.exe --sources HelloCov --export_type cobertura:HelloCov.xml -- "vstest.console.exe" Debug\HelloTest.dll
      - name: Upload Report to Codecov
        uses: codecov/codecov-action@v2
        with:
          files: ./HelloCov.xml
          fail_ci_if_error: true
          functionalities: fix

The actions are described by their respective name properties. They mirror what we manually did to generate local reports i.e setup, build, test,k and coverage. Of importance are the last two actions that generate and upload the report. The OpenCppCoverage command is similar to the one used locally, and the export_type parameter is added to get the report in xml, one of the Codecov supported formats. The absolute path of vstest is not required since setup_vstest action added it to global PATH. When you push this change you can monitor the workflow from the Actions tab of the repo on GitHub.

4. View Codecov Report

If the workflow succeeds, at the end of logs for the upload action, there will be a link to the Codecov report like below:

Navigate to the link for a detailed report including the conspicuous coverage percentage. Under the Files tab, you can view line-by-line coverage for each file.

Under the Graphs tab, you can view graphical representations of your coverage, like the sunburst below:

Sunburst graphs enable you to navigate into project folders in order to discover files that lack coverage. The size of each slice is proportional to the number of coverage lines and the color indicates coverage. Click on a block to enter a folder and click the innermost circle to leave it.

5. Change Report

To view coverage reports as comments on pull requests, set up as follows:

• Add empty codecov.yml to the root of the repo.
• Grant access to the codecov bot to your account.To verify the setup, on a separate branch, write tests to cover the lines that have not been covered:

TEST_METHOD(IsNegativePowerOfTwo)
{
    Assert::IsFalse(is_power_of_two(-8));
}

Push and create a pull request from the branch. You should see a report like below on the pull request when the actions complete:

Observe that coverage increases as expected.

Conclusion

Code coverage is vital to ensuring that your software quality stays consistent over time. Now you should have a good grasp of how to integrate C++ code coverage reports in your CI workflows with Codecov in order to ensure you won’t fall behind on your coverage.

Although the tutorial was specifically for Windows, Codecov is platform-agnostic and can be configured not only with other C++ toolchains but also other languages and code-hosting platforms.