A common request from Codecov users is the ability to see coverage metrics for sections of their code, such as groups of specific files or multiple folders aggregated together. While it has been possible to solve this problem using Flags for quite some time, solutions were unintuitive and required users to change the way they ran tests in their CI in order to upload to Codecov in specific and limiting ways.
Today we’re announcing Components, which is a much more flexible system to group coverage in your repository without having to modify your test setup or how you upload to Codecov.
In short, Components are defined in the Codecov YAML as groups of files that you specify. When Codecov processes coverage reports that you upload, it will look for those files in the individual coverage reports you upload, and aggregate their coverage together. This aggregated coverage can then be used in status checks and the Codecov comment, with support in the Codecov UI coming soon.
How to Use Components, a Straightforward Example
The best way to illustrate how Components can help you better understand your code coverage is to use an example. Consider, for example, you are writing a Blog application with a Comment feature, and you want to understand how well covered all the code that comprises the Comment feature may be. Historically with Codecov, this was a difficult problem to solve, because the Comment feature would likely have front-end and back-end code that may appear in one or many coverage reports. Before Components, a user would have to modify their CI to upload everything related to components individually in their own coverage reports and use a flag at upload time (e.g., -F CommentFeature) to group all code coverage related to the Comment Feature.
With Components, a user would only need to edit his or her YAML to indicate the file paths they’re interested in grouping together, like so:
component_management:
default_rules:
#... like Flags, these are default rules you want to apply to all components
individual_components:
- component_id: comment-component
name: CommentFeature
paths:
- app/Models/Comment.php
- app/Http/Controllers/CommentController.php
- app/Services/CommentService.php
- ...
Now the user only needs to upload as normal and they will receive status checks and information in the Codecov PR comment regarding the coverage of the Comment feature.
Other Uses for Components, Segmenting by Teams
Components can be used to group any concept or idea that is helpful to the team. While the above example showed how to group using a feature, you can use anything, such as grouping all coverage for which a specific team may be responsible.
Consider the example of grouping all the areas of responsibility for a frontend team into a single component:
component_management:
default_rules:
#... like Flags, these are default rules you want to apply to all components
individual_components:
- component_id: frontend-team
name: Frontend
paths:
- src/js/posts/show.js
- src/js/comments.js
- src/js/ui/*.js
- ...
Now the frontend team can understand the overall coverage of the aspects of the codebase they directly control.
Components vs Flags
Both Components and Flags in Codecov serve the purpose of isolating code coverage data, but they do it in slightly different ways which allow for even more customization. Here’s a breakdown of the key similarities and differences between Components and Flags.
Flags
- Flags are assigned to specific reports at the time of upload.
- Flags allow you to identify which lines of code still require testing.
- Flags can also be used to indicate the areas of the codebase that should be excluded from coverage analysis.
Components
- Different than flags, components are fully defined in the codecov.yml and aren’t supplied at upload time.
- Components offer more flexibility in terms of analyzing code coverage data because they can be applied to any coverage report regardless of when it was uploaded.
- Components allow developers to track coverage for individual files or directories within their codebase.
Currently, the resulting coverage is only displayed in a PR Comment, but we will continue to build out further support for this feature in the coming months. If you have feedback or suggestions, please don’t hesitate to reach out to us directly or submit a suggestion here.
