While the Codecov team at Sentry is busy helping developers analyze code coverage, we’re also finding and mitigating vulnerabilities in our code base dependencies to protect our customers. Codecov uses GitHub, therefore we also use Dependabot to alert on vulnerabilities.
To make sure we’re working as efficiently as possible, we automate how we triage vulnerability issues as much as possible with:
We can pull dependency vulnerability alerts directly from GitHub’s GraphQL API. To do this, we utilize the following Python function containing our GraphQL query:
def get_repos(params, client):
"""Function to fetch repo's from Github 20 at a time"""
query = gql("""query RepoCount($cursor: String) {
organization(login:"codecov") {
repositories(first:20, after: $cursor) {
pageInfo {
hasNextPage
endCursor
}
nodes {
name
vulnerabilityAlerts(first:100, states:OPEN) {
nodes {
securityAdvisory {
severity
summary
description
permalink
updatedAt
vulnerabilities(first:100) {
nodes {
package {
ecosystem
name
} } } } } } } } } }""")
response = client.execute(query, params)
return responseThis queries 20 repos at a time, and up to 100 OPEN vulnerabilities, from our GitHub organization. Due to limitations on the amount of data returned in an API call, pagination is required and that’s why we limit the query to 20 repos.
Next, we pull the data from the API with the following:
### GraphQL objects
transport = RequestsHTTPTransport(
url="https://api.github.com/graphql",
headers={'Authorization': 'token ' + GH_TOKEN}
)
client = Client(transport=transport)
### Call get_repos using pagination
vuln_data_list = []
endCursor = None
while True:
params = {"cursor": endCursor}
response = get_repos(params, client)
repos = response['organization']['repositories']['nodes']
vuln_data_list = vuln_data_list + repos
hasNextPage = response['organization']['repositories']['pageInfo']['hasNextPage']
endCursor = response['organization']['repositories']['pageInfo']['endCursor']
if not hasNextPage:
break
The resulting data is parsed, filtered based upon the desired severities (e.g. Critical, High, Moderate only), and organized into singular issues we use to create tickets in Jira and assign to a Dependabot Security Vulnerability epic. The tickets are programmatically assigned due dates based on their severity value and our vulnerability SLA timelines. For example, a High severity vulnerability has an SLA of 30 days. So if a ticket is created today, the due date assigned is 30 days from now.
These tickets are then triaged by our engineering lead and assigned to specific developers based upon their team assignment. This allows our security engineers to not have to triage Dependabot alerts daily in the GitHub UI, and for developers to only have to work with their normal tooling (GitHub PR’s and Jira tickets).
Let’s see an example of what this looks like (unmitigated repo and package names redacted):
Here we see two tickets that were CLOSED (risk accepted on some example code related to Laravel/PHP), three tickets resolved and moved to DONE after fixing, and three vulnerabilities with a SELECTED FOR DEVELOPMENT status assigned to a specific developer on the current sprint.
If we drill into a ticket, we’ll see how we set the due date, tags, and other relevant data from the actual Dependabot alert data pulled via the API (assignee name and repo name redacted):
For this specific ticket, we can see the vulnerability data scraped from the API output, including the repo name, the URL to the Dependabot alert in our GitHub org instance, the severity, a summary description as well as a longer description, the URL to the GitHub advisory for the vulnerability, and the package/ecosystem applicable to the alert.
We also assigned the labels Dependabot and Vulnerability, the Priority, Severity (taken from the Dependabot vulnerability alert itself, and set the due date at ticket creation time plus 30 days. We can also see the epic name assigned, and the fact that it was assigned to TS, and finally it’s in a DONE state.
To aid in automation and efficiency, our script runs daily and queries the Dependabot vulnerabilities. If a new vulnerability is found and no Jira ticket has been created, the script creates it. If a ticket for an existing vulnerability exists and the ticket is in a state of BACKLOG or SELECTED FOR DEVELOPMENT and the vulnerability in GitHub is still open, nothing happens.
However, if the ticket was marked DONE or CLOSED, and the vulnerability reoccurs (for whatever reason), the script opens a new ticket. This accounts for when an older dependency is merged into (or back into) the codebase that shouldn’t have been as it contains a vulnerability.
Curious about other security controls and/or security compliance at Codecov by Sentry? See our security page where we talk about more of our security controls and our SOC2 Type II certification, including our SOC3 report.
