Shipping untested code isn’t an option for Axle Health

Axle Health is a B2B workforce management software company focused on streamlining in-home healthcare. Their platform helps healthcare providers dispatch ambulances, schedule in-home visits, communicate with patients, and manage mobile clinical staff efficiently. Axle Health’s technology improves operational performance by 20-30%, ensuring clinicians reach patients quickly, a critical factor in healthcare, where every minute counts.

The Challenge

To keep up with their users, like many fast-growing startups, Axle Health found themselves shipping code, such as new features daily. However, without the proper quality checks, code would break in production, and the team had to scramble to fix it. “We were constantly reacting to broken code,” explained Ian Arko, Head of Engineering. “But that approach wasn’t sustainable anymore.”

This problem became even more urgent as the engineering team grew fivefold in six months. With more engineers came more complexity, and Ian knew they needed a stronger testing framework to keep everything on track. “I wanted our new engineers to see coverage indicators and understand that they were working on something that could potentially be a liability if it lacked tests,” he stated.

The challenge wasn’t just fixing bugs—it was about building a culture of quality and responsibility. For Ian, improving the code testing process was essential to keeping Axle Health’s high standards, especially in an industry where mistakes can have serious consequences. As Ian put it, “Providing reliable service is more than just meeting an SLA—it’s a moral obligation. Shipping untested code isn’t an option.”

Discovering Codecov

The Axle Health team wanted to improve software reliability and evaluated several tools based on criteria like ease of setup, cost, and seamless CI integration with automatic report additions to PRs. Since they were already using Sentry for error monitoring and found it essential, trying out Codecov was an easy decision.

“Sentry is the most structurally important piece of dev tooling we have right now.”

“The reliability of our software had been suffering, and it became clear that we needed a more structured approach to testing,” Ian reflected. With that, Arko took it upon himself to dive deeper into Codecov’s capabilities.

The turning point for Arko and the team at Axle Health came during a particularly intense period leading up to a big launch. With tight product deadlines coming up, Ian decided to set up Codecov late one night. “I was having a sleepless night, and I thought, ‘We need to do better.’ So I impulsively plugged Codecov into our system and hoped for the best,” he recounted. To Ian’s relief, the onboarding process was incredibly straightforward, taking less than an hour to get everything up and running. Ian found that after setting up Codecov, they were able to re-target and catch lines of code without sufficient test coverage. Adding Ian, “We use this to check that our test suites are appropriately designed and implemented.”

Implementation and Impact

With the ability to visualize code coverage metrics, the engineering team could quickly identify gaps in their testing strategies. “Seeing a function highlighted in red on a coverage report immediately tells developers what they need to address before merging their code,” Ian explained. This immediate feedback loop allowed for added testing before the occurrence of a production incident, decreasing the cost of development and debugging time.

“40% of our engineering capacity was going to fixing things that were broken. Codecov has been a critical tool in reducing that trend to around 10%.”

One of the biggest advantages of adding Codecov was the change it brought to the team. Code testing became a regular part of their process, and developers started taking more responsibility for their code. “We now use patch coverage metrics on pull requests, which has helped us track our progress in improving reliability,” Ian said.

Ian emphasized the importance of code readability in reinforcing this culture of quality further. “I believe code should read as well as plain English,” he stated. This helps individual engineers understand their work and allows future team members to understand the codebase quickly.

A key part of their approach was making sure tests matched the product requirements. Ian set up a rule for the team to name tests after the related product specs. “This way, we can easily see how the tests connect to our goals,” he said. This led to a clear process for both code quality and testing, making it easier for engineers to review and understand the code.

Continuous Improvement

With the engineering team likely to double in size again within the next couple of years, Ian recognizes the need for ongoing enhancements to their engineering processes. “It’s crucial for us to establish a solid foundation to ensure we can handle this growth without sacrificing quality,” Ian remarked.

“It’s not just about hitting a number; it’s about understanding how our coverage evolves as we grow.”

To accomplish this, the team has honed its testing strategies and expanded its use of Codecov. By integrating Sentry and Codecov into their QA process, they can conduct post-mortems for production events to identify and address any testing gaps, ensuring these gaps are systematically covered in the future. One area where they see potential for improvement is in monitoring code coverage over time. “We’ve upgraded to the pro tier of Codecov, giving us project-level coverage over time to ensure that our teams are executing in a way that leads to improvements over time. This allows us to track our coverage metrics more comprehensively,” Ian explained.

The Future of Code Testing with Codecov

Looking ahead, having reliable code testing processes with Codecov in place is crucial to keeping Axle Health’s logistics operations running smoothly and safely. “Given the critical nature of what we do—dispatching ambulances and ensuring that healthcare professionals can reach patients quickly—we can’t afford to cut corners,” Ian emphasized. The consequences of failure are too high, both in terms of patient outcomes and the company’s reputation.

Axle Health’s journey with Codecov illustrates the vital role of proactive testing in high-stakes industries like healthcare logistics. By integrating Codecov into their workflow, the engineering team not only improved operational reliability but also fostered a culture of accountability and quality. By partnering with Codecov, Axle Health hopes to continue to thrive in a challenging environment, ensuring they can deliver on their promise to enhance healthcare efficiency effectively and responsibly.

As Ian concluded, “This is not just about meeting service-level agreements; it’s about a commitment to doing what’s right for our clients and, ultimately, the patients they serve. With tools like Codecov, we are well-equipped to continue making a positive impact in the healthcare industry.”