Challenge
- Missing a way to track code coverage instantly instead of quarterly
- Needed to shift catching bugs toward the beginning of the development cycle instead of in production.
- Long and slow feedback loop
Goals/Use Case
- Maintain an 85% code coverage requirement
- To catch bugs when they’re “cheap” before they ever hit production.
- Reduce tech debt while improving existing systems.
Solution
- Shorter and faster feedback loop.
- Visibility for leadership: insights into real-time code coverage metrics.
- A way to track coverage trends over time
"As we’ve created this positive feedback loop of increased coverage, it’s a reward for adding more test coverage. Maybe you’re not refactoring something, you’re just adding some functionality. You get to increase test coverage while paying off tech debt."
Dave Bullock, CTO at UJET
UJET is the world’s first cloud contact center platform for smartphone-based customer experience (CX) management. UJET modernizes experiences and unifies the enterprise brand experience across sales, marketing, and support, eliminating the frustration of channel switching for consumers. Their platform is enterprise-grade and SOC2 Type II and HIPAA compliant.
UJET has a broad product portfolio with a rapidly expanding surface area in the customer experience space. UJET focuses on a balance of refactoring existing code and releasing new features to their customers while remaining heavily focused on testing and testing coverage. Their customers care deeply about security and compliance and use Codecov to help them accomplish their goals.
As UJET continues to grow its platform, Codecov has helped create a positive feedback loop of increased coverage so that engineers are rewarded for adding more test coverage. They can increase test coverage while paying off tech debt.
Q&A
Question: How would you describe the vision of UJET, what are you trying to accomplish for the customer experience for your customers?
Answer: Our goal is to make our customers have a better experience with their customers when they need support.
Question: What are some of your upcoming engineering priorities? Question: How do you (UJET) approach testing in general?
Answer: To create new features and new functionalities for our customers AND to reduce tech debt and to improve existing systems.
Question: How do you (UJET) approach testing in general?
Answer: As you build systems they become more and more complicated and if you don’t start out writing unit tests to cover individual units of functionality, it’s really difficult to go back after the fact. As we’re writing code, we’re writing tests that cover that code. We’re always looking for ways to improve our testing coverage.
Question: What were the solution steps you were considering at the time you chose Codecov?
Answer: To set up that answer I’d like to step back a bit. The way I think about bugs is that the farther left you catch the bug, the cheaper ie. When you create a ticket before any code is written, it’s almost free at that point. Then you get further down the development cycle where an engineer is looking at the code, writing code, and their unit test catches the bug…that’s still pretty cheap. If you then push it up and it’s ready for code review, it gets more expensive. So say you push code and it’s been in the wild for 5 years, that kills your company. If you implement tools far to the left of the development cycle, the very beginning, that thousand-dollar tool is worth a lot more because you’re catching more bugs there, where it’s cheap. A tool like Codecov puts your test right in a place where everyone is looking. I see every single PR that my team makes. I see red from the Codecov report and I know it’s not up to our standards. It makes it easy for people in leadership, so they don’t have to read every PR, I can see at a high level if what the team is doing is improving our test coverage. It keeps my engineers honest, they know there’s going to be a big red square if they write a bunch of code without coverage. I’m a big fan.
Question: Were there any other expectations or goals you had around the adoption of a code coverage solution?
Answer: One goal I have, a feature that I just haven’t turned on yet, is enforcing code coverage. There are some times that enforced code coverage blocks the pipeline in instances where code coverage just can’t be improved, and sometimes that’s a pain. We’ll turn it on at some point, we just haven’t yet.
Question: Did you consider any other solutions for code coverage to solve this problem for you?
Answer: Not really. Building it out internally is not something we were focused on. I used Codecov in the past [at a previous company]. I had such a great experience that I didn’t spend time looking at other options.
Question: How was the onboarding experience with Codecov and getting the team up to speed with it?
Answer: I brought some folks with me when moving to UJET that had experience with Codecov, so it was a pretty painless experience especially because we had been through it before.
Question: Was there any feedback from those on your team that didn’t have experience with Codecov before?
Answer: It was pretty easy to get the team on board, we already had coverage requirements, so having a tool that made it easier to visualize and track coverage was a win for everyone.
Question: What kind of wins have you seen so far with Codecov and your team/goals?
Answer: It gives us a shorter feedback loop. Before we were reviewing coverage on a quarterly basis and now it’s instantaneous. When code gets pushed it’s obvious what the coverage change has been. The increase in visibility and real-time feedback is a big win for us.
Question: Have products like Codecov helped that balance of refactoring existing code and releasing new features to your customers?
Answer: In a sense, yes. As we’ve created this positive feedback loop of increased coverage, it’s a reward for adding more test coverage. Maybe you’re not refactoring something, you’re just adding some functionality. You get to increase test coverage while paying off tech debt.
Question: How would you describe Codecov in one sentence?
Answer: Codecov is a powerful tool to enable engineers and their leaders to increase important test coverage across applications.
Question: What are you excited about moving forward?
Answer: We’re excited to be scaling and growing our platform, which provides unique challenges that are interesting and fun to solve.