We all know time is money, especially in software development – our latest feature Automated Test Selection was built to help you optimize your test runs, and reduce testing time, and CI costs. Now you can pinpoint the exact tests that need to be executed based on the changes you’ve made to your code – automatically.
TL;DR: Ready to get started in Python?
1.) Review the docs for setting up Automated Test Selection.
2.) Get a Codecov account if you don’t already have one.
Not in Python and want to get notified when we introduce more languages? Join the waitlist.
Want to hear more? Keep reading:
What is Automated Test Selection?
As codebases and test suites mature, an inevitable challenge emerges: it takes more and more time to run tests. This natural side effect can have detrimental consequences on developer productivity. Waiting for extensive test suites to complete can be time-consuming and frustrating. Additionally, as test time increases, many teams attempt to solve the problem via parallelization in CI, which comes with its own cost and complexity issues. Automated Test Selection can help with these problems.
Automated Test Selection pinpoints essential tests in response to code changes to not only accelerate feedback but also conserve time and money spent waiting on tests to run. Automated Test Selection enhances efficiency in your development process, empowering you to tackle larger codebases and test suites without sacrificing productivity. With Automated Test Selection, our testing run times are up to 75% faster.
How Does Automated Test Selection Help?
Automated Test Selection plays a crucial role in cost-saving in software development by improving test coverage and codebase scalability.
- Test Coverage: Now you can write tests without wondering if your test suite is going to take forever to run, you’ll only ever have to run the tests that matter, meaning you can cover more code AND spend less time watching a CI run, the best of all worlds.
- Scalability: Selecting and running only necessary tests ensures cost-effective testing for both small and large codebases. The larger your codebase grows, the more time and money you’ll save.
So, how does it work?
For every PR, Automated Test Selection analyzes the code changes and identifies the tests that must be called to exercise the code change. By focusing only on the tests you need, Codecov now allows you to cut through the noise of unnecessary tests and allows CI runs to complete more quickly, increasing developer productivity and reducing cost.
Automated Test Selection consists of three key components that work together to ensure thorough testing:
- Test Analysis: It examines your codebase (we never store the code itself) and identifies the sections of code that must be tested.
- Label Analysis: Building upon test analysis, label analysis gets a list of test names (labels) and combines it with test analysis results and the commit’s diff. This generates a list of tests to be run, saving time and resources.
- Codecov CLI: The Codecov Command Line Interface (CLI) serves as the user interface to interact with both test and label analysis features. Typically integrated into your Continuous Integration (CI) pipeline, the CLI automates the process, making it an efficient part of your development workflow.
Automated Test Selection optimizes the testing process by intelligently selecting only the necessary tests to evaluate a commit’s changes. To learn more about how it works and how to set it up, check out our docs.
Automated Test Selection currently supports Python. We’ll add support for JavaScript in the near future. Join the waitlist to be notified about more updates as we expand the availability of this feature and support for languages and continue to build out more capabilities.
If you have ideas or questions about Automated Test Selection, create an issue and let us what you think. And if you’re new to Codecov you can try it for free today or request a demo to get started.
