In 2016, I worked for a company that built banking software applications for mid-tier banks. There was a bug reported in production that caused havoc in the QA team—the user had added an emoji in the “account name” field in his iOS banking app, an action that wasn’t supported nor tested at the time. Around that time, there were multiple other issues reported only on the iOS mobile app. As it turned out, the QA team never got to testing the iOS mobile app because there wasn’t enough time allocated in the test plan for mobile app testing.
However, creating a simple test plan does not necessarily guarantee better results; creating a scalable software testing plan should be the ultimate goal. And, with an increase in the number of releases, creating a scalable test plan is necessary. In this guide, you will learn
How to create a test plan that scales
The importance of having a test plan that all members of the team use
How code coverage plays a role in enforcing the test plan
What is a Test Plan?
A software test plan is a vital document that lists out the testing strategy, scope, objectives, risks, and lists of features to be tested, as well as the testing schedule, staffing, and deliverables. You should treat a test plan as a carefully curated and analyzed plan to attack the testing phase for any software project.
There are many benefits of having a test plan. They serve as a guide throughout the testing process. A test plan helps define test strategy to achieve objectives and also provides functional coverage. It also serves as the best tool to estimate the overall effort and testing costs for a dedicated release or an entire project. It gives managers and leads good control and access to track the progress of testing in any project.
In general, the main goal of a test plan is to find defects/bugs in the code and ensure that the requirements are met as intended. Consider a scenario where there’s a team that consists of four software engineers, two test analysts or QA engineers, one product owner, and a QA manager. A good test plan incorporates the roles and responsibilities for everyone in the team. Code coverage and scope also affect the overall testing effort of a project.
Test Plan Basics
First, before diving deep into the plan itself, involve everyone on the team and get their suggestions based on their area of expertise.
- Evaluate the project’s requirements first.
- Analyze the pros and cons of the testing tools, as well as the test case management tool you will be using. You will have to keep in mind the risks, as well as training time, if you decide to use a new tool or methodology in the market.
- Before creating a test plan, identify the resources and their availability. Use this to estimate the time and allocate tasks accordingly.
- Design the test strategy. Define the scope of testing and the various types of testing that will be performed.
- Define what metrics will be used during the testing process. Establish pass/fail criteria for test cases.
- Specify the test environments that will be used for testing.
- Identify the deadlines and number of working days, and estimate the cost and budget.
While it’s important to create a well-thought-out plan, don’t spend too much time on the details. A test plan should provide an overview that effectively communicates your test strategy to the stakeholders.
Defining the Scope of a Test Plan
Start by listing out the features that will be tested during this process. In doing so, you must analyze and plan carefully to accommodate the most important features and platforms. Determining the scope of a test plan is crucial to minimizing costs and finding bugs in the early stages.
For example, if you are creating a test plan for regression testing in a banking mobile app, list out the main features that the team will test and on what platforms. For web testing, list different browsers like Chrome, Firefox, Safari, Edge, etc. For mobile app testing, list the different OS and phone models that will be used for regression testing.
You should also list out the risks associated with the scope of testing. Explain what exactly will be tested and what won’t, why a certain feature is out of scope, or why not testing a certain feature is less risky.
A good test plan describes the various types of testing that will be carried out, such as integration testing, regression testing, unit testing, smoke testing, etc.
Setting acceptance criteria for the tests will be the next step that you will have to plan, based on the type of tests you will run. Mention the definition of “done” for a test case. Determine when a test case can be marked as failed. Determine the process of creating bugs. You will have to state the exit criteria in order to mark the testing effort as completed. List out the number or percentage of test cases that need to pass in order to consider a feature or a release good enough for deployment in production or other environments in your CI/CD pipeline.
Planning for a Scalable Test Plan
Continuous integration and delivery demands faster releases, which, in turn, demands software quality and scalable testing efforts. There is no “one size fits all” when it comes to creating a scalable testing framework and test plan.
In order to keep up with new feature releases and the need to test them, a scalable, automated testing framework is necessary. To do so:
- Determine the critical features that need to be tested. Code coverage plays an important role here, as you need to decide if the resources in-house can accommodate this effort.
- Ensure you have the right infrastructure in place to incorporate different types and levels of tests.
- Manage tasks by allotting them to QA Engineers as per their skill set. For instance, API tests could be handled by a different team member than the one who manages the UI tests, and the manual test suite could be allotted to a different team member altogether. This ensures better accountability.
Code Coverage for a Scalable Test Plan
In order to have a scalable plan in place, you need to prioritize resources and their availability, which in turn relates to the cost of the overall project. Determine if you need to outsource resources—for example if your team can take care of automated regression tests for a release, outsource resources to tackle API tests or other features. This gives you more code coverage and fewer bugs reported post-production.
Complete, 100 percent test coverage is not practically possible, but a scalable test plan helps mitigate risks and increases code coverage to a greater extent. Having automation in place in the CI/CD pipeline improves test execution and accuracy, saves time and money, and provides time for exploratory testing and edge cases.
Making your test plan scalable does not mean you have to find a way to fit 50,000 test cases for one release. Rather, using the right set of tools and the right process will allow you to reap all the benefits of a well-structured and elaborate test infrastructure. Maintaining test cases is one of the major areas of failure for any test plan, and a scalable test plan takes into account the time taken to maintain the test cases in the future. Marking tests as retired, reusing tests, and organizing tests with solid documentation should be practiced in order to create scalable tests.
Establish a process to make sure tests are in place for enhancements and recent changes. Code coverage should be discussed and measured with both developers and test engineers. Decrease downtime, fix errors and issues reported in the early stages of testing and communicate effectively within a team. Code coverage shouldn’t be a target in the test plan—it should be viewed as an aid to improve standards and organize tests in an efficient way in the CI/CD pipeline.
Conclusion
Creating a well-defined, results-oriented test plan is not a one-person job. It is a result of collaboration and suggestions from the entire team on the project. All team members should participate in pointing out the benefits and risks involved with a test plan. All engineers should be held accountable to ensure good coding practices are followed and overall software quality is increased. Maintaining test cases and test plans should be the norm for every release, and test plans should change with changing team size and infrastructure.
Tracking results from the plan, discussing defects, getting feedback, and pondering on the success/failure of the strategy is as important as creating a test plan. Lack of collaboration from the entire team, insufficient automation, and outdated tools and technology are bound to slow down any testing effort. So, follow the steps mentioned above, and discuss and identify the gaps as soon as possible in the early stages of creating a test plan. Once you have gathered the information you need, work on the test strategy and generate a scalable test plan that you’ll commit to updating as the project changes.
