As is the case for most undertakings, the plan is the crucial part of the management of the testing process. The components of a good test plan are
- Objectives. The objectives of each testing phase must be defined.
- Completion criteria. Criteria specifying when each testing phase will be judged to be complete must be specified. This matter is discussed in the next section.
- Schedules. Calendar-time schedules are needed for each phase. They should indicate when test cases will be designed, written, and executed.
- Responsibilities. For each phase, the people who will design, write, execute, and verify test cases, and the people who will repair discovered errors, should be identified. Since disputes often unfortunately arise, in large projects, over whether particular test results represent errors (e.g., because of ambiguities or missing definitions in the specifications), an arbitrator should be identified.
- Test-case libraries and standards. In a large project, systematic methods of identifying, writing, and storing test cases are necessary.
- Tools. The required test tools must be identified, including a plan for who will develop or acquire them, how they will be used, and when they are needed.
- Computer time. This is a plan for the amount of computer time needed for each testing phase.
- Hardware configuration. If special hardware configurations or devices are needed, a plan describing the requirements, how they will be met, and when they are needed is necessary.
- Integration. Part of the test plan is a definition of how the program will be pieced together (e.g., incremental top-down testing). A system containing major subsystems or programs might be pieced together incrementally (e.g., using the top-down or bottom-up approach, but where the building blocks are programs or subsystems, rather than modules); if so, a system integration plan is necessary. The system integration plan defines the order of integration, the functional capability of each version of the system, and responsibilities for producing "scaffolding" (code that simulates the function of nonexistent components).
- Tracking procedures. Means must be identified to track various aspects of testing progress, including the location of error-prone modules and estimation of progress with respect to the schedule, resources, and completion criteria.
- Debugging procedures. Mechanisms must be defined for reporting detected errors, tracking the progress of corrections, and adding the corrections to the system. Schedules, responsibilities, tools, and computer time must also be part of the debugging plan.
- Regression testing. Regression testing is that testing that is performed after making a functional improvement or repair to the program. Its purpose is to determine if the change has regressed other aspects of the program. It is usually performed by rerunning some subset of the program's test cases. Regression testing is important because changes and error corrections tend to be much more error prone than the original code in the program (in much the same way that most typographical errors in newspapers are the result of last-minute editorial changes, rather than errors in the original copy). A plan for regression testing (e.g., who, how, when) is another necessity.