This question appears in Mumbai University > Software Testing & Quality Assurance Subject

Marks: 10 M

Year: May 2014, June 2015

Test Execution Strategy:

• It is important to have a proper game plan ready before the system test group receives a software system for the first time for performing system testing.

• The game plan is in the form of a test execution strategy to carry out the task.

• The following concerns are addressed before initiating system test execution:

1. How many times are the test cases executed and when?

2. What does one do with the failed test cases?

3. What happens when too many test cases fail?

4. In what order are the test cases executed?

5. Which test cases are to be run just before ending the system testing phase?

• In the absence of a good system test execution strategy, it is highly unlikely that the desired system level test is performed within the given time and without overusing resources.

• An effective and efficient execution strategy must take into account characteristics of test execution, defect detection and defect removal.

• A process model is defined for a metric based, multicycle test execution strategy which clearly defines an entry criterion to start system testing, how to prioritize execution of test cases, characterize each test cycle by a set of six parameters : goal, assumptions, test execution, revert and extension criteria, action and exit criteria.

• Multicycle System Test Strategy:

1. It is a test execution strategy with three cycles, where in one test cycle all the test cases in the entire test suite are executed at least once.

2. The total number of test cycles to be used in individual test projects is a matter of management decision based on the level of delivered quality one wants to achieve.

• Characterization of Test Cycles:

1. Each test cycle is characterized by a set of six parameters as follows:

   a.Goals

   b.Assumptions

   c.Test execution

   d.Actions

   e.Revert and extension criteria

   f.Exit criteria

2. Appropriate values must be assigned to these parameters while preparing a concrete test execution strategy.

• Preparing the First Test Cycle:

  1.In order to understand the first test cycle following should be considered :

  a.The different states of a known defect in the form of a life cycle from the new state to the closed state.

  b.Assignment of test cases to test engineers.

  c.An entry criterion indicating when to start the first test cycle.

• Selecting test cases for final test cycle :

  1. Regression testing is applied in the final test cycle. Here test cases are selected based on (i) the results of their prior execution (ii) their membership in certain test groups : basic, functionality, robustness, interoperability, stress, scalability, performance, and load and stability and (iii) their association with software components that have been modified.

     2.Test cases are selected in three steps :

     a. In the first step, the test suite is partitioned into four bins – red, yellow, green and white. The red bin is used to hold the test cases that must be executed, the yellow bin is used to hold the test cases that are useful to execute and the green bin is used to hold the test cases that will not add any value to regression testing and thus can be skipped.

     b.In the second step, test cases from the white bin are moved to the other bins by considering the software components that were modified during system testing and the test cases that are associated with those components.

  c.Finally, in the third step, the red and yellow bins are selected for regression testing.

• Prioritization of Test Cases :

  1. Prioritization of test cases means ordering the execution of test cases according to certain test objectives.

  2. An example of a test objective for prioritization is : Execute the maximum number of test cases without being blocked.

     3.It is necessary to prioritize the execution of test cases to detect critical defects early in the test cycles.

• Details of three test cycles:

  1. Test Cycle 1 : In this cycle, try to detect most of the defects by executing all the test cases.

  2. Test Cycle 2 : The fixes for the defects found in the first test cycle are verified in the second test cycle.

  3. Test Cycle 3 : In the third and the final test cycle, a selected subset of test cases of the original test suite is executed to ensure that the software is stable before it is released to the customer. The objective of this test cycle is to execute all the selected test cases against a single software image.