written 5.3 years ago by |
This type of prioritization is based on the coverage of codes, such as statement coverage and branch coverage, and the fault exposing capability of the test cases. Test cases are ordered based on their coverage. For example, count the number of statements covered by the test cast cases. The test case on that covers the highest number of statements will be executed first. Some of the techniques are discussed below.
Total Statement Coverage Prioritization
This prioritization orders the test cases based on the total number of statements covered. It counts the number of statements covered by the test cases and orders them in a descending order. If multiple test cases cover the same number of statements, then a random order may be used.
For example, if $T_{1}$ covers 5 statements, $T_{2}$ covers 3, and $T_{3}$ covers 12 statements, then according to this prioritization, the order will be $T_{3}, T_{1}, T_{2} .$
Additional Statement Coverage Prioritization
Total statement coverage prioritization schedules the test cases based on the total statements covered. However, it will be useful if it can execute those statements that have not been covered as yet.
Additional statement coverage prioritization iteratively selects a test case $T_{1}$ that yields the greatest statement coverage, and then selects a test case which covers a statement uncovered by $T_{1}$ . Repeat the process until all statements covered by at least one test case have been covered.
Total Branch Coverage Prioritization
In this prioritization, the criterion to order is to consider condition branches in a program instead of statements. Thus, it is the coverage of each possible outcome of a condition in a predicate. The test case which will cover maximum branch outcomes will be ordered first.
Additional Branch Coverage Prioritization
Here, the idea is the same as in additional statement coverage of first selecting the test case with the maximum coverage of branch outcomes and then, selecting the test case which covers the branch, outcome not covered by the previous one.
Total Fault-Exposing-Potential Prioritization
Statement and branch coverage prioritization ignore a fact about test cases and faults:
- Some bugs/faults are more easily uncovered than other faults.
- Some test cases have the proficiency to uncover particular bugs as compared to other test cases.
Thus, the ability of a test case to expose a fault is called the fault exposing potential (FEP). It depends not only on whether test cases cover a faulty statement, but also on the probability that a fault in that statement will also cause a failure for that test case.
To obtain an approximation of the FEP of a test case, an approach was adopted using mutation analysis. This approach is discussed below.
Given a program P and a test suite T,
- First, create a set of mutants $N=\left\{n_{1}, n_{2}, \ldots, n_{n}\right\}$ for P, noting which statement $s_{j}$ in Pcontains each mutant.
- Next, for each test case $t_{i} \in T,$ execute each mutant version $n_{k}$ of $P$ on $t_{p}$ noting whether $t_{i}$ kills that mutant. - Having collected this information for every test case and mutant, consider each test case $t_{p}$ and each statement $s_{j}$ in $P,$ and calculate $\operatorname{FEP}(s, t)$ of $t_{i}$ on $s_{j}$ as the ratio $$\frac{\text { Mutants of } s_{j} \text { killed }}{\text { Total number of mutants of } s_{j}}$$
If $t_{i}$ does not execute $s_{n}$ this ratio is zero.
To perform total FEP prioritization, given these (FEP)(s, t) values, calculate an award value for each test case $t_{i} \in T,$ by summing the $(\mathrm{FEP})\left(s_{p} t_{i}\right)$ values for all statements $s_{i}$ in $P .$ Given these award values, we prioritize the test cases by sorting them in order of descending award value.