Testing Methodologies - Software testing methodology

Testing methodologies are set of rules or guidelines that are followed to minimize the number of test cases and also provide with maximum test coverage

The following methods are commonly used:

· equivalence partioning
· boundary-value analysis
· error guessing

The following are lesser-used methods:

· cause-effect graphing
· syntax testing
· state transition testing
· graph matrix

Equivalence partitioning
Equivalence partitioning is a systematic process that identifies, on the basis of whatever information is available, a set of interesting classes of input conditions to be tested, where each class is representative of (or covers) a large set of other possible tests. If partitioning is applied to the product under test, the product is going to behave in much the same way for all members of the class.
The aim is to minimize the number of test cases required to cover these input conditions.
There are two distinct steps. The first is to identify the equivalence classes (ECs) and the second is to identify the test cases.

(1) Identifying equivalence classes

For each external input:

(i) If the input specifies a range of valid values, define one valid EC (within the range) and two invalid Ecs (one outside each end of the range).

Example: If the input requires a month in the range of 1-12, define one valid EC for months 1 through 12 and two invalid ECs (month<1>12).

(ii) If the input specifies the number (N) of valid values, define one valid EC and two invalid ECs (none, and more than N).

Example: If the input requires the titles of at least three but no more than eight books, then define one valid EC and two invalid ECs (<3>8 books).

(iii) If the input specifies a set of valid values, define one valid EC (within the set) and one invalid EC (outside the set).

Example: If the input requires one of the names TOM, DICK, or HARRY, then define one valid EC (using one of the valid names) and one invalid EC (using the name JOE).

(iv) If there is reason to believe that the program handles each valid input differently, then define one valid EC per valid input.

(v) If the input specifies a “must be” situation, define one valid EC and one invalid EC.

Example: If the first character of the input must be numeric, then define one valid EC where the first character is a number and one invalid EC where the first character is not a number.

(vi) If there is reason to believe that elements in an EC are not handled in an identical manner by the program, subdivide the EC into smaller ECs.

(2) Identifying test cases

(i) Assign a unique number to each EC.
(ii) Until all valid ECs have been covered by test cases, write a new test case covering as many of the uncovered ECs as possible.
(iii) Until all invalid Ecs have been covered by test cases, write a test case that covers one, and only one, of the uncovered invalid ECs.
(iv) If multiple invalid ECs are tested in the same test case, some of those tests may never be executed because the first test may mask other tests or terminate execution of the test case.

Equivalence partitioning significantly reduces the number of input conditions to be tested by identifying classes of conditions that are equivalent to many other conditions. It does not test combinations of input conditions.



Boundary-value analysis

Boundary-value analysis is a variant and refinement of equivalence partitioning, with two major differences:

First, rather than selecting any element in an equivalence class as being representative, elements are selected such that each edge of the EC is the subject of a test. Boundaries are always a good place to look for defects.

Second, rather than focusing exclusively on input conditions, output conditions are also explored by defining output ECs. What can be output? What are the classes of output? What should I create as an input to force a useful set of classes that represent the outputs that ought to be produced?

The guidelines for boundary-value analysis are:

· If an input specifies a range of valid values, write test cases for the ends of the range and invalid-input test cases for conditions just beyond the ends.

Example: If the input requires a real number in the range 0.0 to 90.0 degrees, then write test cases for 0.0, 90.0, -0.001, and 90.001.

· If an input specifies a number of valid values, write test cases for the minimum and maximum number of values and one beneath and beyond these values.

Example: If the input requires the titles of at least 3, but no more than 8, books, then write test cases for 2, 3, 8, and 9 books.

· Use the above guidelines for each output condition.

Boundary-value analysis is not as simple as it sounds, because boundary conditions may be subtle and difficult to identify. The method does not test combinations of input conditions.


Error guessing


Error guessing is an ad hoc approach, based on intuition and experience, to identify tests that are considered likely to expose errors. The basic idea is to make a list of possible errors or error-prone situations and then develop tests based on the list. What are the most common error-prone situations we have seen before? Defects’ histories are useful. There is a high probability that defects that have been there in the past are the kind that are going to be there in the future.

Some items to try are:

· empty or null lists/strings
· zero instances/occurrences
· blanks or null characters in strings
· negative numbers

One of the studies done by Myers (1979) states that the probability of errors remaining in the program is proportional to the number of errors that have been found so far. This alone provides a rich source of focus for productive error guessing.