Inside this Chapter:

3.0.Introduction to Black-Box (or Functional Testing)

3.1.Boundary Value Analysis (BVA)

3.2.Equivalence Class Testing

3.3.Decision Table Based Testing

3.4.Cause-Effect Graphing Technique

3.5.Comparison on Black-Box (or Functional) Testing Techniques

3.6.Kiviat Charts

The term Black-Box refers to the software which is treated as a black-box. By treating it as a black-box, we mean that the system or source code is not checked at all. It is done from the customer’s viewpoint. The test engineer engaged in black-box testing only knows the set of inputs and expected outputs and is unaware of how those inputs are transformed into outputs by the software. We will now discuss various techniques of performing black-box testing.

It is a black-box testing technique that believes and extends the concept that the density of defect is more towards the boundaries. This is done for the following reasons:

    i.Programmers usually are not able to decide whether they have to use <= operator or < operator when trying to make comparisons.

   ii.Different terminating conditions of for-loops, while loops, and repeat loops may cause defects to move around the boundary conditions.

  iii.The requirements themselves may not be clearly understood, especially around the boundaries, thus causing even the correctly coded program to not perform the correct way.

Strongly typed languages such as Ada and Pascal permit explicit definition of variable ranges. Other languages such as COBOL, FORTRAN, and C are not strongly typed, so boundary value testing is more appropriate for programs coded in such languages.

The basic idea of BVA is to use input variable values at their minimum, just above the minimum, a nominal value, just below their maximum, and at their maximum. That is, {min, min+, nom, max–, max}. This is shown in Figure 3.1.

BVA is based upon a critical assumption that is known as single fault assumption theory. According to this assumption, we derive the test cases on the basis of the fact that failures are not due to a simultaneous occurrence of two (or more) faults. So we derive test cases by holding the values of all but one variable at their nominal values and letting that variable assume its extreme values.

If we have a function of n-variables, we hold all but one at the nominal values and let the remaining variable assume the min, min+, nom, max–, and max values, repeating this for each variable. Thus, for a function of n variables, BVA yields (4n + 1) test cases.

1.Boolean and logical variables present a problem for boundary value analysis.

2.BVA assumes the variables to be truly independent which is not always possible.

3.BVA test cases have been found to be rudimentary because they are obtained with very little insight and imagination.

Another variant to BVA is robustness testing. In BVA, we are within the legitimate boundary of our range. That is, we consider the following values for testing:

{min, min+, nom, max–, max} whereas in robustness testing, we try to cross these legitimate boundaries also. So, now we consider these values for testing:

{min–, min, min+, nom, max–, max, max+}

Again, with robustness testing, we can focus on exception handling. With strongly typed languages, robustness testing may be very awkward. For example, in PASCAL, if a variable is defined to be within a certain range, values outside that range result in run-time errors that abort normal execution.

For a program with n-variables, robustness testing will yield (6n + 1) test-cases. So, we can draw a graph now. (Figure 3.2)

Each dot represents a test value at which the program is to be tested. In robustness testing, we cross the legitimate boundaries of input domain. In the graph of Figure 3.2, we show this by dots that are outside the range [a, b] of variable x₁. Similarly, for variable x₂, we have crossed its legitimate boundary of [c, d] of variable x₂.

This type of testing is quite common in electric and electronic circuits.

Furthermore, this type of testing also works on single fault assumption theory.

If we reject our basic assumption of single fault assumption theory and focus on what happens when we reject this theory—it simply means that we want to see what happens when more than one variable has an extreme value. This is multiple path assumption theory. In electronic circuit analysis, this is called as “worst-case analysis.” We use this idea here to generate worst-case test cases.

For each variable, we start with the five- element set that contains the min, min+, nom, max–, and max values. We then take the Cartesian product of these sets to generate test cases. This is shown in Figure 3.3.

For a program with n-variables, 5ⁿ test cases are generated.

Robust worst-case testing yields 7ⁿ test cases.

Before we generate the test cases, first we need to give the problem domain:

Problem Domain: “The triangle program accepts three integers, a, b, and c, as input. These are taken to be the sides of a triangle. The integers a, b, and c must satisfy the following conditions:

C_1: 1 ≤ a ≤ 200    C_4: a < b + c

C_2: 1 ≤ b ≤ 200    C_5: b < a + c

C_3: 1 ≤ c ≤ 200    C_6: c < a + b

The output of the program may be: Equilateral, Isosceles, Scalene, or “NOT-A-TRIANGLE.”

How to Generate BVA Test Cases?

We know that our range is [1, 200] where 1 is the lower bound and 200 is the upper bound. Also, we find that this program has three inputs—a, b, and c. So, for our case

n = 3

BVA yields (4n + 1) test cases, so we can say that the total number of test cases will be (4 × 3 + 1) = 12 + 1 = 13.

Table 3.1 shows those 13 test cases.

TABLE 3.1  BVA test cases for triangle problem.

Please note that we explained above that we can have 13 test cases (4n + 1) for this problem. But instead of 13, now we have 15 test cases. Also, test case ID number 8 and 13 are redundant. So, we ignore them. However, we do not ignore test case ID number 3 as we must consider at least one test case out of these three. Obviously, it is mechanical work!

We can say that these 13 test cases are sufficient to test this program using BVA technique.

Question for Practice

1.Applying the robustness testing technique, how would you generate the test cases for the triangle problem given above?

Before we generate the test cases for the Next Date function, we must know the problem domain of this program:

Problem Domain Next Date is a function of three variables: month, date, and year. It returns the date of next day as output. It reads current date as input date. The conditions are:

If any of conditions C₁, C₂, or C₃ fails, then this function produces an output “value of month not in the range 1...12.”

Because many combinations of dates exist this function just displays one message: “Invalid Input Date.”

Complexities in Next Date Function

A very common and popular problem occurs if the year is a leap year. We have taken into consideration that there are 31 days in a month. But what happens if a month has 30 days or even 29 or 28 days? A year is called as a leap year if it is divisible by 4, unless it is a century year. Century years are leap years only if they are multiples of 400. So, 1992, 1996, and 2000 are leap years while 1900 is not a leap year.

Furthermore, in this Next Date problem, we find examples of Zipf’s law also, which states that “80% of the activity occurs in 20% of the space.” Here also, much of the source-code of the Next Date function is devoted to the leap year considerations.

How to Generate BVA Test Cases for This Problem?

The Next Date program takes date as input and checks it for validity. If valid, it returns the next date as its output.

As we know, with single fault assumption theory, (4n + 1) test cases can be designed. Here, also n = 3. So, the total number of test cases are (4 × 3 + 1) = 12 + 1 = 13.

The boundary value test cases are

So, we have applied BVA on our Next Date problem.

Question for Practice

1.Applying robustness testing, how would you generate the test cases for the Next Date function given above.

Before we generate the test cases, we must formulate the problem statement or domain for commission problem.

Problem Domain: A rifle salesperson sold rifle locks, stocks, and barrels that were made by a gunsmith. Locks cost $45, stocks cost $30, and barrels cost $25. This salesperson had to sell at least one complete rifle per month, and the production limits were such that the most the salesperson could sell in a month was 70 locks, 80 stocks, and 90 barrels. The salesperson used to send the details of sold items to the gunsmith. The gunsmith then computed the salesperson’s commission as follows:

a.10% on sales up to and including $1000

b.15% of the next $800

c.20% on any sales in excess of $1800

The commission program produced a monthly sales report that gave the total number of locks, stocks, and barrels sold, the salesperson’s total dollar sales and finally, the commission.

How to Generate BVA Test Cases for this Problem?

We have 3 inputs in this program. So, we need (4n + 1) = 4 * 3 + 1 = 12 + 1 = 13 test cases to test this program.

The boundary value test cases are listed below in Table. We can also find that the monthly sales are limited as follows:

1 ≤ locks ≤ 70

1 ≤ stocks ≤ 80

1 ≤ barrels ≤ 90

Out of these 15 test cases, 2 are redundant. So, 13 test cases are sufficient to test this program.

1.The normal versus robust values and the single-fault versus the multiple-fault assumption theory result in better testing. These methods can be applied to both input and output domain of any program.

2.Robustness testing is a good choice for testing internal variables.

3.Keep in mind that you can create extreme boundary results from non-extreme input values.

The use of equivalence classes as the basis for functional testing has two motivations:

a.We want exhaustive testing

b.We want to avoid redundancy

This is not handled by the BVA technique as we can see massive redundancy in the tables of test cases.

In this technique, the input and the output domain is divided into a finite number of equivalence classes. Then, we select one representative of each class and test our program against it. It is assumed by the tester that if one representative from a class is able to detect error then why should he consider other cases. Furthermore, if this single representative test case did not detect any error then we assume that no other test case of this class can detect error. In this method, we consider both valid and invalid input domains. The system is still treated as a black-box meaning that we are not bothered about its internal logic.

The idea of equivalence class testing is to identify test cases by using one element from each equivalence class. If the equivalence classes are chosen wisely, the potential redundancy among test cases can be reduced.

For example, in our triangle problem, we would certainly have a test case for an equilateral triangle and we might pick the triple (10, 10, 10) as inputs for a test case. If this is so then it is obvious that there is no sense in testing for inputs like (8, 8, 8) and (100, 100, 100). Our intuition tells us that these would be “treated the same” as the first test case. Thus, they would be redundant. The key and the craftsmanship lies in the choice of the equivalence relation that determines the classes.

Four types of equivalence class testing are discussed below:

1.Weak normal equivalence class testing

2.Strong normal equivalence class testing

3.Weak robust equivalence class testing

4.Strong robust equivalence class testing

We will discuss these one by one.

The word “weak” means single fault assumption. This type of testing is accomplished by using one variable from each equivalence class in a test case. We would, thus, end up with the weak equivalence class test cases as shown in Figure 3.5.

Each dot in Figure 3.5 indicates a test data. From each class we have one dot meaning that there is one representative element of each test case. In fact, we will have, always, the same number of weak equivalence class test cases as the classes in the partition.

This type of testing is based on the multiple fault assumption theory. So, now we need test cases from each element of the Cartesian product of the equivalence classes, as shown in Figure 3.6.

Just like we have truth tables in digital logic, we have similarities between these truth tables and our pattern of test cases. The Cartesian product guarantees that we have a notion of “completeness” in two ways:

a.We cover all equivalence classes.

b.We have one of each possible combination of inputs.

The name for this form of testing is counter intuitive and oxymoronic. The word “weak” means single fault assumption theory and the word “robust” refers to invalid values. The test cases resulting from this strategy are shown in Figure 3.7.

Two problems occur with robust equivalence testing. They are listed below:

1.Very often the specification does not define what the expected output for an invalid test case should be. Thus, testers spend a lot of time defining expected outputs for these cases.

2.Also, strongly typed languages like Pascal and Ada, eliminate the need for the consideration of invalid inputs. Traditional equivalence testing is a product of the time when languages such as FORTRAN, C, and COBOL were dominant. Thus, this type of error was common.

This form of equivalence class testing is neither counter intuitive nor oxymoronic but is redundant. As explained earlier “robust” means consideration of invalid values and “strong” means multiple fault assumption. We obtain the test cases from each element of the Cartesian product of all the equivalence classes. This is shown in Figure 3.8.

We find here that we have 8 robust (invalid) test cases and 12 strong or valid inputs. Each is represented with a dot. So, totally we have 20 test cases (represented as 20 dots) using this technique.

As stated in the problem definition earlier, we note that in our triangle problem four possible outputs can occur:

a.NOT-A-TRIANGLE

b.Scalene

c.Isosceles

d.Equilateral

We can use these to identify output (range) equivalence classes as follows:

01 = {<a, b, c>: the triangle is equilateral}

02 = {<a, b, c>: the triangle is isosceles}

03 = {<a, b, c>: the triangle is scalene}

04 = {<a, b, c>: sides a, b, and c do not form a triangle}

Now, we apply these four techniques of equivalence class partitioning one by one to this problem.

a.The four weak normal equivalence class test cases are:

b.Because no valid subintervals of variables a, b, and c exist, so the strong normal equivalence class test cases are identical to the weak normal equivalence class test cases.

c.Considering the invalid values for a, b, and c yields the following additional weak robust equivalence class test cases:

d.While strong robust equivalance class test cases are:

Please note that the expected outputs describe the invalid input values thoroughly.

The actual craft of choosing the test cases lies in this example. Recall that Next Date is a function of three variables— month (mm), day (dd), and year (yyyy). Assume that their ranges are

So, based on valid values, the equivalence classes are:

And the invalid equivalence classes are:

a. & b.Because the number of valid classes equals the number of independent variables, only one weak normal equivalence class test case occurs and it is identical to the strong normal equivalence class test case:

So, we get this test case on the basis of valid classes – M₁, D₁, and Y₁ above.

c.Weak robust test cases are given below:

So, we get 7 test cases based on the valid and invalid classes of the input domain.

d.Strong robust equivalence class test cases are given below:

Modified Equivalence Class for this Problem

We need the modified classes as we know that at the end of a month, the next day is 1 and the month is incremented. At the end of a year, both the day and the month are reset to 1 and the year is also incremented. Finally, the problem of leap year makes determining the last day of a month interesting. With all this in mind, we postulate the following equivalence classes:

So, now what will be the weak equivalence class test cases?

As done earlier, the inputs are mechanically selected from the approximate middle of the corresponding class:

This mechanical selection of input values makes no consideration of our domain knowledge and thus, we have two impossible dates. This will always be a problem with “automatic” test case generation because all of our domain knowledge is not captured in the choice of equivalence classes.

The strong normal equivalence class test cases for the revised classes are:

So, three month classes, four day classes, and three year classes results in 3 × 4 × 3 = 36 strong normal equivalence class test cases. Furthermore, adding two invalid classes for each variable will result in 150 strong robust equivalence class test cases.

It is difficult to show these 150 classes here.

The valid classes of the input variables are as follows:

The corresponding invalid classes of the input variables are as follows:

a.& b.  As the number of valid classes is equal to the number of independent variables, so we have exactly one weak normal equivalence class test case and again, it is identical to the strong normal equivalence class test case. It is given in the following table.

c.Also, we have seven weak robust test cases as given below:

d.And finally, the strong robust equivalence class test cases are as follows:

The following are guidelines for equivalence class testing:

1.The weak forms of equivalence class testing (normal or robust) are not as comprehensive as the corresponding strong forms.

2.If the implementation language is strongly typed and invalid values cause run-time errors then it makes no sense to use the robust form.

3.If error conditions are a high priority, the robust forms are appropriate.

4.Equivalence class testing is approximate when input data is defined in terms of intervals and sets of discrete values. This is certainly the case when system malfunctions can occur for out-of-limit variable values.

5.Equivalence class testing is strengthened by a hybrid approach with boundary value testing (BVA).

6.Equivalence class testing is used when the program function is complex. In such cases, the complexity of the function can help identify useful equivalence classes, as in the next date problem.

7.Strong equivalence class testing makes a presumption that the variables are independent and the corresponding multiplication of test cases raises issues of redundancy. If any dependencies occur, they will often generate “error” test cases, as shown in the next date function.

8.Several tries may be needed before the “right” equivalence relation is established.

9.The difference between the strong and weak forms of equivalence class testing is helpful in the distinction between progression and regression testing.

Of all the functional testing methods, those based on decision tables are the most rigorous because decision tables enforce logical rigor.

Decision tables are a precise and compact way to model complicated logic. They are ideal for describing situations in which a number of combinations of actions are taken under varying sets of conditions.

It is another popular black-box testing technique. A decision table has four portions:

a.Stub portion

b.Entry portion

c.Condition portion

d.Action portion

A column in the entry portion is a rule. Rules indicate which actions are taken for the conditional circumstances indicated in the condition portion of the rule. Decision tables in which all conditions are binary are called limited entry decision tables. If conditions are allowed to have several values, the resulting tables are called extended entry decision tables.

To identify test cases with decision tables, we follow certain steps:

Number of Test Cases = Number of Rules

For a limited entry decision table, if n conditions exist, there must be 2ⁿ rules.

Advantages of Decision Tables

1.This type of testing also works iteratively. The table that is drawn in the first iteration acts as a stepping stone to derive new decision table(s) if the initial table is unsatisfactory.

2.These tables guranatee that we consider every possible combination of condition values. This is known as its “completeness property.” This property promises a form of complete testing as compared to other techniques.

3.Decision tables are declarative. There is no particular order for conditions and actions to occur.

Disadvantages of Decision Tables

Decision tables do not scale up well. We need to “factor” large tables into smaller ones to remove redundancy.

Applications of Decision Tables

This technique is useful for applications characterized by any of the following:

a.Prominent if-then-else logic.

b.Logical relationships among input variables.

c.Calculations involving subsets of the input variables.

d.Cause-and-effect relationships between inputs and outputs.

e.High cyclomatic complexity.

Technique: To identify test cases with decision tables, we interpret conditions as inputs and actions as output. The rules are interpreted as test cases. Because the decision table can mechanically be forced to be complete, we know we have a comprehensive set of test cases.

Example of a Decision Table: The triangle problem

Each “-” (hyphen) in the decision table represents a “don’t care” entry. Use of such entries has a subtle effect on the way in which complete decision tables are recognized. For limited entry decision tables, if n conditions exist, there must be 2ⁿ rules. When don’t care entries indicate that the condition is irrelevant, we can develop a rule count as follows:

Also, we can have 2ⁿ entries, i.e., 2⁶ = 64 entries. Now we establish the rule and the rule count for the above decision table.

From the previous table we find that the rule count is 64. And we have already said that 2ⁿ = 2⁶ = 64. So, both are 64.

The question, however, is to find out why the rule count is 32 for the Rule-1 (or column-1)?

We find that there are 5 don’t cares in Rule-1 (or column-1) and hence 2ⁿ = 2⁵ = 32. Hence, the rule count for Rule-1 is 32. Similarly, for Rule-2, it is 2⁴ = 16 and 2³ = 8 for Rule-3. However, from Rule-4 through Rule-11, the number of don’t care entries is 0 (zero). So rule count is 2⁰ = 1 for all these columns. Summing the rule count of all columns (or R₁-R₁₁) we get a total of 64 rule count.

Many times some problems arise with these decision tables. Let us see how.

Consider the following example of a redundant decision table:

Please note that the action entries in Rule-9 and Rules 1–4 are NOT identical. It means that if the decision table were to process a transaction in which C₁ is true and both C₂ and C₃ are false, both rules 4 and 9 apply. We observe two things

1.Rules 4 and 9 are in-consistent because the action sets are different.

2.The whole table is non-deterministic because there is no way to decide whether to apply Rule-4 or Rule-9.

Also note carefully that there is a bottom line for testers now. They should take care when don’t care entries are being used in a decision table.

We have already studied the problem domain for the famous triangle problem in previous chapters. Next we apply the decision table based technique on the triangle problem. The following are the test cases:

So, we get a total of 11 functional test cases out of which three are impossible cases, three fail to satisfy the triangle property, one satisfies the equilateral triangle property, one satisfies the scalene triangle property, and three ways to get an isoceles triangle.

In the previous technique of equivalence partitioning we found that certain logical dependencies exist among variables in the input domain. These dependencies are lost in a Cartesian product. The decision table format allows us to use the notion of “impossible action” to denote impossible combinations of conditions. Thus, such dependencies are easily shown in decision tables.

From the problem domain for the next date function, we know that there are some critical situations like the treatment of leap years, special treatment to year 2000 (Y2K), and special treatment to December month and a 28 day month is also to be given. So, we go for 3 tries before we derive its test cases.

First try: Special treatment to leap years.

Second try: Special treatment to year = 2000.

Third try: Special treatment to December month and days = 28.

First try: Special treatment to leap years.

The art of testing and the actual craftsmanship lies in identifying appropriate conditions and actions. Considering the following equivalence classes again:

Based on these classes, we draw the decision table:

Herein, we have 3 × 4 × 2 = 24 elements and 2ⁿ = 2⁸ = 256 entries. Many of the rules would be impossible (note that the classes Y₁ and Y₂ collapse into one condition C₈). Some of these rules are impossible because

a.there are too many days in a month.

b.they cannot happen in a non-leap year.

c.they compute the next date.

Second try: Special treatment given to year 2000.

As we know, the year 2000 is a leap year. Hence, we must give special treatment to this year by creating a new class-Y₁ as follows:

This results in 3 × 4 × 3 = 36 rules that represents the Cartesian product of 3 month classes (M₁.... M₃), 4 day classes (D₁....D₄), and 3 year classes.

So, after second try, our decision table is

Why is the rule count value 3 in column 1?

In column 1 or Rule 1 of this decision table, we have 3 possibilities with don’t care:

i.e., with “–” or don’t care we have either Y₁ or Y₂ or Y₃.

Also note that in Rule 8, we have three impossible conditions, shown as “?” (question mark). Also, we find that this table has certain problems with December month. We will fix these next in the third try.

Third try: Special treatment to December month and to number of days = 28.

Because we know that we have serious problems with the last day of last month, i.e., December. We have to change month from 12 to 1. So, we modify our classes as follows:

The Cartesian product of these contain 40 elements. Here, we have a 22-rule decision table. This table gives a clearer picture of the Next Date function than does the 36-rule decision table and is given below:

In this table, the first five rules deal with 30-day months. Notice that the leap year considerations are irrelevant. Rules (6 – 10) and (11 – 15) deal with 31-day months where the first five with months other than December and the second five deal with December. No impossible rules are listed in this portion of the decision table.

Still there is some redundancy in this table. Eight of the ten rules simply increment the day. Do we really require eight separate test cases for this sub-function? No, but note the type of observation we get from the decision table.

Finally, the last seven rules focus on February and leap year. This decision table analysis could have been done during the detailed design of the Next Date function.

Further simplification of this decision table can also be done. If the action sets of two rules in a decision table are identical, there must be at least one condition that allows two rules to be combined with a don’t care entry. In a sense, we are identifying equivalence classes of these rules. For example, rules 1, 2, and 3 involve day classes as D₁, D₂, and D₃ (30 day classes). These can be combined together as the action taken by them is the same. Similarly, for other rules other combinations can be done. The corresponding test cases are shown in the table as in Figure 3.17.

Because, we have 22 rules there are 22 test cases that are listed above.

The commission problem is not suitable to be solved using this technique of decision table analysis because very little decisional logic is used in the problem. The variables in the equivalence classes are truly independent, therefore no impossible rules will occur in a decision table in which conditions correspond to the equivalence classes. Thus, we will have the same test cases as we did for equivalence class testing.

The following guidelines have been found after studying the previous examples:

1.This technique works well where lot of decision making takes place such as the triangle problem and next date problem.

2.The decision table technique is indicated for applications characterized by any of the following:

3.Decision tables do not scale up well. We need to “factor” large tables into smaller ones to remove redundancy.

4.It works iteratively meaning that the table drawn in the first iteration, and acts as a stepping stone to design new decision tables, if the initial table is unsatisfactory.

Cause-effect graphing is basically a hardware testing technique adapted to software testing. It is a black-box method. It considers only the desired external behavior of a system. This is a testing technique that aids in selecting test cases that logically relate causes (inputs) to effects (outputs) to produce test cases.

A “cause” represents a distinct input condition that brings about an internal change in the system. An “effect” represents an output condition, a system transformation, or a state resulting from a combination of causes.

Myer suggests the following steps to derive test cases:

Basic cause-effect graph symbols used are given below:

Consider each node as having the value 0 or 1 where 0 represents the “absent state” and 1 represents the “present state.” Then the identity function states that if c₁ is 1, e₁ is 1, or we can say if c₁ is 0, e₁ is 0.

The NOT function states that if C₁ is 1, e₁ is 0 and vice versa. Similarly, the OR function states that if C₁ or C₂ or C₃ is 1, e₁ is 1 else e₁ is 0. The AND function states that if both C₁ and C₂ are 1, e₁ is 1; else e₁ is 0. The AND and OR functions are allowed to have any number of inputs.

We follow the steps listed in Section 3.4.1 to design the test cases for our triangle problem:

Step 1. First, we must identify the causes and its effects. The causes are:

C_1: Side x is less than sum of y and z

C_2: Side y is less than sum of x and z

C_3: Side z is less than sum of x and y

C_4: Side x is equal to side y

C_5: Side x is equal to side z

C_6: Side y is equal to side z

The effects are:

e_1: Not a triangle

e_2: Scalene triangle

e_3: Isosceles triangle

e_4: Equilateral triangle

e_5: Impossible

Step 2. Its cause-effect graph is shown in Figure 3.18.

Step 3. We transform it into a decision table:

Step 4. Because there are 11 rules, we get 11 test cases and they are:

Problem 1. Consider the payroll system of a person.

a.If the salary of a person is less than $70,000 and expenses do not exceed $30,000, then a 10% tax is charged by the IT department.

b.If the salary is greater than $60,000 and less than or equal to $3000 and expenses don’t exceed $40,000, then a 20% tax is charged by the IT department.

c.For a salary greater than $3000, a 5% additional surcharge is also charged.

d.If expenses are greater than $40,000, the surcharge is 9%.

Design test-cases using decision table based testing technique.

Solution. See the following steps:

Step 1. All causes and their effects are identified:

Step 2. It’s cause-effect graph is drawn.

Step 3. We transform this cause-effect graph into a decision table. Please note that these “causes” and “effects” are nothing else but “conditions” and “actions” of our decision table. So, we get:

That is, if C₁ and C₄ are 1 (or true) then the effect (or action) is E₁. Similarly, if C₂ and C₅ is 1 (or true), action to be taken is E₂, and so on.

Step 4. Because there are 4 rules in our decision table above, we must have at least 4 test cases to test this system using this technique.

These test cases can be:

1.Salary = 20,000, Expenses = 2000

2.Salary = 100,000, Expenses = 10,000

3.Salary = 300,000, Expenses = 20,000

4.Salary = 300,000, Expenses = 50,000

So we can say that a decision table is used to derive the test cases which can also take into account the boundary values.

1.If the variables refer to physical quantities, domain testing and equivalence class testing are indicated.

2.If the variables are independent, domain testing and equivalence class testing are indicated.

3.If the variables are dependent, decision table testing is indicated.

4.If the single-fault assumption is warranted, boundary value analysis (BVA) and robustness testing are indicated.

5.If the multiple-fault assumption is warranted, worst-case testing, robust worst-case testing, and decision table testing are identical.

6.If the program contains significant exception handling, robustness testing and decision table testing are indicated.

7.If the variables refer to logical quantities, equivalence class testing and decision table testing are indicated.

The functional methods that we have studied so far vary both in terms of the number of test cases generated and the effort to develop these test cases. To compare the three techniques, namely, boundary value analysis (BVA), equivalence class partitioning, and decision table based technique, we consider the following curve shown in Figure 3.21.

The domain-based techniques have no recognition of data or logical dependencies. They are very mechanical in the way they generate test cases. Because of this, they are also easy to automate. The techniques like equivalence class testing focus on data dependencies and thus we need to show our craft. The thinking goes into the identification of the equivalence classes and after that, the process is mechanical. Also note that from the graph, the decision table based technique is the most sophisticated because it requires the tester to consider both data and logical dependencies. As we have seen in our example of the next date function, we had to go three times but once we get a good and healthy set of conditions, the resulting test cases are complete and minimal.

Now, consider another graph to show the effort required to identify the test cases versus its sophistication.

We can say that the effort required to identify test cases is the lowest in BVA and the highest in decision tables. The end result is a trade-off between the test case effort identification and test case execution effort. If we shift our effort toward more sophisticated testing methods, we reduce our test execution time. This is very important as tests are usually executed several times. Also note that, judging testing quality in terms of the sheer number of test cases has drawbacks similar to judging programming productivity in terms of lines of code.

The examples that we have discussed so far show these trends.

What we found in all of these functional testing strategies is that either the functionality is untested or the test cases are redundant. So, gaps do occur in functional test cases and these gaps are reduced by using more sophisticated techniques.

We can develop various ratios of the total number of test cases generated by method-A to those generated by method-B or even ratios on a test case basis. This is more difficult but sometimes management demands numbers even when they have little meaning. When we see several test cases with the same purpose, sense redundancy, detecting the gaps is quite difficult. If we use only functional testing, the best we can do is compare the test cases that result from two methods. In general, the more sophisticated method will help us recognize gaps but nothing is guaranteed.

How can we find out the effectiveness of the testing techniques?

a.By being dogmatic, we can select a method, use it to generate test cases, and then run the test cases. We can improve on this by not being dogmatic and allowing the tester to choose the most appropriate method. We can gain another incremental improvement by devising appropriate hybrid methods.

b.The second choice can be the structural testing techniques for the test effectiveness. This will be discussed in subsequent chapters.

Note, however, that the best interpretation for testing effectiveness is most difficult. We would like to know how effective a set of test cases is for finding faults present in a program. This is problematic for two reasons.

1.It presumes we know all the faults in a program.

2.Proving that a program is fault free is equivalent to the famous halting problem of computer science, which is known to be impossible.

What Is the Best Solution?

The best solution is to work backward from fault types. Given a particular kind of fault, we can choose testing methods (functional and structural) that are likely to reveal faults of that type. If we couple this with knowledge of the most likely kinds of faults, we end up with a pragamatic approach to testing effectiveness. This is improved if we track the kinds of faults and their frequencies in the software we develop.

1.If the variables refer to physical quantities then domain testing and equivalence class testing are used.

2.If the variables are independent then domain testing and equivalence class testing are used.

3.If the variables are dependent, decision table testing is indicated.

4.If the single-fault assumption is warranted then BVA and robustness testing are used.

5.If the multiple-fault assumption is warranted then worst case testing, robust worst case testing, and decision table based testing are indicated.

6.If the program has significant exception handling, robustness testing and decision table based testing are identical.

7.If the variable refers to logical quantities, equivalence class testing and decision table testing are indicated.

A kiviat chart visually displays a set of metrics that provides easy viewing of multiple metrics against minimum and maximum thresholds. Each radial of a Kiviat chart is a metric. All metrics are scaled so that all maximums are on a common circle and all minimums are on a common circle.

In the charts below, the circle is the maximum threshold and the band is the minimum threshold. The band between the two is the acceptable range. The chart on the left shows that all metrics are well within the acceptable range. The chart on the right shows an example where all metrics are above maximum limits.

The best graphical or visual representation of complex concepts and data can be communicated through Kiviat charts. In addition, Kiviat charts provide a quick focus on the areas that are in most need of attention and can be customized to use different scales to represent various problems. This deliverable will help you get smart on what a Kiviat chart is and how to use it properly.

For an update on Kiviat charts using R! visit: http://metrico.statanything.com/diagramas-kiviat-spider-charts-em-r/

Why Is It Needed?

The problem – multiple dimensions:

Computer: processor

•printer

•disks

•CDROM

•modem

Software

Personnel

All these together create multiple dimensions and hence the concept of balance is needed.

Kiviat Charts: The Method

The following steps are usually followed to draw Kiviat charts:

We shall now consider four different cases to draw Kiviat charts.

Case 1: Kiviat charts—A perfectly balanced system.

Consider a sample data given below. Its Kiviat chart is also shown below:

Case 2: Kiviat charts—A well-balanced system.

Consider another sample data for a well-balanced system. Using the steps listed earlier, we draw its kiviat chart.

Case 3: Kiviat charts—A poorly balanced system.

Now consider a poorly balanced system. Its kiviat chart is shown below:

Case 4: Kiviat charts—A perfectly balanced system.

Consider another set of sample data. We draw its kiviat chart.

Kiviat Charts—A Different Example (CASE STUDY)

Internet Dimensions Compared

Internet Dimensions Compared in the Form of a Kiviat Chart

Problem with Kiviat Charts

The Cost/Utilization Method

Illustrative Cost/Utilization Histograms

Cost/Utilization—The Method

The following steps are shown below:

1.Choose factors to be measured.

2.Determine the cost of each factor as a percent of total system cost.

3.Determine the utilization of each factor.

4.Prepare a chart showing the cost and utilization of each factor.

5.Compute the measure of cost/utilization, F.

6.Compute the measure of balance, B.

7.Evaluate the resulting chart and measures.

Cost/Utilization—The Measures

Cost/Utilization:
where:	ui = percent utilization of factor i
	pi = cost contribution of factor i
Balance:

Conclusions

It is essential to maintain balance between system components in order to:

reduce costs.

maintain smooth functioning with no bottlenecks.

attain effectiveness AND efficiency.

We summarize the scenarios under which each of these techniques will be useful:

1.Which is not a functional testing technique?

a.BVA

b.Decision table

c.Regression testing

d.None of the above.

2.One weakness of BVA and equivalence partitioning is

a.They are not effective.

b.They do not explore combinations of input circumstances.

c.They explore combinations of input circumstances.

d.None of the above.

3.Decision tables are useful in a situation where

a.An action is taken under varying sets of corditions.

b.A number of combinations of actions are taken under varying sets of conditions.

c.No action is taken under varying sets of conditions.

d.None of the above.

4.“Causes” and “Effects” are related to

a.Input and output

b.Output and input

c.Destination and source

d.None of the above.

5.Functionality of a software is tested by

a. White-box testing.

b.Black-box testing.

c.Regression testing.

d.None of the above.

6.If n represents the number of variables in a program then BVA yields how many test cases?

a. 4n + 2

b.4n + 1

c.n + 2

d.n + 1

7.For a function of n variables, the robustness testing will yield

a. 6n + 1 test cases.

b.6n + 2 test cases.

c.6n + 4 test cases.

d.None of the above.

8.Worst case testing yields

a. 5n test cases.

b.5n + 1.

c.5ⁿ.

d.None of the above.

9.BVA is based upon

a.Single fault assumption theory.

b.Multiple fault assumption theory.

c.Both of the above.

d.None of the above.

10.In decision tables, which of the following is true?

a.Number of test cases is equal to number of rules (or columns)

b.Number of test cases is not equal to number of rules (or columns)

c.Both (a) and (b)

d.None of the above.

1.c.

2.b.

3.b.

4.a.

5.b.

6.b.

7.a.

8.c.

9.a.

10.a.

1.Perform the following:

2.Explain any five symbols used in the cause-effect graphing technique?

3.How do you measure:

4.Write a short paragraph:

5.Explain the significance of boundary value analysis. What is the purpose of worst case testing?

6.Describe cause-effect graphing technique with the help of an example.

  7.a.Discuss different types of equivalence class tests cases.

8.How can we measure and evaluate test effectiveness? Explain with the help of 11 step S/W testing process.

9.What is the difference between:

Equivalence partitioning and boundary value analysis methods?

10.Consider the previous date function and design test cases using the following techniques:

The function takes current date as an input and returns the previous date of the day as the output.

All variables have integer values subject to conditions as follows:

C1: 1 ≤ month ≤ 2

C2: 1 ≤ day ≤ 31

C3: 1920 ≤ year ≤ 2000.

11.Compare the single/multiple fault assumption theory with boundary value and equivalence class testing.

12.Explain the cause-effect graphing technique in detail. Why is it different from other functional testing techniques?

13.Differentiate between positive and negative functional testing.

14.What is domain testing? Illustrate through a suitable example.

15.a.“Software testing is an incremental process.” Justify the statement.

16.a.Consider a program that calculates the roots of a quadratic equation with a, b, and c in range [1, 100]. Design test cases using boundary value and robustness testing.

17.Write a short paragraph on cause-effect graphing.

18.Discuss any two model-based black-box testing approaches?

19.A program computes the grade of each student based on the average of marks obtained by them in physics, chemistry, and math. Maximum marks in each subject is 60. Grades are awarded as follows:

Design robust test cases and identify equivalence class test cases for output and input domains for this problem.

20.What is the difference between weak normal and strong normal equivalence class testing?

21.Consider a program for the determination of previous date. Its input is a triple of day, month, and year with the values in the range:

1 ≤ month ≤ 12

1 ≤ day ≤ 31

1900 ≤ year ≤ 2025

The possible outputs are “Previous date” and “Invalid date.” Design a decision table and equivalence classes for input domain.

22.Consider a program given below for the selection of the largest of numbers.

23.Consider the above program and show that why is it practically impossible to do exhaustive testing?

24.a.Consider the following point-based evaluation system for a trainee salesman of an organization:

Generate the test cases using equivalence class testing.

25.Design test cases using the functional testing taking any example program? Explain how and why complete testing is not possible by highlighting some cases of the example given.

26.Let us consider an example of grading the students in an academic institution. The grading is done according to the following rules:

Generate test cases using the equivalence class testing technique.

27.Consider the following point-based evaluation system for a salesman of an organization.

Generate the test cases using equivalence class testing.

28.Consider a program for the determination of previous date. Its input is a triple of day, year, month with values in range:

1 ≤ month ≤ 12

1 ≤ day ≤ 31

1912 ≤ year ≤ 2020

Design a decision table for the given scenario.

29.Consider the program for the determination of next date in a calender. Its input is a triple of day, month, and year with the following range:

1 ≤ month ≤ 12

1 ≤ day ≤ 31

1900 ≤ year ≤ 2025

The possible outputs would be next date or invalid date. Design boundary value, robust, and worst test cases for this program.

 30.a.Explain cause-effect graphing technique in detail. Why is it different from other functional techniques?

31.Consider an example of grading a student in a university. The grading is done as below:

The marks of any three subjects are considered for the calculation of average marks. Scholarships of $1000 and $500 are given to students securing more than 90% and 85% marks, respectively. Develop a decision table, cause effect graph, and generate test cases for the above scenario.