Activity: Perform Unit Tests

Unit means not only a class in an object-oriented language, but also free subprograms, such as functions in C++.

For each unit (implemented class) you perform the following steps:

Identify the required types of tests

Different types of test may be necessary to ensure that a unit has been thoroughly tested and all classes of defects have been identified and resolved.

These tests include:

• White-box tests (also known as glass-box or structure test). These tests rely upon knowledge of how the code is written for the development of test cases.  
• Black-box tests (also known as functional or specification test). These tests rely upon the use or intended use of the code for the development of test cases.
• Performance analysis tests. These tests monitor the execution of the code and identify performance related issues, such as possible performance bottlenecks and inefficiencies.
• Reliability tests. These tests monitor the execution of code and identify run-time errors which can result in crashes and poor performance, such as memory leakage, un-initiated memory, and memory stack errors.

Identify and Describe the Test Cases

Identify White-Box Tests

A white-box test verifies a unit's internal structure and requires that you apply your knowledge of how the unit is implemented. White-box testing requires knowledge of how the unit is designed internally.

Theoretically, you should test every possible path through the code, but that is possible only in very simple units. At the very least you should exercise every decision-to-decision path (DD-path) at least once because you are then executing all statements at least once. A decision is typically an if-statement, and a DD-path is a path between two decisions.

To get this level of test coverage, it is recommended that you choose test data so that every decision is evaluated in every possible way. Toward that end, the test cases should make sure that:

• Every Boolean expression is evaluated to true and false. For example the expression (a<3) OR (b>4) evaluates to four combinations of true/false
• Every infinite loop is exercised at least zero times, once, and more than once.

Use code-coverage tools to identify the code not exercised by your white box testing. Reliability testing should should be done simultaneously with your white-box testing.

Example:

Assume that you perform a structure test on a function member in the class Set of Integers. The test - with the help of a binary search - checks whether the set contains a given integer.

The member function and its corresponding flowchart. Dotted arrows illustrate how you can use two test cases to execute all the statements at least once.

Theoretically, for an operation to be thoroughly tested, the test case should traverse all the combinations of routes in the code. In member, there are three alternative routes inside the while-loop. The test case can traverse the loop either several times or not at all. If the test case does not traverse the loop at all, you will find only one route through the code. If it traverses the loop once, you will find three routes. If it traverses twice, you will find six routes, and so forth. Thus, the total number of routes will be 1+3+6+12+24+48+…, which in practice, is an unmanageable number of route combinations. That is why you must choose a subset of all these routes. In this example, you can use two test cases to execute all the statements. In one test case, you might choose Set of Integers = {1,5,7,8,11} and t = 3 as test data. In the other test case, you might choose Set of Integers = {1,5,7,8,11} and t = 8.

See Artifact:  Test Case.

Identify Black-Box Tests

The purpose of a black-box test is to verify the unit's specified behavior without looking at how the unit implements that behavior.   Black-box tests focus and rely upon the unit's input and output.

Equivalence partitioning is a technique for reducing the required number of tests. For every operation, you should identify the equivalence classes of the arguments and the object states. An equivalence class is a set of values for which an object is supposed to behave similarly. For example, a Set has three equivalence classes: empty, some element, and full.

Use code-coverage tools to identify the code not exercised by your white box testing. Reliability testing should should be done simultaneously with your black-box testing.

The next two subsections describe how to select test data for specific arguments.

Selecting Test Data for Input Arguments

An input argument is an argument used by an operation. You should select the following test data for each input argument in each operation.

• Normal values from each equivalence class.
• Values on the boundary of each equivalence class.
• Values outside the equivalence classes.
• Illegal values.

Remember to treat the object state as an input argument. If, for example, you test an operation add on an object Set, you must test add with values from all of Set's equivalence classes, that is, with a full Set, with some element in Set, and with an empty Set.

Selecting Test Data for Output Arguments

An output argument is an argument that an operation changes. An argument can be both an input and an output argument. Select input so that you get output according to each of the following.

• Normal values from each equivalence class.
• Values on the boundary for each equivalence class.
• Values outside the equivalence classes.
• Illegal values.

Remember to treat the object state as an output argument. If for example, you test an operation remove on a List, you must choose input values so that List is full, has some element, and is empty after the operation is performed (test with values from all its equivalence classes).

If the object is state-controlled (reacts differently depending on the object's state), you should use a state matrix such as the one in the following figure.

A state matrix for testing. You can test all combinations of state and stimuli on the basis of this matrix.

See Artifact:  Test Case.

Structure Test Procedures

The previously described test cases are insufficient for the implementation and execution of unit test. Proper structuring of  test procedures includes identifying the following information:

• Set-up:  how to create the condition(s) for the test case(s) that is (are) being tested and what data is needed (either as input or within the test database).
• Starting condition, state, or action for the structured test procedure

• Instructions for execution: the detailed exact steps / actions to be to implement / execute the tests
• Data values entered (or referenced test case)
• Expected result (or referenced test case)

• Evaluation of results:  the method and steps used to analyze the actual results obtained comparing them with the expected results
• Ending condition, state, or action for the structured test procedure

See Artifact:  Test Procedure.

Review and Assess Test Coverage

Test coverage measures are used to identify how complete the testing is or will be.

There are two methods of determining test coverage:

• Requirements based coverage (primarily used for black-box testing)
• Code based coverage (primarily used for white-box, performance analysis, and run-time error detection testing)

Both  identify the percentage of the total testable items that will be (or have been) tested, but they are collected or calculated differently.

• Requirements based coverage is based upon using use cases, requirements, use case flows, or test conditions as the measure of total test items and can be used during test design.
• Code based coverage uses the code generated as the total test item and measures a characteristic of the code that has been executed during testing (such as lines of code executed or the number of branches traversed). This type of coverage measurement can only be implemented after the code has been generated.

Identify the method to be used and state how the measurement will be collected, how the data should be interpreted, and how the metric will be used in the process.

Implement Unit Tests

The following steps are performed to create or acquire test scripts:

1. Review existing test scripts for potential use
2. Set-up the test environment
3. Initialize the environment
4. Create or acquire the test scripts:
   1. Record / capture:  for each structured test procedure, execute the test procedure to create a  new test script by following the steps / actions identified in the structured test procedure and using the appropriate recording techniques (to maximize reuse and minimize maintenance)
   2. Modifying existing scripts: edit the existing manually, or delete the non-required instructions and re-record the new instructions using the recording description above
   3. Programming:  for each structured test procedure, generate the instructions using the appropriate programming techniques
5. Continue to create or acquire test scripts until the desired / required test scripts have been created
6. Modify the test scripts as necessary

Test / debug test scripts:

Upon the completion of creating or acquiring test scripts, they should be tested / debugged to ensure the test scripts implement the tests appropriately and execute properly. This step should be performed using the same version of the software build used to create / acquire the test scripts.

The following steps are performed to test / debug test scripts:

1. Set-up the test environment (if necessary)
2. Re-initialize the environment
3. Execute the test scripts
4. Evaluate Results
5. Determine appropriate next action:
   1. Results as expected / desired: no actions necessary
   2. Unexpected results:  determine cause of problem and resolve

Execute Unit Test

To execute the tests, the following steps should be followed:

1. Set-up the test environment to ensure that all the needed components
2. Initialize the test environment to ensure all components are in the correct initial state for the start of testing
3. Execute the test procedures

Note:  executing the test procedures will vary dependent upon whether testing is automated or manual.

• Automated testing:  The test scripts created during the Implement Unit Test step are executed.
• Manual execution:  The structured test procedures developed during the Structure Test Procedure activity are used to manually execute test.

Evaluate Execution of Test

If testing terminates normally, continue with Evaluate Unit Test.

If testing terminates abnormally, do the following:

1. determine the actual cause of the problem
2. correct the problem
3. re-set-up test environment
4. re-initialize test environment
5. re-execute tests

Evaluate Unit Test

See Activity: Evaluate Test.

How to Test Inherited Operations

If an inherited operation does not work in the descendant, it is classified as an interaction problem between the descendant and the ancestor. You can verify the interaction among units when testing use cases. Do not test inherited operations when you test units. Inherited operations are tested when the use cases are tested.

An inherited operation can fail in two situations:

• The descendant class modifies instance (or member) variables for which the inherited operation assumes certain values.
• Operations in the ancestor invoke an operation implemented in the descendant.

You can avoid the first by forbidding ancestors to modify inherited instance (or member) variables other than through inherited operations. You can avoid the second by thoroughly testing the invoked operations.

How to Test Abstract Classes

Classes that are not instantiated, but exist only for others to inherit, must, of course, be tested. Exactly what that entails may not be obvious, since testing instances is not meaningful because by definition an abstract class has no instances. An abstract class can, however, be inherited, and instances of its descendants can be created. Thus, one goal of testing such classes is to determine if inheritance is possible and if any instances of descendant classes exist. A second goal is to determine whether potential calls to the abstract class itself (this in C++, self in Smalltalk) will get through. To test this, let the testing program create a descendant class that inherits the abstract class. The test program tests the unit by testing the descendant class.

The test program creates a descendant of the tested unit.

Testing and Polymorphism

Polymorphism is a programming language concept that makes the code easier to change, but it makes testing more difficult. In the following example you cannot test the code with every subclass of Shape. You must test this when you test the use cases.

An interesting effect of polymorphism in object-oriented languages is that every sending of a message in Smalltalk, and function call in C++ is a potential CASE statement.

Example:

Assume you have the following class hierarchy, and the class Shape has an operation Draw.

An example of object-oriented code. This is how the code would look if you declared a variable of class Shape and sent the stimulus Draw.

It appears that Draw is sent to an instance of Shape, but in reality it could also be to an instance of any of its descendants. You cannot decide from the code whether a Shape, a Circle, a Triangle or a Square will be called.

In a traditional language without inheritance, the code would look like this:

Here, it is obvious that all three branches in the case statement must be traversed.