Tool Mentor: Using TestFactoryTM
to Measure and Evaluate Code-based Test Coverage
Purpose
This tool mentor describes how to use Rational TestFactory to measure and evaluate
code-based test coverage for a Robot script that tests an application written in Visual
Basic.
Related Rational Unified Process activities:
Overview
This Tool Mentor is applicable when running Windows NT 4.0.
To use TestFactory to evaluate code-based test coverage for a Robot script:
- Instrument the application-under-test.
- Play back the script.
- Review code coverage values for
the script.
- Review the lines of source code
covered.
To measure the code coverage of an executed script, you need to add instrumentation
points to the application-under-test. The instrumentation points are counters that tally
which parts of the code a script executes. You can instrument either the source code or
the object code of the application.
If you have access to source code files, it is recommended that you instrument the
source code for the application-under-test. The reasons for this are as follows:
- TestFactory instruments source code at the branch level, while it instruments object
code at the statement level. Branch coverage takes into account the if, then,
and else conditions in instrumented source code, therefore TestFactory
calculates more accurate code coverage values for instrumented source code than it can for
instrumented object code.
Note: In some cases, the code coverage values calculated for an
application-under-test with instrumented object code are slightly higher than those
calculated for the same application-under-test with instrumented source code. The higher
value is inflated as a result of statement-level object code instrumentation.
- With access to instrumented source code, TestFactory provides more detailed coverage
information on scripts than it can with instrumented object code.
To instrument object code, you must have an uncompressed version of the executable
file, with symbolic debug information in a .pdb file.
To instrument the source code of the
application-under-test:
- Start Rational TestFactory.
- Click Tools > Instrument, or click the Instrument button on the
toolbar.
- In the Instrument Source Code dialog box, click Add.
- Select the Visual Basic project file associated with the application-under-test, and
then click OK.
- If the application-under-test calls supplemental executable files and if you want to
include these files in the code coverage metrics, do the following for each executable
file:
- Click Add.
- Select the Visual Basic project file associated with the executable file, and then click
OK.
- Click Instrument.
- When instrumentation and rebuild are complete, click Close.
To instrument the object code of the
application-under-test:
- Obtain a copy of the application's .exe and .pdb files from a build that was compiled
with the No Optimization and Create Symbolic Debug Info options selected.
- Start Rational TestFactory.
- Click Tools > Options.
- On the Instrumentation line, click Object code.
- Click OK.
- Click Tools > Instrument, or click the Instrument button on the
toolbar.
- In the Instrument Object Code dialog box, click Add.
- Select the Visual Basic executable file associated with the application-under-test, and
then click OK.
- If the application-under-test calls supplemental executable files and if you want to
include these files in the code coverage metrics, do the following for each executable
file:
- Click Add.
- Select the Visual Basic executable file, and then click OK.
- Click Instrument.
- When instrumentation is complete, click Close.
Play back the Robot script in TestFactory. When you play back the script, two actions
occur:
- TestFactory passes the script to Robot to execute.
- As the script executes, TestFactory monitors the instrumentation points in the code to
tally the code coverage information.
To play back the script:
- In left pane of the TestFactory main window, expand the Robot Scripts folder.
- Under the folder, click the script that you want to play back.
- Click Script > Play Back, or click the Play Back Script button on the
toolbar.
Review the code coverage information in TestFactory. When the script you played back is
the active object in the left pane, the Coverage tab in the right pane displays an
overall code coverage valuefor both the application and all the instrumented
supplemental executable file(s) called by the application. The tab also displays coverage
information for each file and subroutine in the application.
To review the code coverage values for a script:
- If the script that you played back in TestFactory is not the active object in the left
pane, expand the Robot Scripts folder, and then click the script.
- In the right pane, click the Coverage tab.
- Review the overall code coverage value.
- Expand the code coverage line.
- Review the code coverage values for the source files.
- Expand the code coverage line for a source file.
- Review the code coverage values for the subroutines in the source file.
Use the Coverage Browser in TestFactory to review the lines of source code that the
script covered. If source code is available for the application-under-test, you can review
each of the source files and examine which source statements were executed and which were
not.
To review the lines of source code covered:
- Review code coverage values for
the script.
- On the Coverage tab, determine the source file for which you want to review the
code coverage and the first subroutine in the source file that you want to examine.
- Double-click the subroutine line that you chose in step 2. The Coverage Browser window
opens, with the source file scrolled to display the subroutine at the top of the window.
- In the Coverage Browser window, use the buttons and boxes in the top area of the window
to do the following:
- To jump to the next line of code not covered by the script, click the Next Not
Covered button.
- To go to a specific line of code, type the line number in the left box, and then click
the Go To Line button.
- To go to a line of code that contains a specific text string, type the text in the right
box, and then click the Find Text button.
- To review all the lines of code in the file, use the scroll box. To determine the color
code used for the different segments of text, click the Coverage Text Colors
button.
- Close the Coverage Browser window.
- To review the lines of code that the script covers in another source file, return to
step 2.
| |

|