Tool Mentor: Using Rational Robot^TM to Record or Program Test Scripts

Purpose

This tool mentor describes how to use Rational Robot^TM to record test scripts and how to extend them by editing the scripts after they have been recorded.

Related Rational Unified Process activities:

• Section "Record or Program Test Scripts" of Activity: Implement Test

Overview

This Tool Mentor is applicable when running Microsoft Windows 95/98/NT 4.0. 

To record and extend a script using Robot:

1. Record the script in Robot.
2. Insert a verification point.
3. Edit the script if necessary.
4. Play the script back.
5. View the results in the LogViewer.
6. Analyze verification point results with the comparators.

1.   Record the script in Robot

When you record a script, Robot records:

• Your actions as you interact with the application-under-test. These user actions include keystrokes and mouse clicks that help you navigate around the application.
• Verification points that you create to capture and save information about specific objects. A verification point is a point in a script that you create to confirm the state or behavior of an object across builds. During recording, the verification point captures object information and stores it as the baseline. During playback, the verification point recaptures the object information and compares it to the baseline.

The recorded script establishes the baseline of expected behavior for the application-under-test. When revised builds of the application become available, you can play back the script to test new builds of the application against the established baseline in a fraction of the time it would take to perform the testing manually.

The Robot Object-Oriented Recording technology examines objects in the application-under-test at the Windows layer during recording and playback. As a result, script playback does not usually depend on absolute screen coordinates. Object-Oriented Recording insulates the script from minor user interface changes and simplifies script maintenance.

To record a new script:

2. Insert a verification point

To insert a verification point while recording or editing a script:

1. Do one of the following:

• If recording, click the Display GUI Insert Toolbar button on the GUI Record toolbar.
• If editing, position the pointer in the script and click the Display GUI Insert Toolbar button on the Standard toolbar.

2. Click a verification point button on the GUI Insert toolbar.

The verification point is named with the verification point type (and a number if there is more than one of the same type in the script).

3. Edit the name as appropriate.
4. Optionally, set the wait state options.
5. Optionally, set the expected results option.
6. Click OK.
   
7. If prompted to select an object, drag the object finder tool over the desired object and release the left mouse button to select the object.  Optionally, the object may be selected from a list of objects by clicking on the Browse button. When the desired object is selected, click OK.

For further details, see the Robot online Help and the chapter Creating Verification Points in GUI Scripts in the Using Rational Robot manual.

3. Edit the script if necessary

You can edit existing scripts by manually changing the text of a script or by adding new features. For example, you can:

• Edit the text of a script (delete a line, move text, etc.)
• Add a new user action to an existing script (select a menu command, click a button, etc.)
• Add a new feature to an existing script (add verification points, comments, etc.)
• Go to a line with a compiler error (go to a specific line number, find the next error, etc.)

For details, see the Robot online Help and the chapter Editing, Compiling, and Debugging Scripts in the Using Rational Robot manual.

4. Play the script back

When you play back a script, Robot repeats your recorded actions and automates the software testing. With automation, you can test each new build of your application faster and more thoroughly than by testing it manually. This decreases testing time and increases both coverage and overall consistency.

There are two general phases of script playback:

• Test Implementation  – Play back the scripts to verify that they work as intended, using the same build of the application-under-test that you used to record. This verifies the baseline of expected behavior for the application-under-test.
• Test Execution and Regression – Play back the scripts to compare the latest build of the application-under-test to the baseline established during test implementation. Ongoing play back of all test scripts during test execution reveals any differences that may have been introduced into the application since the last build. These differences can be evaluated to determine if they are actual defects or deliberate changes.

Before playback, Robot automatically compiles a script if it has changed since it last ran.

For information on preparing the test environment for script playback, see the Tool Mentor: Setting Up the Test Environment.

To play back a script:

1. Click the Playback Script button on the toolbar. The Playback dialog box will appear.
2. Type a script name or select it from the list.

The query determines which names and fields appears in the list. To change the list, select a query from the Query drop-down list. You create new queries in TestManager.

3. To change the Playback options, click Options. When finished, click OK. For information on playback options, see the Tool Mentor: Setting Up the Test Environment.

4. Click OK to continue.
5. If the Specify Log Information dialog box appears, do the following:

• Select a build from the list. To create a new build, click the build button on the right. Builds can also be created in TestManager.
• Select a log folder from the list. To create a new log folder, click the log folder button on the right. Log folders can also be created in TestManager.
• Accept the default log name, which is the same as the script name, or type a new name.
• Click OK.

6. If a prompt appears asking if you want to overwrite the log, click Yes.

Note: To stop playback of a script, press the F11 key. Robot recognizes the F11 key only when playing back Object-Oriented commands. The F11 key does not stop playback during low-level actions.

5.   View the results in the LogViewer

After playback completes, use the Rational LogViewer to view the playback results, including verification point failures, procedural failures, aborts, and any additional playback information.

To control the log information and display of the LogViewer, you can set options in the Log tab in the GUI Playback Options dialog box:

• To update the repository with the playback results, select Output playback results to log. Click the amount of detail to output to the log.
• To display the LogViewer automatically after playback, select View log after playback.

If this is not selected, you can display the LogViewer after playback by clicking Tools > Rational Test > LogViewer.

If the playback of the test scripts resulted in any errors, the specific cause of the error must be identified. 

Scripts which contain errors are identified in the LogViewer by the word Fail (enclosed in a red box) in the Results column, or Warning (in a yellow box).  There are several classes of errors to be addressed:  fatal errors, script errors, and verification point failures.  Fatal errors typically require that the test environment should be investigated and the failure resolved.   Script errors typically require test script maintenance.  Verification point failures (during test implementation) typically require changes to the verification point parameters or indicate a difference between the desired state of the application-under-test when the script was recorded versus when it was played back.   See the next section for analyzing verification points.

Additional information regarding the error condition may be viewed in the Log Event Properties window. Open this window by clicking View > Log Event Properties.  Selecting the Results tab will display a description of the error and additional information regarding the type of error encountered. 

For each script in which a failure (or warning) occurred, review the information regarding the failure and if necessary, edit the test script (see Tool Mentor:  Using Rational Robot^TM to Record or Program Test Scripts).  The test script (and Robot) may be opened from the LogViewer by clicking View > Script.

After editing the script, the environment should be re-set to its appropriate initial state (see Tool Mentor: Setting Up the Test Environment.) and the tests re-executed (Play the script back.)

6.   Analyze the verification point results with the comparators

Use the Comparators to analyze verification point results. To open a Comparator from the LogViewer:

1. In the Log Event column, double-click a verification point.  The appropriate Comparator dialog window will appear. The comparators may also be opened from Robot by double-clicking a verification point in the Asset pane. However, when you open a Comparator this way, you can view only the baseline file.
   
2. Differences between the baseline and the actual results will be highlighted. 
3. Review the differences and determine the appropriate course of action, including:
   (Note:  the available actions will differ based upon the type of verification point)

• Copy Property to baseline - copy the current individual actual property and over-write the baseline property.
• Copy All Properties to baseline - copy all the actual property values and over-write all the baseline properties.
• Edit individual property (menu, value, etc.) by double-clicking the property (menu, value, etc.) and entering a different value.
• Editing the properties list (to identify which properties should be captured by the verification point).
• Create / modify masks.

4. When finished making the appropriate changes, click the Exit button to close the comparator and save the changes.

After editing the script and / or verification points, the environment should be re-set to its appropriate initial state (see Tool Mentor: Setting Up the Test Environment.) and the tests re-executed (Play the script back.)