Purpose

This tool mentor describes how to use Rational LogViewer to evaluate test execution.

Related Rational Unified Process activities:

Overview

This Tool Mentor is applicable when running Microsoft Windows 95/98/NT 4.0.

Use the Rational LogViewer after you execute (play back) a test script to determine the next appropriate step.

The following steps are performed using the LogViewer:

    1. Evaluate Execution of Test
    2. Recover from Halted Tests
    3. Verify Test Results
    4. Investigate Unexpected Results

1. Evaluate Execution of Test To top of page

NOTE: The LogViewer can only be used to evaluate the execution of test if results were written to the test log. To ensure that data is written to the test log, see the section Setting GUI script playback options in the Tool Mentor: Setting up the Test Environment.

  1. Open the LogViewer:
    • From Rational Robot or TestManager, click Tools -> Rational Test -> LogViewer or click on the LogViewer button.
    • From Start -> Programs, click Rational program group -> Rational Test 7-> LogViewer.

  2. Click File -> Open or click the Open button.
  3. Select a log file and click OK.
  4. Determine the next step, based on the information in the Log Event and Result columns. Specifically, look for the following:

    • In the Log Event column:

    User Abort: indicates someone interrupted the playback of the script.

    Script Command Failure: indicates a script failed, that the script command was unable to execute.

    General Protection Fault: indicates that a GPF was encountered during script playback.

    Unexpected Active Window: indicates that an active window, other than the one expected, was encountered during script playback.

    • In the Results column:

    Pass: (on green background) indicates the Log Event occurred or executed as expected and was completed successfully.

    Warning: (on yellow background) indicates the Log Event occurred expectedly, and the script was able to recover and continue successfully.

    Fail: (on red background) indicates the Log Event occurred unexpectedly and the test script was unable to complete successfully. Additional information for some types of events are included in the Results column.

  5. To locate failures, click View -> Next Failure or View -> First Failure.  Alternately, click the Next Failure or First Failure buttons.
  6. If your test scripts executed completely without any failures or warnings, the Results column will contain the word Pass on a green background for each event listed in the Log Event column. If all the results pass, then test execution is complete and you should generate the Execution Test Coverage report and continue with the Activity: Evaluate Test. 

See Tool Mentor: Using TestManager to Review and Evaluate Requirements-based Test Coverage.

  1. If your test scripts executed completely (the only failures identified were due to verification points, or yellow warnings) proceed to the tool mentor step: Verify the Results below.
  2. If your test scripts did not execute completely, continue with the next tool mentor step: Recover from halted tests.

2. Recover from halted tests To top of page

Use the LogViewer to provide additional information when test execution is halted due to test script related errors.

To view additional information for a log event:

  1. Select the log event and click View > Log Event Properties.
  2. Click the Results tab to display a description of the error.
  3. For each failure, review the properties and edit the test script(s) as necessary.

    To open the test script from the LogViewer, click View -> Script. 

See Tool Mentor: Using Rational Robot to Record or Program Test Scripts

  1. After editing the script(s), reset the environment to its appropriate initial state and reexecute the tests.

    See Tool Mentor: Setting Up the Test Environment.

3. Verify Test Results To top of page

Perform this step when test scripts have executed completely, without any script or system failures, but there are warnings and verification point failures. In this step you review each verification point failure to identify the cause of the failure and determine the appropriate corrective action, such as logging a defect (the defect is in the target-of-test) or modifying the test script (the defect was in the test script).

Use one of the Comparators to analyze the verification points and if necessary, change the baseline values. There are four Comparators that you can use:

  • Grid Comparator (used with the Menu, Clipboard, and Object Data verification points)
  • Object Properties Comparator (used with the Object Property verification point)
  • Text Comparator (used with the Alphanumeric verification point)
  • Image Comparator (used with the Window Image and Region Image verification points)

The Comparator window contains two panes, the Data pane and the Differences list.

  • The Data pane displays the data from recording and playback of a verification point in Robot. The Baseline grid shows the data file that serves as an expected file for a comparison. The Actual data file shows data from the current playback in Robot. You can control the display of the Baseline and Actual data files using the View -> Baseline, View -> Actual, View -> Both Horizontal, and View -> Both Vertical menu or toolbar commands.
  • The Differences list, in the lower pane, lists the items that have differences between the Baseline and the Actual files.

To open a Comparator from the LogViewer:

  1. Select the verification point failure in the LogViewer and double-click.   Alternately, click View -> Verification Point.
  2. Click an item in the Differences list to highlight the difference in the upper pane.
  3. Scroll through the data pane vertically and horizontally as necessary.  To synchronize the scrolling between baseline and actual, select View -> Synchronize Scrolling.
  4. Determine if the difference is a defect in the target-of-test (the baseline is correct), or if it is a defect in the test verification point (the actual value is correct).
  5. If the baseline value is correct, log a defect. See Tool Mentor: Submitting a Change Request Form to use ClearQuest to log (enter) a defect.
  6. If the actual value is correct, you can either edit the item in the data pane (except for the Image Comparator), or replace all baseline values with the actual values (all comparators).
  • To replace all the values in the baseline with the actual, select File -> Replace Baseline with Actual. Alternately, click Replace Baseline on the toolbar.
  • To edit individual values in the baseline, select the baseline value, double click the item, and enter or select a new value. Alternately, select the actual value and select Edit -> Copy to baseline.
  • Select File -> Save Baseline, or click Save Baseline to save your changes.

4.   Investigate Unexpected Results To top of page

Use the Comparators to analyze any unexpected results. Unexpected results include:

  • Test verification failures (described above)
  • Unexpected windows (a window was active other than the expected active window)
  • Missing windows (expected window was not found)

Test Verification Failures

  • To investigate test verification failures, follow the instructions in the tool mentor step above: Verify Test Results

Unexpected Active Windows

NOTE: You can only perform the steps described below if Robot has been instructed to capture or respond to Unexpected Active Windows. See step Set GUI script playback options in Tool Mentor: Setting Up the Test Environment for instructions on setting Robot to capture / respond to Unexpected Active Windows.

Two pieces of information are provided by the LogViewer and Image Comparator.  In the LogViewer, the log event identified an Unexpected Active Window and the method used to respond to it (as set in the GUI playback options).  

The LogViewer and the Image Comparator both provide you with information about Unexepected Active Windows. The LogViewer shows the log event identified as an Unexpected Active Window and the method used to respond to it (as set in the GUI playback options). The Image Comparator displays a bitmap image of what was displayed on your screen when the Unexpected Active Window was encountered. 

To investigate unexpected failures:

  1. Enlarge the Log Event column to view the response method and double click on the Unexpected Active Window ...   log event to view the Image Comparator.
  2. Determine the cause of the Unexpected Active Window.
  3. If necessary,  click View -> Script in the LogViewer and edit the script. 

See Tool Mentor: Using Rational Robot to Record or Program Test Scripts

  1. After editing the script(s), the environment should be re-set to its appropriate initial state and the tests re-executed.

See Tool Mentor: Setting Up the Test Environment.

Missing Windows

NOTE: You can only perform the steps described below if Robot has been instructed to Save window positions and Auto record window size.  See the step Set GUI script record options in Tool Mentor: Setting Up the Test Environment.

The Log Event column identifies the event and type of warning, such as Unable to find window or Window size is different.

To view the log event properties:

  1. Select the log event and click View -> Log event properties.
  2. Click the Result tab to view more details.
  3. If necessary, click View -> Script in the LogViewer and edit the script. 

See Tool Mentor: Using Rational Robot to Record or Program Test Scripts

  1. After editing the script(s), reset the environment to its appropriate initial state and reexecute the tests.

See Tool Mentor: Setting Up the Test Environment.

 

Display Rational Unified Process using frames

 

© Rational Software Corporation 1998 Rational Unified Process 5.1 (build 43)