Concepts: Quality

Quality is something we all strive for in our products, processes, and services.   Yet when asked to define it, describe it, or identify who is responsible for it, everyone has a different opinion.  Some of the common responses include:

or

Perhaps the most frequent reference to quality (related to software) is the remark regarding its absence:

These commonplace responses are telling, but they offer little room to rigorously examine quality and improve upon its execution.  These comments all illustrate the need to define quality in a manner in which it can be measured and achieved.

This page defines and discusses quality as it relates to software development and the software development process, and includes the following topics:

• Definition
• Requirements
• Measurement
• Process
• Ownership

Definition

The definition of quality (as taken from The American Heritage Dictionary of the English Language, 3rd Edition, Houghton Mifflin Co., c 1992, 1996) is:

This definition demonstrates that quality is not a single dimension, but many.   In order for us to use the definition and apply it to software development, the definition must be refined. 

Therefore, for the purposes in Rational Unified Process, quality is defined as:

Given this definition, achieving quality is not simply "meeting requirements" or producing a product that meets user needs, expectations, etc.  But rather, quality also includes identifying the measures and criteria (to demonstrate the achievement of quality), and the implementation of a process to ensure that the resulting product has achieved the desired degree of quality (and can be repeated and managed).

For software development, our concern for quality is focused in two areas:   product quality and process quality. 

Product quality is the quality of the principal product being produced (the software or system) and all the elements comprising it (components, sub-systems, architecture, etc.).  

Process quality refers to the degree to which an acceptable process (including measurements and criteria for quality) was implemented and adhered to in order to produce the product.   Additionally, process quality also is concerned with the quality of the artifacts (such as iteration plans, test plans, use-case realizations, design model, etc.) produced in support of the principal product.  The same concepts of product quality apply to these artifacts.

A common misconception is that quality can be added to, or tested into a product.   Just as a product cannot be produced if there is no description of what it is, what it needs to do, who uses and how, etc.  Quality, and its achievement, cannot be attained if it is not described,  measured, and part of the process of creating the product.

Another misconception is that quality is owned by or is the responsibility of one group, sometimes called Quality Assurance (other names include: Test, Quality Control, or Quality Engineering).  Quality is the responsibility of everyone.  Everyone is responsible for the quality of the products (or artifacts) they produce and the implementation of the process for which they are involved in.  More on this is discussed under the topic titled Ownership.

Additional misconceptions regarding quality, include:

These misconceptions, and further details regarding quality are described in greater detail in the following sections.

Requirements 

Product Quality:

Software products are described by their requirements, stated as use cases or supplemental requirements (such as marketing, performance, etc.).  To achieve quality, the requirements of the product must be known and must be stated in a manner that is clear, concise, and verifiable (testable). For software, not all requirements will be the target of testing by a test worker, (such as market penetration or sales revenue).  For those that will be the target of test, a test designer must be capable of specifying a method to verify that the requirement has been achieved (as stated), does not deviate from its intent, and is without flaw (defects).

See Concepts: Requirements for additional information.

See Artifact: Use Case and Artifact: Supplementary Specification for additional information.

See Guidelines: Test Case for additional information.

Process Quality:

The implementation of a process, the measurement of adherence to the process, and the quality of the artifacts (other than the principal product) is the responsibility of everyone involved. In general, however, everyone is responsible for implementing and adhering to the agreed upon process, and that the quality of the artifacts produced achieve the agreed upon quality. Specific workers, such as the  Project Manager, may have specific tasks that identify / impact process quality.  Additional details regarding process quality are discussed in Measurement, Process, and Ownership sections below.

See Workflow: Introduction to Project Management for additional information.

See Process Configuration for additional information regarding configuring Rational Unified Process.

Measurement 

Product Measurement

Stating the requirements in a clear, concise, and testable fashion is only part of achieving product quality.  It is also necessary to identify the measures and criteria that will be used to identify the desired level of quality and determine if it has been achieved. Measures describe the method used to capture the data used to asses quality, while criteria defines the level or point at which the product has achieved acceptable (or unacceptable) quality.

Defect analysis is the most common form of measurement used for product quality. 

Other forms of measurements will vary based upon the nature of the requirement, such as Usability requirements for ease of use, or failover / recovery requirements to ensure a system can operate for long periods and does not fail, or if a system fails, it can be recovered to a determined, acceptable state in an appropriate time period.

In addition to measuring the quality of the product being produced, it is important to measure the quality of the artifacts produced as part of the process.  The measurements and criteria for these artifacts are the same as those for the principal product.

See Concepts:  Key Measures of Test for additional information.

See Guidelines:  Metrics for additional information.

All measure use criteria to identified to determine the degree or level at which of acceptable quality is attained.  The level of acceptable quality is negotiable and variable, and needs to be determined and agreed upon early in the development lifecycle   For example, in the early iterations, a high number of application defects are acceptable, but not architectural ones.  In late iterations, only aesthetic defects are acceptable in the application. 

Criteria may be stated in many ways, usually includes more than one criteria.   Common criteria include:

• The number of defects identified and addressed (often this is stated as the number of defects by criticality, severity, or priority) that remain open (not fixed).
• The percentage of test coverage obtained (usually used with the defect criteria identified above).  Usually a minimum percentage of test coverage must be achieved before quality is measured.
• The time a specified action (use case, operation, or other event) takes to complete.   This is criteria is commonly used for Performance testing, Failover and recovery testing, or other tests in which time criticality is essential.
• Acceptability or satisfaction.  This criteria is usually used with subjective measures, such as usability or aesthetics. 

Process Measurement

Process quality is measured in two manners:

1. The degree of adherence to the standards, guidelines, and implementation of an accepted process
2. The quality of the artifacts produced (using product quality measures).

See Guidelines:  Metrics for additional information.

Process

Quality cannot be achieved if a process is not implemented, adhered to, and measured.   The purpose of Rational Unified Process is to provide a disciplined approach to assigning tasks and responsibilities within a development organization. Its goal is to ensure the production of high-quality software that meets the needs of its end users, within a predictable schedule and budget. The Rational Unified Process captures many of the best practices in modern software development in a form that can be tailorable for a wide range of projects and organizations. 

See Introduction to the Rational Unified Process for additional information.

Processes can be configured and quality (criteria for acceptability) can be negotiated based upon several factors.  The most common factor are:

• Risk (including liability)
• Market opportunities
• Revenue requirements
• Staffing or scheduling issues
• Budgets

Changes in the process and criteria for acceptability should be identified and agree upon at the outset of the project. 

See Workflow: Introduction to Project Management for additional information.

See Process Configuration for additional information regarding configuring Rational Unified Process.

See Management Guidelines: Risk for additional information.

Ownership

A common misconception of quality is that it is owned by, or is the responsibility of one group, sometimes called Quality Assurance (other names include: Test, Quality Control, or Quality Engineering). 

Quality is the responsibility of everyone.   Each worker has the responsibility to contribute to quality in the following ways:

• Product quality - the contribution to the overall achievement of quality in principal product being produced.
• Process quality - the definition and achievement of quality in the process artifacts and activities for which they are involved.

Everyone shares in the responsibility and glory for the achievement of a high-quality product (or the shame of low-quality).  But only those directly involved in specific process component are responsible for the glory (or shame) for the quality of those process components (and the artifacts).

See Workers Overview for additional information.