Introduction

Purpose

This paper provides guidance for a ClearCase implementation for component-based development. The primary focus is on using ClearCase during the construction phase of a project. The construction phase begins after the project requirements have been gathered and a system architecture has been defined.

This document recommends one way of successfully organizing and managing component-based software development using ClearCase. It is not the only way to achieve success with ClearCase. It is up to you, working with your customer, to determine if the approach described here is a good fit. This paper discusses some of the situations that may lead you to recommend a modified or different approach.

The implementation details in this paper assume you are in a Windows NT development environment using ClearCase 3.2 with view profiles. The ClearCase concepts are identical across platforms but the implementation details vary. A separate document will address UNIX and mixed environments.

Audience

If you have general knowledge about the software engineering process and you are interested in configuration management, you can benefit from reading this paper.  If you plan to implement the recommendations made in this paper, you need to start with at least a basic understanding of ClearCase, for example, the level of understanding you would acquire in the ClearCase Fundamentals course.

Organization

• Introduction (this section) introduces the paper and identifies the audience.
• Conceptual Overview lays the conceptual groundwork for the configuration management process described in the remainder of the paper. This section describes the workflow and the types of configuration items.
• Establishing the Initial CM Environment describes how to set up a configuration management (CM) environment at the start of a project. It describes how to use the system architecture as a basis for the structure of the CM environment. It also suggests a directory structure.
• Managing a Software Project in a ClearCase Environment describes how to use ClearCase to manage a CM environment for projects. It describes how developers typically interact with ClearCase. It also discusses two approaches to promotion.
• Referencing Subsystems between Projects discusses how to share subsystems among multiple project teams.
• Advanced Topics briefly describes working in a more complex environment.
• Appendix A, Glossary defines common CM terminology, and where appropriate, describes the mapping to ClearCase terminology.
• Appendix B, References lists additional reading referenced throughout this paper.
• Appendix C, Process Support provides the code for a trigger which supports the configuration management model described in this paper.

Conceptual Overview

This section provides an overview of configuration management as part of a software development project. Subsequent sections both elaborate on the material in this section and provide implementation details. The responsible people and their activities are illustrated in Figure 1. The structural relationship of configuration items is shown in Figure 2.



Figure 1. Process Overview

Preparing for the Construction Phase

Typically, software development requirements and budgets are established at project inception and evolve during elaboration. The project architect uses the requirements to create a system architecture or to modify an existing architecture. The system architecture serves as a logical framework to satisfy the requirements. Models represent the system architecture, providing different views¹ of the system. Of these views, the most important for CM is the implementation view. The implementation view maps logical system packages to the physical files and directories that will be placed under version control.
 

1The term view as used here is a Rational Unified Process term which refers to an abstraction of a system architecture. This abstraction is seen from a given perspective or vantage point and omits entities that are not relevant to this perspective. This single use of view is not related to the ClearCase view object. Elsewhere in this paper, any mention of the term view refers to a ClearCase view.

Defining Subsystems

A subsystem is a group of files and directories that implement one or more logical packages. These files and directories are versioned, shared, and released as a single unit. A large system typically consists of many subsystems. A small system may be contained by only one subsystem if that system is the only piece that is versioned, shared, and released as a whole.

Creating Repositories

1. They conceptually map the system architecture to subsystems.
2. They map these subsystems to CM repositories.
3. The CM Administrator creates the CM repositories and establishes the project directory structure.
4. The CM Administrator or another project member imports existing files and directories into the repository to create an initial set of development configuration items.

Defining Working Configurations

• Which subsystems make up the working set for the project. A working set is the list of subsystems which must be referenced or modified to do a piece of work. This list may or may not be the entire system.
• The specific versions of these subsystems that can be modified or referenced.

Development Work

Each developer creates a private workspace which is isolated from the workspaces of other team members. A workspace provides a place for a developer to make changes to the system without seeing or affecting other developers’ changes. The workspace configuration is defined by associating it with one of the workspace profiles that the project leader defined (see Defining Working Configurations). A developer who is done with a piece of work can promote changes to the project’s integration workspace. The developer can also update their own workspace to see changes that other developers have promoted into the project integration workspace.

Integrating and Building

The project integrator use two workspaces, the integration workspace and the build workspace, to integrate changes, perform builds, and create project baselines:

• The integration workspace is used to select a fixed set of versions that have been promoted and to identify candidate build versions.
• The build workspace is used to perform the build itself.

Releasing a Subsystem

The configuration manager places all product deliverables under version control. The product deliverables may reside in one of the development repositories or may reside in a separate repository reserved for deliverables. These configuration items are also associated with a baseline. Other development teams can select and share deliverable or source configuration items by referring to the baseline.

Process Iterations

• The cycle of making a change, promoting the change, building, creating a baseline, and updating the view represents on-going development.
• The cycle of making a change, promoting the change, building, creating a baseline, and releasing the software represents iterative releases. It also represents the cycle of sharing between project teams working on different subsystems and contributing to a common system release.

Figure 2. Structural Overview

Establishing the Initial CM Environment

This section describes how to set up a configuration management environment. It discusses establishing repositories, mapping the system architecture to subsystems, and mapping subsystems to a development directory structure.

Creating the Initial System Repository

During early phases of a project, project members create artifacts which should be placed under version control. (When you plan an item under version control, it becomes a configuration item.) For example, configuration items include RequisitePro documents that describe the system requirements, and Rose model files that describe the system architecture.

We assume that you have created a CM system repository to contain the configuration items for those artifacts produced at the beginning of any project. In ClearCase, a CM repository is called a VOB (versioned object base). For more details about creating VOBs, see Creating the CM Repositories. See Defining the Product Directory Structure for suggestions about directory naming.

Mapping the Architecture to Subsystems

Using traditional structural software design methods, you typically decompose a system into a set of high-level functions which you in turn decompose into sub-functions. By contrast, when you use object-oriented design methods and the UML in particular, you typically decompose a system by grouping modeling elements (typically classes) into packages.

• You can build, test, and release subsystems separately, reducing system complexity.
• You can develop iteratively, which is especially helpful in an environment with overlapping releases.
• Groups can share subsystems. Teams can share stable versions of subsystems with other project teams and avoid the problems of working with unstable code.
• Development teams can be organized by subsystem. Depending upon the size of the team and the system being developed, the team can be further organized so that developers are assigned to one or more components in a single subsystem. The goal is to have a minimum number of developers working in the same space. This technique isolates development work, reducing parallel development issues. Packages and subsystems provide the framework for this.

Figure 3. Architecture of a typical system.

• A team of 8 developers is responsible for all the graphic user interfaces.
• A team of 10 developers is responsible for the modeler.
• A team of 12 developers is responsible for the storage facilities.

Mapping Subsystems to CM Repositories

• How many subsystems are in your project?

• How many elements are stored in any single VOB? (How many files make up the subsystem)?
• 
  How many concurrent users will access the elements in a VOB? (How many users need concurrent access to a particular subsystem)?

• Does a geographically distant site need access to a subsystem?

Creating the CM Repositories

The CM Administrator typically creates the VOBs. For details about creating VOBs, see Setting Up ClearCase VOBs in the ClearCase Administrator’s Manual. The same manual also contains details on selecting a VOB server machine and sizing requirements. If you are a Rational technical representative, refer to Appendix B for pointers to white papers about server sizing.

Essential VOB Maintenance

Since VOBs contain all the elements and version history for the entire system, it is critical that your system administrator backs them up. For more information, read about VOB and View Backup and Restore in the ClearCase Administrator’s Manual.

Defining the Product Directory Structure

Table 1. Suggested directory structure

Managing a Software Project in a ClearCase Environment

This section enumerates the steps for setting up and managing a software project using ClearCase. The goal of the project can be to develop a new system or to deliver a modified version of an existing system. For small systems there may be one single project team. For large systems, there may be multiple project teams. This section is focused on an individual project team of developers. Later sections discuss how multiple project teams can interact when working on a very large system.

Allocating Subsystems to the Project Teams

1. Creates a project label type in all the VOBs (subsystems) which that project may modify or reference.
2. Applies the label to all elements in the project VOBs.

• <PROJECT>_BUILD_CANDIDATE identifies a potential baseline of versions which will be built and smoke-tested. This label will be applied to all the versions that have been promoted for integration. It identifies the fixed set of versions ready to be built. This label is also applied to referenced subsystems to identify the version of those subsystems which your project should reference during a build.

Table 2. Recommended build label

Creating and Maintaining View Profiles

• Contains information about which VOBs (subsystems) need to be available.
• Specifies where to create and promote versions (in ClearCase terms, on which branch you will work)
• Identify the stable baselines that are available for the project team.

1. Name the view profile descriptively, for example, thorn2k_release1 or thorn2k_gui_release1.
2. Provide a description of the view profile. The description helps a developer pick the right view profile. The description should describe the type of work done using that view profile (for example, for development for release 1 of Thorn 2000).
3. Describe the scope of work for the view profile – what subsystems may be modified or referenced during development. Since each subsystem is stored in its own VOB, select the VOBs which pertain to the subsystems that your project will need to modify and reference. ClearCase will automatically ensure that these VOBs are available to users who work with this view profile.
4. [Optional] Large systems may have one central administrative VOB. The administrative VOB contains all meta-data used by other project repositories. As described in Allocating Subsystems to Project Teams, we recommend that you use the system repository (or system VOB) as the administrative VOB. For more information about setting up and maintaining administrative VOBs see Administrative VOBs and Global Types in the ClearCase Administrator’s Manual.
5. Select the branch on which the project work will be performed. A ClearCase branch contains a linear sequence of versions. The branch is the location where you want new versions to be created. Each element in ClearCase has an initial branch, main. You are prompted to select the main branch or another branch for development work. In general, select another branch. See When should you work on the main branch? later in this section for more information.
6. A branch other than main is called the integration branch for your project. You use an integration branch to collect changes made by all project developers.
7. 
   When you designate an integration branch for your project, you are prompted for a checkpoint label. The label you specify must identify a baseline of the entire system. This is the initial project baseline. Because you are storing each subsystem in its own VOB, the label applied to one version of every element in that VOB forms a baseline for that subsystem. A label applied to all VOBs serves as a system baseline label.
   When you supply or select the checkpoint label, you are identifying the baseline of the entire system from which you want to start your work. If you do not have an existing checkpoint label to use, then you should either create one or refer to Step 5 to determine whether you should be working on the main branch. Creating a Baseline for a Subsystem and the System discusses how to apply labels to subsystems and how to identify a single baseline for the entire subsystem.
8. Identify the integration branch for your project. Each project team should use its own project integration branch. For example, if you are working on release 2 and have two teams A and B, create two branch types release2_a and release2_b. If both projects use the same branch, then both teams will see promoted changes for the other team. By using different branches, you can defer seeing another team’s changes until the team reaches a stable point. Create the project branch type if it does not already exist.

• You are starting to develop a new system from scratch.
• You are developing a system with one project team. Only one release of the system is developed at the same time. You do not need to support previous releases.

Figure 4. Development on branches

Creating Development Views

• For the usage model described in this document, developers should create snapshot views. A snapshot view provides the developer with a stable, unchanging working environment. When a developer wants to see changes made by other team members, she updates her view. This style of working is characterized as a pull model.
• ClearCase also supports dynamic views. Dynamic views offer immediate updates because no local copies of files are made. Dynamic views also offer build auditing. The tradeoff of using a dynamic view is the expense (in terms of speed and network reliability) of working over a network. In the model described in this document dynamic views are used for the integration view and build views.

• You have limited client-side disk space
• You would like to audit development builds
• You want to take advantage of derived object sharing
• The development team must work with the latest versions of the code

Working with Configuration Items

1. Check out the files that you want to change.
2. Make changes.
3. Perform unit tests to verify the changes.
4. Check in the changes.
5. Promote the changes, either as part of Step 4, or as a separate step.
6. Periodically update your snapshot view.

• Working in the View
• Updating a Snapshot View
• Using Snapshot Views with Microsoft® Visual Studio 97
• Working on a Team

Coordinating Team Development Efforts

• Making quick stabilization changes on the integration branch
• Making small changes which require 3 or fewer days of effort
• Coordinating small teams of fewer than 8 developers

• You cannot create intermediate versions of partially completed changes. As soon as you check in the file, the partial changes become visible to everyone.
• You cannot check in work in progress, so that intermediate work may not be backed up.
• Checkouts which last a long time can hinder parallel development on shared files. (ClearCase does permit unreserved checkouts on the same branch. However, when you check in from an unreserved checkout, you may be required to merge your changes.
• If you update a snapshot view before completing a change, your working environment may become inconsistent.

• You want the ability to checkpoint your work
• You are making changes that will likely take a long time
• You are making changes that may destabilize the project (for example, a database schema change or an interface change)
• You are making changes that may not be included in the current release

• You are working on a different branch, meaning you will need to perform a merge to promote your changes. While this is usually a trivial operation, it may require manual intervention if others have made changes to the same file while you were working on the branch.
• Unless your CM administrator installs a ClearCase trigger, you will not be notified if someone else is also working on the same element on their own private branch (See Appendix C).

• A name for the branch you are creating.
• Answer no when asked if you want to base the private branch on the latest versions as of the current time.
• 
  There are times when you may want to use a time rule as a branch point. Typically, you finish a private branch, you want to start work on a new change, you want to see the change you just finished, and the change you just finished has not been included in a project baseline. In this case, you can choose to start a private branch from the latest versions as of the current time. The disadvantage to this approach is that these latest versions may not constitute a stable set of sources. For development teams who have the discipline to maintain a stable integration branch, this is a viable option.
• Select a View Profile checkpoint label.
• 
  The View Profile checkpoint labels identify versions that serve as the branch point when creating instances of your new private branch. The labels listed here identify stable starting points on which your private branch will be based. In general, choose the default, which is the last label added to the view profile by your project leader or CM administrator and which identifies the latest known stable point on the integration branch.

• Leave the current set of changes on the private branch
• 
  If you select this option, all work is left on the private branch, as-is, but the current view no longer selects those particular versions. Select this option when you have worked on a change, but you want to abandon it either temporarily or permanently. The view is reset to select versions on the integration branch. It no longer selects the changes you made on the private branch.
• Merge the current set of changes onto the integration branch
• 
  You will usually select this option to promote your changes to the integration branch. The Merge Manager starts, seeded with all the information it needs to start the merge. It presents a list of all versions that require merging as part of the current change. ClearCase can handle all trivial merges automatically. Non-trivial merges may require your intervention. Remember to check in your changes on the integration branch after you have completed the merge.

1. Check in any files that are currently checked out.
2. Perform a Finish Private Branch operation. Select the suspend option without merging your changes. This resets your view back to the integration branch. For efficiency, you can cancel the update to your view.
3. Perform a Setup Private Branch operation. Select the original settings, but select the new baseline label you want to update to. Answer yes to the warning that you are selecting an existing branch.
4. 
   You are now viewing the new baseline for any files you have not modified. The next few steps describe how to use the Merge Manager to resolve any conflicts in the files you modified.
5. Start the Merge Manager from ClearCase Home Base.
6. Select all the VOBs in which you have made changes. When in doubt, include it!
7. Perform the merge based on a label, and select the same baseline label you just updated to.
8. Merge all files that the Merge Manager lists. These are the files which have been updated by someone else since you started working on your private branch. In this step, you are integrating their changes with your own.
9. Test and check in.

Creating a Baseline for a Subsystem and the System

• A baseline provides a stable point from which individual team members can create private branches, isolating themselves from ongoing changes to the project's main branch.
• A baseline is a stable point from which new projects can create a project branch, isolating themselves from subsequent changes to the existing project's main branch.
• A baseline provides a simple way for a team to roll back changes to the project’s main branch, if those changes are deemed unstable or suspect.
• A baseline can provide a simple way to reproduce a reported bug because you can recreate the configuration where a particular release was built.

1. Select the view profile.
2. From the context menu, select Define new checkpoint. (Checkpoint is the view profile term for a baseline.)
3. Add the new baseline label you defined and applied. The view profile tool will ensure that this label has been attached to all the necessary VOBs.

Integrating, Building, and Testing the Project

1. Use the integration view you created.
2. Apply the label in all the VOBs that your project team modifies and references.
3. When you are asked to Move existing label, answer yes.

As team members promote changes over the course of the project, reapply the <PROJECT>_BUILD_CANDIDATE when you are ready to integrate development work. You may perform new builds nightly, weekly, or at any interval you choose. Do not move this label while you are doing a build or applying a baseline label to a known stable build.

Creating and Using a Build View

The integrator also builds the system or subsystems. Create another dynamic view (for example, t2k_gui_build). It is important to create a dynamic view for these official builds so that ClearCase audits the derived objects created during the build. After the build, you can use the audit to determine how each output file was constructed and which element versions were used during its construction.

Associate this build view with the build view profile for the project. The build view now selects the versions with the <PROJECT>_BUILD_CANDIDATE label, that is, the versions of elements ready for a build of your system or subsystems. You can start this view on your machine and perform the project build.

Backing Out a Change at Build Time

On rare occasions, it may be necessary to replace one or more versions of files in a build. For example, some file does not compile. For each file that has this problem, you can manually move the <PROJECT>_BUILD_CANDIDATE backward on the integration branch. Generally, you should move the label to the version which is identified by the latest stable baseline label. The easiest way to make this change is from the Version Tree Browser. This technique can be problematic, since there is no easy way to identify what other files were modified as part of a complete change. However, this technique can be helpful in getting a clean build.

You can also move a label forward on one element to accept a single change. We recommend that you do this rarely. Be aware that the option exists when you are dealing with unusual circumstances.

Making changes visible to developers

Once a build has been accepted and is "known to be stable", create the <PROJECT>_<RELEASE>.BL<#> label and apply it to all versions that are identified by the <PROJECT>_BUILD_CANDIDATE label. Increment the integer # for each new project build label. Use the Apply Label tool:
 

1. Use the build view.
2. Create and apply the <PROJECT>_<RELEASE>.BL<#> label.
3. It should be applied to all VOBs used and referenced by your project.

Releasing the Project

Referencing Subsystems between Projects

What if your project works on more than one subsystem?

In general, ClearCase view profiles support work on an entire system. They generate a configuration specification which specifies the versions to select and they define where to place a checked-out file. However, you may be working on a collections of subsystems rather than an entire system. Consider that different teams create baselines at different times. For example, you may want to work with Baseline 3 of one system and Baselevel 15 of another. Therefore you want the flexibility to select different versions of each subsystem, rather than selecting a single version of the entire system to work on.

Selecting a different version of other subsystems

The process is best described through example. You are working on the Thorn 2000 system, with three subsystems: GUI, storage, and modeler. The overall plan is to develop release 2 of Thorn 2000. The work has been divided into three subteams: the GUI project, the storage project, and the modeler project. You are in charge of the GUI project. There are four VOBs, \thorn2000 which holds the project related data, models, requirements, and other documentation. \t2k_gui, \t2k_storage, and \t2k_modeler, which hold the source code for each subsystem.

To select different versions of subsystems, add configuration specification rules to the development view profile. This example will show you how to add some of these rules.

When you set up the integration view profile, you select release2 as the branch type on which to work. As a starting baseline for the release 2 work, you have selected the t2k_r1.0 system label which identifies the first release of the Thorn 2000 system across all subsystems. The default configuration specification generated for the integration profile looks like this:
 

# [CC_PROJECT - Checked Out Rule
element * CHECKEDOUT
#
# Any modifications to the Profile config spec should
# be made following this comment.
# CC_PROJECT]
 

# [CC_PROJECT - Profile Config Spec
# Do not directly modify the text below, it has been
# automatically generated by the ClearCase View Profile
# Tool. To change the Profile config spec, use the
# ClearCase View Profile Wizard to update the Profile
# status as needed.
element * ...\release2\LATEST
element * T2K_R1.0 -mkbranch release2
element * \main\0 -mkbranch release2
# CC_PROJECT]
 

The asterisk (*) after the element identifier applies the rule to all elements regardless of the system or subsystem in which they are contained. To select specific versions of a specific subsystem, add a line to the view profile in the place indicated in the comments.

For example, say we know that the modeler subsystem has already created a new version that has passed its initial test. It has a new baseline label, T2K_MODELER_R2.0.BL1. The bl identifies an internal baseline and the 1 is an incremental number. To select this baseline for the view profile, add the following line:
 

element \t2k_modeler\… T2K_MODELER_R2.0.BL1

This line selects the T2K_MODELER_R2.0.BL1 version of all the elements (files and directories) in the t2k_modeler subsystem.

Here is the final configuration specification:
 

# [CC_PROJECT - Checked Out Rule
element * CHECKEDOUT
#
# Any modifications to the Profile config spec should
# be made following this comment.
# CC_PROJECT]
element \t2k_modeler\… T2K_MODELER_R2.0.BL1
# [CC_PROJECT - Profile Config Spec
# Do not directly modify the text below, it has been
# automatically generated by the ClearCase View Profile
# Tool. To change the Profile config spec, use the
# ClearCase View Profile Wizard to update the Profile
# status as needed.
element * ...\release2\LATEST
element * T2K_R1.0 -mkbranch release2
element * \main\0 -mkbranch release2
# CC_PROJECT]

 

Hint: After you move the <PROJECT>_BUILD_CANDIDATE label in the project VOBs, you only need to specify individual subsystems (VOBs) when you change the version you are selecting.

For more details on configuration specifications, see Defining View Configurations in the ClearCase User’s Manual.

Advanced Topics

This section describes the ramifications of working in a more complex environment.

Coordinating Multiple Projects

• Maintaining the stability of the source base with a baselevel-plus-changes development strategy
• Use of integration branches to support parallel development at the inter-project level
• Use of private branches to support parallel development at the intra-project level

Coordinating Multiple Parallel Releases

Appendix A: Glossary

The following terms are used in this paper. Where appropriate, the mapping between ClearCase terminology and more common terminology is presented.

 

Term	Definition
Architecture	The highest-level concept of a system in its environment [IEEE]. The architecture of a software system (at a given point in time) is its organization or structure of significant components interacting through interfaces, those components being composed of successively smaller components and interfaces.
Baseline	A named version of elements. Represented by the application of a label in ClearCase. May also be referred to as a checkpoint.
Build Workspace	A view configured to select the versions of a project identified as candidates to be included in the build.
CM Repository	The database that stores configuration items. In ClearCase, a VOB (versioned object base).
Component-Based Development	A development strategy which improves the quality and time to market of a large-scale system by decomposing the system into components with known interfaces. Also refers to the development strategy of assembling reusable components with a few changes to quickly develop a new system.
Configuration Item	A physical file or directory which is under configuration management. It is either atomic (in ClearCase, an element) or composite (a subsystem).
Configuration Management (CM)	The process of combining and evolving the logical and physical artifacts of a software system to produce a set of physical artifacts that can be deployed and subsequently run on a user’s platform.
Configuration Specification	A set of rules which defines the scope and individual versions to display in a ClearCase view.
Integration Branch	The branch where the system integration occurs and to which development changes are promoted.
Integration Workspace	A ClearCase view which selects the set of versions that have been promoted by developers and that are ready for integration.
Package	A group of logical entities that have a well-defined interface and boundary.
Promote	Make a set of elements available to other members of the team. There are different types of promotion, for example, promotion to other developers and promotion to the integrator.
Subsystem	A group of files and directories which are versioned, shared, and released as a single unit. Generally you store each subsystem in its own VOB.
System Repository	A CM Repository which holds system-wide or system-level data.
Update	Causes newer versions in the VOB to be made visible in a snapshot view.
View	A ClearCase mechanism used to display a specific set of versions.
View profile	A set of rules that imply a scope of work and configuration specification for a view. A view plus a view profile define a workspace.
VOB	Versioned Object Base, see CM Repository
Workspace	A place to work without interfering with other team members. In ClearCase, a view.
Workspace profile	In ClearCase, a view profile.

Appendix B: References

Appendix C: Process Support

This section provides the code for a ClearCase trigger, written in Perl, which you can install. The trigger warns a user who is checking out a file which is already being modified by another developer. The trigger will work if the checkouts are on the same branch or if they are on different branches.