Validation of the Sage 4 Software Schedule and Cost Estimation Model
It is important to understand the
genealogy of the Jensen Software Development Model, the history of the
validation of this model, and the validation of the Sage implementation of
this model. The course of the
discussion will show the model has been validated continually over the
past 25 years, and by extension, all correct implementations of the model
have also been validated.
The
Jensen Model is mathematically described by the following relationships:
S = Cte
* sqrt (K) Td
(1)
and
D = K /
T^3
(2)
where S
= effective software size,
Cte =
effective development technology constant,
K =
total life cycle cost from start of development through end of life
(person years),
T =
development time from start of development through FQT (years), and
D
= project complexity measure.
The technology constant contains
close to 30 parameters that describe the development environment and
project characteristics. All
Jensen Model implementations contain these 30 parameters.
Sage adds one management parameter and more precisely defines 3 of
the existing parameters to describe management effects.
These relationships were originally
developed at the Hughes Aircraft Company Space and Communications Group in
1979 to support studies in productivity improvement.
The model (referred to as Seer) was based upon a collection of
project data accumulated within the software development group.
The data was restricted to “crisis mode” projects: that is,
projects that were constrained by schedule and funding.
That data was augmented by a subset of the data used by Putnam in
the development of the SLIM estimating model that satisfied the crisis
mode criteria.
The first official validation of the
Jensen Model was conducted by NSA in 1982 under the direction of Troy Cox.
Cox was trained in the correct use of the model, after which he
conducted the validation study with classified NSA project data.
The validation was successful.
The detailed study results were never publicly released.
The Jensen Model was commercially
implemented by Computer Economics, Inc. as the JS1 (1981), the JS2 (1982),
and System-3 (1985). The JS2
was the implementation used by NSA in the validation study.
Galorath Inc. released the Jensen Model in 1990 as the SEER-SEM
software estimating tool. Each
of these implementations continued the validation of the underlying model.
Project estimates using the model as
far back as 1979 were credible only when applied to classical software
development environments. Classical
environments are still best typified by cubicle floor plans.
Non-classical environments such as those involving modern
management approaches (pair programming, skunk works, TQM and agile
techniques for example) were outside the validated project range and
produced suspect estimates. All
of the historic data used to support the model development and validation
was obtained from classical projects.
Sage (1995) was the first commercial
implementation of the Jensen Model containing modifications to the
technology constant parameters to account for the impact of modern
environments and management. The
technology constant improvements were derived from non-classical project
data collected over the preceding 20 years.
The improvements have an additional decade of validating data.
The underlying validated model was unchanged in the process.
When a classic environment is described in the updated technology
constant parameters, the cost and schedule estimates are mathematically
identical to estimates produced prior to the Sage technology constant
upgrade. Sage and SEER-SEM
cost and schedule estimates for classic environments are identical.
In a sense, the valid operating range of the Jensen Model was
broadened in this evolution step to include management effects.
Sage 3 introduced valuable new
estimating facilities that were unavailable in the existing commercial
implementations of the model. First,
the addition of software size growth estimates based on a variation of the
widely-used Holchin growth algorithm makes it possible to project growth
according to the current state of a software system definition.
Second, a risk analysis capability was introduced that is based on
the statistical variance of the estimate input parameters rather than the
traditional point estimate. Third,
complete support for function point size projection based on the IFPUG 4.2
standard. The function point capability also includes extensions to
support 3D function points defined by S. Whitmire of Boeing. The
Whitmire extension makes it possible to use Sage 3 for object point
counting.
The advent of requirements to
produce realistic estimates prior to Milestone A led to the evolution of
Sage 4. Pre-Milestone A created a requirement to develop estimates
before the system has been decomposed to a component architecture and the
software component developer has been identified. The result (Sage
4) has a simple input data requirement that supports early system
development estimates that is unique in the software estimating industry.
As a concluding note, there is a
common misconception that “properly calibrated and validated” tools
eliminate major estimate errors. Estimating
tools, of which the Jensen Model is a fine example (Sage and SEER-SEM are
commercial implementations), are simply estimating tools.
Proper validation reduces errors inherent in the tools themselves.
The model, or tool, is like a good scalpel.
The estimate quality still lies primarily in the quality, training,
and experience of the user.