Training Impact

Calibration

Templates

Function Points

Golub's Laws

There is a popular phrase in the software acquisition community suggesting that -- through the use of a calibrated, validated model -- estimating errors will disappear.  Not likely!  The impact of training and experience cannot be ignored in software cost estimating.  There are four sources of estimate error:

The model is the least significant error source.  The estimator is the most significant error source.  The estimator collects the estimate data, establishes the estimating model input parameters, performs the cost and schedule analysis, and produces the final estimate.  A realistic estimate requires the estimator be proficient with the estimating tool.  Proficiency equates to training and experience.  Training requires more than access to a User's Guide and experience.  Neither training nor experience is not instantaneous.  There are approximately 30 input parameters required by all of the major software estimating tools (Sage has 31 inputs).  If a 1 percent error is made in each input, the resulting cost error is near 35 percent.  A 1 percent error is small, especially in the pre-source-selection acquisition phase where the major estimating activities occur.  

What happens to the estimate if the estimator approaches the estimate with a "benefit of the doubt" attitude?  What impact will a bad day ("sock it to 'em") have on the estimate?  What happens if the estimator misinterprets the definition of one or more parameters?  The induced errors are multiplicative, not additive. 

Lack of training and experience amplifies the estimator error.  This error is described in Augustine's XXIV Law, the Law of Inestimable Consequences.  There are no shortcuts to expertise with any estimating tool, in spite of claims of minimal training requirements.  Training and experience are essential to realistic estimates.  Training is only a small portion of a major acquisition cost, but a major driver in the software estimate.  (Topic)

The most unsuccessful three years in the education of cost estimators appears to be fifth-grade arithmetic

Point Calibrate

Point calibrate is a facility unique to Sage.  This facility analyzes the differences between actual project results (cost and schedule) and the project estimate.  Point Calibrate does NOT calibrate the estimating tool, but is used to evaluate the estimate inputs (a far greater source of error).  All inputs in a post mortem analysis (especially the product size) are known can be specified.  We assume the underlying estimating model is not the primary estimate error source, and modifying the model through calibration requires re-validation after each calibration change before the model can be trusted for future work.  Believe it or not, most estimating tools are validated before they are released, and validation is not a trivial task.

The Point Calibrate evaluation is used by the estimator to:

The Point Calibrate facility also provides a basis for developing product and management  templates that is based on actual project results.  Thus, for a given product type, the template parameter combination  realistically represents the product type or development environment. (Topic)

Estimate error can be minimized, or nearly eliminated, through the use of calibrated product and management templates.  A template is a set of parameters that define a specified product or development environment.  The template ("knowledge base") was not originated in Sage.  Sage templates, however, are non-overlapping.

What is "non-overlapping"?  Overlapping is a problem that occurs when two or more independent templates overwrite a previously loaded template.  Sage template types (management and product) have no estimate parameters in common; thus, they DO NOT overlap.  Other knowledge base schemes have large numbers of common, and these knowledge bases do overwrite previously loaded parameter sets.  The only knowledge base not corrupted is the last one loaded.

  Sage management templates can be defined for organizations from the generic to the individual manager level.  Since organizations and development environment change slowly over time, post mortem project calibration is very helpful in building these templates.

Sage product templates are defined for specific product types, and contain only estimate parameters related to the product.  The input Environment Description dialog contain 6 tabs of which the last 2 tabs relate only to the product template parameters.

The bottom line: When working with an estimate based on non-Sage knowledge bases, be sure to verify that changes you make to your project environment parameters are not corrupted through inadvertent modifications made by knowledge bases.     (Topic)

Ultimate Law of Accuracy:

And it ought to be remembered that there is nothing more difficult to take in hand, more perilous to conduct, or more uncertain in its success, than to take the lead in the introduction of a new order of things. Because the innovator has for enemies all those who have done well under the old conditions, and lukewarm defenders in those who may do well under the new. This coolness arises partly from fear of the opponents, who have the laws on their side, and partly from the incredulity of men, who do not readily believe in new things until they have had a long experience of them.

N. Machiavelli, The Prince, Chap VI

Function point capabilities have been added to the repertoire of input metrics in Sage 3.  The implementation is fully compatible to the IFPUG 4.2 Counting Practices Manual including the inclusion of calculation of the Value Adjustment Factor omitted in the other software cost and schedule estimating tools.

The additional of 3D function points ala Steve Whitmire introduced at Boeing Company makes it possible to extend the classic function point method to real-time system and object point counting methods.

1. Fuzzy project objectives are used to avoid the embarrassment of estimating the corresponding costs.

2. A carefully planned project takes three times longer to complete than expected; a carefully planned project will take only twice as long.

3. The effort required to correct course increases geometrically with time.

4. Project teams detest weekly progress reporting because it so vividly manifests their lack of progress.  (Topic)