COMPARING CRYSTAL BALL® WITH ACEIT

There are a variety of methods used throughout the DoD cost analysis community to quantify and calculate the risk inherent in a government project cost estimate. Crystal Ball is a popular application that is in wide use across DoD. The DoD has also sponsored its own application Automated Cost Estimating Integrated Tools (ACEIT www.aceit.com) is an automated architecture and framework for cost estimating. ACEIT is a government developed tool that has been used for over a decade to standardize and simplify the Life Cycle Cost estimating process in the government environment. We often get questions from the field asking why Crystal Ball results do not “match” those from ACEIT. Almost always, we have traced these problems to either confusion on terminology, differences in default settings and/or modeling technique inconsistencies. A specific concern to military cost analysts is the manner in which correlation is dealt with in a risk simulation. Crystal Ball employs Spearman Rank correlation and ACEIT uses Pearson Product Moment. To test correlation and other cost risk analysis issues, we modeled several case studies published by leaders in the profession. We were careful to pick case studies that have published analytical results for representative cost models. We compared the analytical results to results generated by Crystal Ball and ACEIT to demonstrate that if key modeling decisions are made consistently across the models, the results match well. 1 BACKGROUND