Reliability of telecommunications software: assessing sensitivity of least squares reliability estimates

BNR, an R & D subsidiary of Northern Telecom and Bell Canada, has one of the largest software systems in the world, with code libraries exceeding 12 million source lines of a high level language. This software is used in the high-end digital switching systems that Northern Telecom markets. Software reliability methods have been applied successfully to many levels of this software and at various stages of development testing. Prior research in this environment suggests that models that utilize least squares methods for parameter estimation provide as good if not better estimates than Maximum Likelihood. However, to estimate these parameters, failures are grouped into sets of equal size with the grouping size determined at the discretion of the investigator. This paper examines the sensitivity of these estimates to the grouping size with some empirical failure data from telecommunications software. A measure, called the coefficient of variation, is adapted to quantify this sensitivity. Also, various graphical methods are suggested to assist in assessing the appropriateness of a particular model and grouping size when using least squares estimates.