Assessing the State of Software in a Large Enterprise.

To be relevant to the goals of an enterprise, an industrial software engineering research organization must identify problems of interest to, and find solutions that have an impact on, the software development organizations within the company. Using a systematic measurement program both to identify the problems and assess the impact of solutions is key to satisfying this need. Avaya has had such a program in place for about seven years. Every year we produce an annual report known as the State of Software in Avaya that describes software development trends throughout the company and that contains prioritized recommendations for improving Avaya's software development capabilities. We start by identifying the goals of the enterprise and use the goal-question-metric approach to identify the measures to compute. The result is insight into the enterprise's problems in software development, recommendations for improving the development process, and problems that require research to solve. We will illustrate the process with examples from the Software Technology Research Department in Avaya Labs whose purpose is to improve the state of software development and know it. "Know it" means that improvement should be subjectively evident and objectively quantifiable. "Know it" also means that one must be skilled at identifying the data sources, performing the appropriate analyses to answer the questions of interest, and validating that the data are accurate and appropriate for the purpose. Examples will include how and why we developed a measure of software quality that appeals to customers, how and why we are studying the effectiveness of distributed software development, and how and why we are helping development organizations to adopt iterative development methods. We will also discuss how we keep the company and the department apprised of the current strengths and weaknesses of software development in Avaya through the publication of the annual State of Software in Avaya Report. Our purpose is both to provide a model for assessment that others may emulate, based on seven years of experience, and to spotlight analyses and conclusions that we feel are common to software development today.