Report to the Manager: Cleanroom Software Engineering

A Brief Outline Overview

David C. Wofford

 

 

 

 

  1. Introduction

Cleanroom Software Engineering is a quality process that is designed to stem the glut of poorly designed software.  As Gene F. Hoffnagle wrote in 1994:

“The acceptance of software, from the mundane to the complex, depends fundamentally on the degree of quality evidenced by that software. Low-quality software is a burden to users and is eventually either discarded or, in the absence of alternatives, tolerated. High-quality software is accepted and promoted. Knowledge about software quality and the ability to practice it have been progressing slowly but inexorably throughout the industry.” [HOF94]

 

The process is an attempt to “Do it right the first time.” Using a quality assurance process that relies on mathematical verification in the design phase and certification of the software in the testing phase. [PRE00]

  

 “Cleanroom process combines formal methods of object-based box structure specification and design, function-theoretic correctness verification, and statistical usage testing for reliability certification to produce software approaching zero defects.” [HAU94]

 

 

  1. Description
    1. “The Cleanroom approach was first used in the early 1980s at IBM's Federal Systems Division-the division responsible for space and defense systems. There, the cost of a software failure was measured in millions of dollars and potentially in human lives. Dr. Harlan Mills observed the quality gap between hardware and software components of a system, and proposed that software developers adopt the successful practices of precision manufacturing. The approach was called "Cleanroom." Cleanroom software engineering combines successful techniques of precision manufacturing with the best practices of software engineering.” [CLE01]
    2. “Cleanroom software engineering differs form the conventional and object-oriented views because:

                                                              i.      It makes explicit use of statistical quality control.

                                                            ii.      It verities design specification using a mathematically based proof of correctness.

                                                          iii.      It relies heavily on statistical usage testing to uncover high-impact errors. “[HON98]

3.    A successful project begins with certain basic development and management practices already in place. These include:

                                                              i.      management that is committed to quality, and just as committed to continuous quality improvement and cost reduction;

                                                            ii.      technical staff that is skilled and competent;

                                                          iii.      tools and education that are appropriate to the project;

                                                           iv.      knowledge of the customer and the problem domain.

The specific Cleanroom practices are tailored to fit each project, and are phased in over time. [CLE01]

  1. Advantages
    1. Proven Practice:

Several major products have been developed using Cleanroom and delivered to customers. These include:

                                                              i.      IBM COBOL/SF restructuring tool. (85,000 lines of PL/I code) This application reported a ten-fold reduction in total defects per thousand lines of code found during testing, compared to similar projects. At the same time, it reported a five-fold improvement in developer productivity measured in lines of code per labor month. This application was delivered to customers in 1988 and logged only seven errors in its first three years of service, all of which were simple fixes.

                                                            ii.      IBM AOExpert/MVS™ system outage analyzer. (107,000 lines of code) This product combined knowledge-based techniques with systems software. Despite its complexity, it achieved a more than ten-fold reduction in total errors per thousand lines of code found during testing, compared to similar projects. At the same time it reported a three-fold improvement in productivity. No operational errors were reported during beta testing.

                                                            iii.      Ericsson Telecom OS32 operating system. (350,000 lines of C and PLEX) This telecommunications switch operating system reported a rate of one failure per thousand lines of code in testing, a seventy percent improvement in development productivity, and a 100 percent improvement in testing productivity.  [CLE01]

 

    1. Low Error Rate, Highly Reliable Code
    2. Increased Productivity
    3. Can Be Incorporated Within A Quality Management Program Such As CMM Or ISO 9000.
  1. Risks
    1. A belief that the Cleanroom methodology is too theoretical, too mathematical, and too radical for use in real software development. This is a proven error in thinking. [POO93], [HEN95]
    2. It advocates no unit testing by developers but instead replaces it with correctness verification and statistical quality control--concepts that represent a major departure from the way most software is developed today.

3.    The maturity of the software development industry. The use of Cleanroom processes requires rigorous application of defined processes in all lifecycle phases. Since most of the industry is still operating at the ad hoc level (as defined by the Software Engineering Institute Capability Maturity Model), the industry has not been ready to apply those techniques. [HEN95]

    1. Programmer’s Misconceptions Regarding Implementation

                                                              i.      Forced Upon Them

                                                            ii.      Denied Use Of Compiler

                                                          iii.      Unable To Use Pointers Or Arrays

    1. Correctness Proofing
  1. Summary
    1. “Cleanroom is a SHIFT in practice from

                                                              i.      Individual Craftsmanship To Peer Reviewed Engineering

                                                            ii.      Sequential Development To Incremental Development

                                                          iii.      Informal Design To Disciplined Engineering Specification And Design

                                                           iv.      Individual Unit Testing To Team Correctness Verification

                                                             v.      Informal Or Coverage Testing To Statistical Usage Testing

                                                           vi.      Unknown Reliability To Measured Reliability”[CLE01]

                                                                            

                      

 

                      

 


 


Text Box: Figure 1 [HON98]

REFERENCES:

 

 

1.      [HON98] Hong, M.M., “Cleanroom Software Engineering for Improving Software Process Quality,” http://sern.ucalgary.ca/courses/seng/621/W98/hongm/SENG621_ESSAY.html, 1998

2.      [PRE00] Pressman, R., “Software Engineering Resources”, http://www.rspa.com/spi/cleanroom.html, 2000

3.      [HOF94] Hoffnagle, G.F., "Preface", IBM Systems Journal, vol. 33, no. 1, January 1994, p.2.

4.      [CLE01] Cleanroom Software Engineering, Inc., "An Introduction to Cleanroom Software Engineering for Managers", http://www.cleansoft.com/cleansoft_mgrguide.html, 1994-2001

5.      [CUR86] Curritt, P.A., M. Dyer, and H.D. Mills, "Certifying the Reliability of Software," IEEE Trans, Software Engineering, vol. SE-12, no. 1, January 1994.

6.      [HAU94] Hausler, P.A., R. Linger, and C. Rrammen, "Adopting Cleanroom Software Engineering with a Phased Approach," IBM Systems Journal, vol. 33, no. 1, January 1994, pp.89-109.

7.      [HEN95] Henderson, J., "Why Isn't Cleanroom the Universal Software development Methodology?” Crosstalk, vol.8, no. 5, May 1995, pp. 11-14.

8.      [HEV93] Hevner, A.R., and H.C. Mills, "Box Structure Methods for System Development with Objects," IBM Systems Journal, vol. 32, no. 2, February 1993, pp. 232 -251.

9.      [LIN79] Linger, R. M., H.D. Mills, and B.I. Witt, Structured Programming: Theory and Practice, Addison-Wesley, 1979.

10. [LIN88] Linger, R.M., and HD Mills, "A Case Study in Cleanroom Software Engineering: The IBM COBOL Structuring Facility," Proc. COMPSAC'88, Chicago, October 1988.

11. [LIN94] Linger, R., "Cleanroom Process Model," IEEE Software, March 1994, pp. 50-58.

12. [MIL87] Miils, H.D., M. Dyer, and R. Linger, "Cleanroom Software Engineering," IEEE Software, September 1987, pp.19-24.

13. [MIL87] Musa, J.D., A. Lannino, and K. Okumoto, Engineering and Managing Software with Reliability Measures, McGraw-Hill, 1987.

14. [POO88] Poore, J.H., and H.D. Mills, "Bringing Software Under Statistical Quality Control, Quality Progress, November 1988, pp. 52-55.

15. [POO93] Poore, J.H., H. D. Mills, and D. Mutchler, "Planning and Certifying Software System Reliability," IEEE Software, vol. 10, no. 1, January 1993, pp. 89-99.

16. [WOH94] Wohlin, C., and P. Runeson, "Certification of Software Components," IEEE Trans, Software Engineering, vol. 20, no. 6, June 1994, pp.494-499.

17. [DEC97] Deck, M.D., "Cleanroom Software Engineering Myths and Realities," Quality Week 1997, San Francisco, CA, May, 1997

18. [PRE97] Roger S. Pressman, Software Engineering, A Practioner's Approach, 4th Edition, McGraw-Hill, 1997.

19. [HUM89] Humphrey, W.S., Managing the Software Process, Addison-Wesley. 1989.