A Capability Maturity Model for Security Engineering
Shared by: kdq54459
A Capability Maturity Model for Security Engineering Karen Ferraiolo, Doug Landoll, Joel Sachs ARCA Systems, Inc. 8229 Boone Blvd., Suite 610 Vienna, VA 22182 Phone: (703) 734-5611 1.0 Introduction Lessons learned in maturing the software engineering process and the immaturity of current security engineering practice point to the need to focus on maturing the security engineering process. Realizing the benefits gained from use of maturity models in the software engineering industry, a Security Engineering Capability Maturity Model (SE CMM) is being developed to guide process improvement in the practice of security engineering. This presentation advocates the need for the development of a capability maturity model for security engineering. First, background and concepts on process improvement and maturity models and benefits that have been gained from their implementation will be presented. Next, motivation for the development of a SE CMM and a brief introduction to the model will be given, with recommendations for immediate work. 2.0 Why Process Improvement To address the problem of late, over-budget programs, significant work has been accomplished in advancing practices for improved quality and cost and schedule predictability of software dependent systems. In support of the Software Engineering Institute's (SEI) overall objective to enable organizations to improve their software capability, the SEI CMM for Software was developed. The model was developed for organizations to use in assessing their process capability to develop software, determine where to focus efforts, and guide them in developing and implementing improvement plans. Currently other work has begun in extending the concepts of the CMM for Software to the development of Systems Engineering CMMs (one, a joint effort by the SEI and the National Council on Systems Engineering; another, an Air Force developed model). This section will present motivation for process improvement by describing the concepts and use of the SEI CMM for Software. 2.1 Background/Concepts The SEI CMM for Software defines software measures and a measurement process that can be used to systematically and repeatedly measure software development progress, products and processes. It is used by: 1) software engineering organizations to assess themselves and identify the most important improvement needs of their organization; and 2) acquisition agencies to identify the risks associated with awarding business to contractors, as well as manage on-going projects.[SEI93a] The model defines five levels of software process maturity -- each indicating an organization’s software process capability. A Maturity Level is a well-defined evolutionary plateau on the path toward becoming a mature software organization. Each level provides a step in continuous process improvement and is composed of several Key Process Areas (KPAs). A KPA is a group of related practices that when performed collectively, achieve a set of goals that are sufficient for establishing a process capability at a given level. Each KPA contains Key Practices -- the policies and activities that collectively provide for implementation of a KPA by achieving the goals of the KPA. The Key Practices are organized within each KPA by Common Features. These Common Features are Commitment to Perform, Ability to Perform, Activities Performed, Measurement and Analysis, and Verifying Implementation. This organization provides an infrastructure that supports continuous improvement by elaborating the institutionalization factors which most affect an organization's ability to exhibit the capability implied by a particular maturity level. Each Maturity Level of the model includes the KPAs of the previous level with additional KPAs added. Each level then provides the foundation for improvements undertaken to reach the next level. Process improvements are implemented in stages, since some processes cannot be effective unless others are stable. To support assessments, some of the Key Practices are identified as the most critical for implementing a KPA and are selected for developing the Maturity Questionnaire. The Maturity Questionnaire is used by evaluation teams to initiate the investigation of an organization’s software process capability in order to identify strengths and weaknesses in the organization’s software process. 2.2 Motivation for a Focus on Process Implementation of the concepts in the SEI CMM for Software has led to institutionalization of the software process across organizations, resulting in higher quality software produced, on time and within budget. Observations from initial use of the CMM indicate that the software process provides an important leverage point for addressing productivity and quality problems with sustained, cost-effective benefits to development organizations. Results from software process improvement efforts within Lockheed, Hughes, and Raytheon demonstrate these results within the software industry. [CUR92], [OVE93] 3.0 Potential Benefits of a SE CMM 3.1 The Security Engineering Problem We have observed three problems facing Security Engineering today. First, current practice does not include well defined processes that an organization can use to focus its engineering and improvement efforts. This practice results in security engineering activities considered as separate from, rather that integrated with, systems and software development activities. The lack of these well-defined processes is the second problem. Acquisition organizations are left without a mechanism for assessing the security engineering capability of organizations. With no such mechanism, awards are not given to those with the most appropriate security engineering experience and capability. The third problem relates to assurance in secure systems. Development efforts most often emphasize the end results, while often actually targeting an evaluation of the process involved in its creation. That is, by examining a system’s functionality and its supporting documentation, it is often assumed that the developers followed a well defined process when this may not have been the case. See [SAC94] for a detailed discussion. 3.1 Benefits of a SE CMM Noting these problems, the need for a capability assessment / evaluation mechanism for security engineering similar to the SEI model, has become apparent. Availability of a SE CMM would promote emphasis on the process of developing a trusted system in the following ways: 1) The model would be the basis for simplifying and expediting evaluation and certification efforts by facilitating evaluations/certifications based both on the process and the end results. 2) By allowing an evaluation of a security engineering group’s capability, the SE CMM would support the choice of a well qualified integration team on Government acquisitions. 3) The model could be used by a security engineering group, in conjunction with management, in defining their process used in secure development efforts along with identifying areas in which to develop and implement improvement plans. 4) Over time, improvement in security engineering processes will promote improvement in the organizations of which they are a part, which would then promote an industry-wide improvement of security engineering capability. 4.0 The Security Engineering CMM A Security Engineering CMM (SE CMM) is being developed by NSA (I91), and an ARCA/ARC team under the SSEMS contract. This section gives a brief overview of the model's development 4.1 Framework 4.1.1 Maturity Levels The Maturity Levels for the SE CMM are as defined by the SEI CMM for Software. The levels are: Level 1 - Initial (few processes are defined, and success depends on individual effort talent and heroic effort); Level 2 - Repeatable (the necessary process discipline is in place to repeat earlier successes on projects with similar applications); Level 3 - Defined (the process for both management and engineering activities is documented, standardized, and integrated into an organization-wide process and used by all projects); Level 4 - Managed (both the process and end-products are quantitatively understood and controlled using detailed measures); and Level 5 - Optimizing (continuous process improvement is enabled by quantitative feedback from the process and from testing innovative ideas and technologies). We have identified maturational goals for the SE CMM. These goals are to guide definition of KPAs and practices at the different maturity levels of the model: 1) Process Maturational Goals: - controllability: ability to predict, measure, and control cost, schedule, and quality; - codification: state-of-the-art knowledge is codified within the practices; - trustability: degree of assurance that practices are performing as intended. 2) Organizational Maturational Goals: - institutionalization: organization-wide use of defined process; - integration: organization-wide process integration; - improvement: continuous process improvement. 4.1.2 Key Process Areas The KPAs of the SEI CMM for Software have been defined to reside at one Maturity Level (that is, they do not change at the upper levels). The additional objectives of the SE CMM indicate the need to define practices that build on those of the previous level within a KPA. Our initial organization of KPAs for the SE CMM is by four Major Process Areas. The KPAs are defined within each of these major areas at each of the maturity levels: 1) Security Risk Management - processes dealing with estimating risk at each of the maturity levels; 2) Engineering - processes involved with architecting a system and managing security requirements; 3) Assurance Management - processes dealing with generating, managing, presenting assurance evidence (this is discussed in more detail in [LAN94]; 4) Coordination - processes that coordinate security engineering activities with other engineering disciplines. Currently, work is being done to define the Key Practices for the KPAs within these areas. The practices are organized by Common Feature as in the SEI CMM for Software, providing the necessary infrastructure that supports continuous improvement. 5.0 Recommendations The SE CMM is currently under development with the result of the initial effort being a Strawman document. The following are recommended for its success: 1) Continue development of a SE CMM Strawman by first defining process areas that are important to security engineering and retaining as a goal its use as a measure for assurance. 2) During development of the SE CMM, remain involved in related work (for example, SEI, NCOSE, ISO). 3) Develop a plan for finalization, promotion, and support of the SE CMM (that is a Working Group, User Group, and Support structure). References [ASQ87] American Society for Quality Control, “Quality Systems - Model for Quality Assurance in Design/Development, Production, Installation, and Servicing,” ANSI/ASQC Q91-1987, 1987. [CUR92] Curtis, Dr. Bill, “Software Process Improvement Seminar for Senior Executives,” Briefing at NIST, Gaithersburg, MD, December 1992. [FER93] Ferraiolo, Karen, Sachs, Joel, "Determining Assurance Levels by Security Engineering Process Maturity," Proceedings of the Fifth Annual Canadian Computer Security Symposium, May 1993. [GAR93] Suzanne Garcia, "Principles of Maturity Modeling," Presentation at Software Engineering Symposium, September 1993. [GOO93a] John B. Goodenough, Mark H. Klein, "Maturity Models for the Technical Aspects of Software Engineering," Draft, August 6, 1993. [GOO93b] John B. Goodenough, "Maturity Models for the Technical Aspects of Software Engineering," Presentation at Software Engineering Symposium, September 1993. [HEA92] Heaney, Jody E., “Software Development and Security Engineering,” Position Paper for Software Development Group, The CERT Workshop on Research in Incident Handling, SEI, 5-6 November, 1992. [ISO91] International Organization for Standardization, “Quality Management and Quality Assurance Standards - Part 3: Guidelines for the application of ISO 9001 to the development, supply and maintenance of software,” ISO 900-3, 1991. [LAN94] Landoll, Doug, Ferraiolo, Karen, Sachs, Joel, "Process as a Measure for Assurance Using the Security Engineering Capability Maturity Model," position paper submitted to the Invitational Workshop on IT Assurance and Trustworthiness, January, 1994. [OVE93] Over, James, W., “Motivation For Process-Driven Development,” CrossTalk - The Journal of Defense Software Engineering, January 1993. [SAC94] Sachs, Joel, Ferraiolo, Karen, Landoll, Doug, "Advocacy for an Engineering Process Oriented Approach to Assurance," position paper submitted to the Invitational Workshop on IT Assurance and Trustworthiness, January, 1994. [SEI93a] Software Engineering Institute, “Capability Maturity Model for Software,” Version 1.1, CMU/SEI-93-TR-24, ESC-TR-93-177, February 1993. [SEI93b] Software Engineering Institute, “Key Practices of the Capability Maturity Model,” Version 1.1, CMU/SEI-93-TR-25, ESC-TR-93-178, February 1993. [WOO92] Woodruff, D., "Saturn," Business Week, Aug. 17, 1992.