A Capability Maturity Model for Security Engineering

							A Capability Maturity Model for Security Engineering


Karen Ferraiolo, Doug Landoll, Joel Sachs
ARCA Systems, Inc.
8229 Boone Blvd., Suite 610
Vienna, VA 22182
Phone: (703) 734-5611


1.0 Introduction
Lessons learned in maturing the software engineering process and the immaturity of
current security engineering practice point to the need to focus on maturing the security
engineering process. Realizing the benefits gained from use of maturity models in the
software engineering industry, a Security Engineering Capability Maturity Model (SE
CMM) is being developed to guide process improvement in the practice of security
engineering.
This presentation advocates the need for the development of a capability maturity
model for security engineering. First, background and concepts on process
improvement and maturity models and benefits that have been gained from their
implementation will be presented. Next, motivation for the development of a SE CMM
and a brief introduction to the model will be given, with recommendations for
immediate work.
2.0 Why Process Improvement
To address the problem of late, over-budget programs, significant work has been
accomplished in advancing practices for improved quality and cost and schedule
predictability of software dependent systems. In support of the Software Engineering
Institute's (SEI) overall objective to enable organizations to improve their software
capability, the SEI CMM for Software was developed. The model was developed for
organizations to use in assessing their process capability to develop software, determine
where to focus efforts, and guide them in developing and implementing improvement
plans. Currently other work has begun in extending the concepts of the CMM for
Software to the development of Systems Engineering CMMs (one, a joint effort by the
SEI and the National Council on Systems Engineering; another, an Air Force developed
model).
This section will present motivation for process improvement by describing the
concepts and use of the SEI CMM for Software.
2.1 Background/Concepts
The SEI CMM for Software defines software measures and a measurement process that
can be used to systematically and repeatedly measure software development progress,
products and processes. It is used by: 1) software engineering organizations to assess
themselves and identify the most important improvement needs of their organization;
and 2) acquisition agencies to identify the risks associated with awarding business to
contractors, as well as manage on-going projects.[SEI93a]
The model defines five levels of software process maturity -- each indicating an
organization’s software process capability. A Maturity Level is a well-defined
evolutionary plateau on the path toward becoming a mature software organization.
Each level provides a step in continuous process improvement and is composed of
several Key Process Areas (KPAs). A KPA is a group of related practices that when
performed collectively, achieve a set of goals that are sufficient for establishing a
process capability at a given level. Each KPA contains Key Practices -- the policies and
activities that collectively provide for implementation of a KPA by achieving the goals
of the KPA. The Key Practices are organized within each KPA by Common Features.
These Common Features are Commitment to Perform, Ability to Perform, Activities
Performed, Measurement and Analysis, and Verifying Implementation. This
organization provides an infrastructure that supports continuous improvement by
elaborating the institutionalization factors which most affect an organization's ability to
exhibit the capability implied by a particular maturity level.
Each Maturity Level of the model includes the KPAs of the previous level with
additional KPAs added. Each level then provides the foundation for improvements
undertaken to reach the next level. Process improvements are implemented in stages,
since some processes cannot be effective unless others are stable.
To support assessments, some of the Key Practices are identified as the most critical for
implementing a KPA and are selected for developing the Maturity Questionnaire. The
Maturity Questionnaire is used by evaluation teams to initiate the investigation of an
organization’s software process capability in order to identify strengths and weaknesses
in the organization’s software process.
2.2 Motivation for a Focus on Process
Implementation of the concepts in the SEI CMM for Software has led to
institutionalization of the software process across organizations, resulting in higher
quality software produced, on time and within budget. Observations from initial use
of the CMM indicate that the software process provides an important leverage point for
addressing productivity and quality problems with sustained, cost-effective benefits to
development organizations. Results from software process improvement efforts
within Lockheed, Hughes, and Raytheon demonstrate these results within the software
industry. [CUR92], [OVE93]
3.0 Potential Benefits of a SE CMM
3.1 The Security Engineering Problem
We have observed three problems facing Security Engineering today. First, current
practice does not include well defined processes that an organization can use to focus its
engineering and improvement efforts. This practice results in security engineering
activities considered as separate from, rather that integrated with, systems and software
development activities.
The lack of these well-defined processes is the second problem. Acquisition
organizations are left without a mechanism for assessing the security engineering
capability of organizations. With no such mechanism, awards are not given to those
with the most appropriate security engineering experience and capability.
The third problem relates to assurance in secure systems. Development efforts most
often emphasize the end results, while often actually targeting an evaluation of the
process involved in its creation. That is, by examining a system’s functionality and its
supporting documentation, it is often assumed that the developers followed a well
defined process when this may not have been the case. See [SAC94] for a detailed
discussion.
3.1 Benefits of a SE CMM
Noting these problems, the need for a capability assessment / evaluation mechanism
for security engineering similar to the SEI model, has become apparent. Availability of
a SE CMM would promote emphasis on the process of developing a trusted system in
the following ways:
1) The model would be the basis for simplifying and expediting evaluation and
certification efforts by facilitating evaluations/certifications based both on the process
and the end results.
2) By allowing an evaluation of a security engineering group’s capability, the SE CMM
would support the choice of a well qualified integration team on Government
acquisitions.
3) The model could be used by a security engineering group, in conjunction with
management, in defining their process used in secure development efforts along with
identifying areas in which to develop and implement improvement plans.
4) Over time, improvement in security engineering processes will promote
improvement in the organizations of which they are a part, which would then promote
an industry-wide improvement of security engineering capability.
4.0 The Security Engineering CMM
A Security Engineering CMM (SE CMM) is being developed by NSA (I91), and an
ARCA/ARC team under the SSEMS contract. This section gives a brief overview of
the model's development
4.1 Framework
4.1.1 Maturity Levels
The Maturity Levels for the SE CMM are as defined by the SEI CMM for Software. The
levels are: Level 1 - Initial (few processes are defined, and success depends on
individual effort talent and heroic effort); Level 2 - Repeatable (the necessary process
discipline is in place to repeat earlier successes on projects with similar applications);
Level 3 - Defined (the process for both management and engineering activities is
documented, standardized, and integrated into an organization-wide process and used
by all projects); Level 4 - Managed (both the process and end-products are
quantitatively understood and controlled using detailed measures); and Level 5 -
Optimizing (continuous process improvement is enabled by quantitative feedback from
the process and from testing innovative ideas and technologies).
We have identified maturational goals for the SE CMM. These goals are to guide
definition of KPAs and practices at the different maturity levels of the model:
1) Process Maturational Goals:
- controllability: ability to predict, measure, and control cost, schedule, and quality;
- codification: state-of-the-art knowledge is codified within the practices;
- trustability: degree of assurance that practices are performing as intended.
2) Organizational Maturational Goals:
- institutionalization: organization-wide use of defined process;
- integration: organization-wide process integration;
- improvement: continuous process improvement.
4.1.2 Key Process Areas
The KPAs of the SEI CMM for Software have been defined to reside at one Maturity
Level (that is, they do not change at the upper levels). The additional objectives of the
SE CMM indicate the need to define practices that build on those of the previous level
within a KPA.
Our initial organization of KPAs for the SE CMM is by four Major Process Areas. The
KPAs are defined within each of these major areas at each of the maturity levels:
1) Security Risk Management - processes dealing with estimating risk at each of the
maturity levels;
2) Engineering - processes involved with architecting a system and managing security
requirements;
3) Assurance Management - processes dealing with generating, managing, presenting
assurance evidence (this is discussed in more detail in [LAN94];
4) Coordination - processes that coordinate security engineering activities with other
engineering disciplines.
Currently, work is being done to define the Key Practices for the KPAs within these
areas. The practices are organized by Common Feature as in the SEI CMM for
Software, providing the necessary infrastructure that supports continuous
improvement.
5.0 Recommendations
The SE CMM is currently under development with the result of the initial effort being a
Strawman document. The following are recommended for its success:
1) Continue development of a SE CMM Strawman by first defining process areas that
are important to security engineering and retaining as a goal its use as a measure for
assurance.
2) During development of the SE CMM, remain involved in related work (for
example, SEI, NCOSE, ISO).
3) Develop a plan for finalization, promotion, and support of the SE CMM (that is a
Working Group, User Group, and Support structure).
References
[ASQ87] American Society for Quality Control, “Quality Systems - Model for Quality
Assurance in Design/Development, Production, Installation, and Servicing,”
ANSI/ASQC Q91-1987, 1987.
[CUR92] Curtis, Dr. Bill, “Software Process Improvement Seminar for Senior
Executives,” Briefing at NIST, Gaithersburg, MD, December 1992.
[FER93] Ferraiolo, Karen, Sachs, Joel, "Determining Assurance Levels by Security
Engineering Process Maturity," Proceedings of the Fifth Annual Canadian Computer
Security Symposium, May 1993.
[GAR93] Suzanne Garcia, "Principles of Maturity Modeling," Presentation at Software
Engineering Symposium, September 1993.
[GOO93a] John B. Goodenough, Mark H. Klein, "Maturity Models for the Technical
Aspects of Software Engineering," Draft, August 6, 1993.
[GOO93b] John B. Goodenough, "Maturity Models for the Technical Aspects of
Software Engineering," Presentation at Software Engineering Symposium, September
1993.
[HEA92] Heaney, Jody E., “Software Development and Security Engineering,”
Position Paper for Software Development Group, The CERT Workshop on Research in
Incident Handling, SEI, 5-6 November, 1992.
[ISO91] International Organization for Standardization, “Quality Management and
Quality Assurance Standards - Part 3: Guidelines for the application of ISO 9001 to the
development, supply and maintenance of software,” ISO 900-3, 1991.
[LAN94] Landoll, Doug, Ferraiolo, Karen, Sachs, Joel, "Process as a Measure for
Assurance Using the Security Engineering Capability Maturity Model," position paper
submitted to the Invitational Workshop on IT Assurance and Trustworthiness, January,
1994.
[OVE93] Over, James, W., “Motivation For Process-Driven Development,” CrossTalk
- The Journal of Defense Software Engineering, January 1993.
[SAC94] Sachs, Joel, Ferraiolo, Karen, Landoll, Doug, "Advocacy for an Engineering
Process Oriented Approach to Assurance," position paper submitted to the Invitational
Workshop on IT Assurance and Trustworthiness, January, 1994.
[SEI93a] Software Engineering Institute, “Capability Maturity Model for Software,”
Version 1.1, CMU/SEI-93-TR-24, ESC-TR-93-177, February 1993.
[SEI93b] Software Engineering Institute, “Key Practices of the Capability Maturity
Model,” Version 1.1, CMU/SEI-93-TR-25, ESC-TR-93-178, February 1993.
[WOO92] Woodruff, D., "Saturn," Business Week, Aug. 17, 1992.