Principle of Risk Management and Insurance by ooh34326

VIEWS: 23 PAGES: 45

More Info
									DRAFT
Things to look for in this manual:

Talk about areas that the Program Officer can influence through approval process of PEP,
procurements, cooperative agreements, MOU’s, approval of safety plan, QA plan,
budget, etc.

Structure oversight processes so that they are not a limiting factor in the development of
the project. The project development should be limited by the rate at which technical
developments can be completed, not the rate at which oversight steps are approved, so
make sure that the necessary oversight processes are well thought our and absolutely
necessary.

Make it clear that the project officer is NOT the project manager, but is someone that
should have a critical eye in viewing and approving the plans for execution of the project.
I think it is important that the specific guidance below be presented as background
information that the PO can use to evaluate the reported progress and plans of the project,
but is not intended so that the PO will give specific guidance in how to execute the
project. With that in mind, I have a few additional suggestions:

Schedules:
    Developing and managing a critical path – what to look for in the awardee’s plan
    Minimizing contractor dependencies and potential for subcontractor interference
      in the schedules – look for schedules with few interfaces and lots of float.
    Minimizing GFE in cooperative agreements and MOU’s.

Procurement packages –
    prioritization and tiering of subcontracts – a good topic for a splinter session in a
       management review of the project
    foreign currency fluctuations – how are these handled in the budget proposal?
    sensitivities to material and labor rate fluctuations – price in units

Land use agreements for surrounding area and site access
    In the case of bilateral agreements between NSF and the provider of the land, it is
       important that the land use agreement be thoroughly reviewed by legal and that
       one understand any restrictions or intended uses for the surrounding property.
       Also, easements and right of ways, and responsible for maintaining roads which
       will be accessed by the public are things to watch out for.

Multiple organizations and interfaces
 put scope, budget, and authority in the place within the organization best equipped to
   deal with this problem
Even within a project, look at the org chart to understand how it is aligned with
workscope, procurements, and risk possibilities.


                                                                                             1
RISK MANAGEMENT
1.1 INTRODUCTION

Risk management is concerned with future events, whose exact outcome is unknown, and
how to deal with these uncertainties, i.e., a range of possible outcomes. In general,
outcomes are categorized as favorable or unfavorable, and risk management is the art and
science of planning, assessing, and handling future events to ensure favorable outcomes
to the extent possible. The alternative to risk management is crisis management, a
resource-intensive process that is normally constrained by a restricted set of available
options.

The objective of this document is to provide an overview of the subject of risk
management from an NSF perspective. There are many different types of risk; the
discussion herein pertains to project risk. This document is not intended to be a
comprehensive discussion of the subject of risk management. Additional sources of
information on Risk Management are included in Appendix A. Briefly, the objectives of
this document are:

      To provide an overview of the risk management process from an NSF
       perspective.

      To guide the PO as to their responsibilities in the area of risk management.

      To enable the PO to understand how and when a risk assessment should be
       performed and when a Risk Management Plan should be written and what it
       should address.

      To help the PO understand the benefits of risk management and the importance of
       risk management in ensuring project success.

The Federal risk management concept is based on the principle that risk management
must be forward-looking, structured, informative, and continuous. The key to successful
risk management is early recognition, planning and aggressive execution. Good planning
enables an organized, comprehensive, and iterative approach for identifying and
assessing the risk and risk handling options. To support these efforts, assessments should
be performed as early as possible in the life cycle to ensure that critical technical,
schedule, and cost risks are addressed with mitigating actions incorporated into planning
and budget projections.




                                                                                         2
At the NSF, risk management is a shared responsibility of the Program Officers (POs)
and the Awardees. Most of the risk planning and risk assessment activities will be
performed by the Awardee under the oversight of the PO. The Facilities Management
and Oversight Guide requires Awardees to commence risk management early in the
Concept/Development stage of the project. The Awardee’s Risk Management Plan is
incorporated into the Project Execution Plan (PEP). If necessary, the NSF Program
Officer may perform a separate Risk Assessment (from an NSF perspective) and include
such in the Internal Management Plan (IMP) for the project. All Risk Assessments
should be frequently updated throughout the life of the project.

Effective risk management requires involvement of the entire project team (for example,
the Project Advisory Team members) and also requires help from outside experts (such
as the Facilities Panel) knowledgeable in critical risk areas (e.g., technology, design,
manufacturing, construction, networking, logistics, schedule, and cost). In addition, the
risk management process should cover hardware, software, the human element, and
integration issues.




1.2    DEFINITIONS

Risk is a measure of the potential inability to achieve overall project objectives within
defined cost, schedule, and technical constraints. It has two components: (1) the
probability/likelihood of failing to achieve a particular outcome, and (2) the
consequences/impacts of failing to achieve that outcome.

Risk events are elements things that could go wrong in a project. The events should be
defined to a level where an individual can comprehend the potential impact and its
causes. For example, a potential risk event for a turbine engine could involve a turbine
blade vibration. Related to this vibration could be a series of potential risk events that
should be selected, examined, and assessed by subject-matter experts.

The relationship between the two components of risk—probability and consequence/
impact—is complex. To avoid obscuring the results of an assessment, the risk associated
with an event should be characterized in terms of its two components. As part of the
assessment, there is also a need for backup documentation containing the supporting data
and assessment rationale.

Risk management is the act or practice of dealing with risk. It includes planning for risk,
assessing (identifying and analyzing) risk areas, developing risk-handling options,
monitoring risks to determine how risks have changed, and documenting the overall risk
management program.




                                                                                             3
Risk planning is the process of developing and documenting an organized,
comprehensive, and interactive strategy and methods for identifying and tracking risk
areas, developing risk-handling plans, performing continuous risk assessments to
determine how risks have changed, and assigning adequate resources.

Risk assessment is the process of identifying and analyzing project areas and critical
technical process risks to increase the probability/likelihood of meeting cost, schedule,
and performance objectives. Risk identification is the process of examining the project
areas and each critical technical process to identify and document the associated risk.

Risk analysis is the process of examining each identified risk area or process to refine the
description of the risk, isolating the cause, and determining the effects. It includes risk
rating, quantification, and prioritization in which risk events are defined in terms of their
probability of occurrence, severity of consequence/impact, and relationship to other risk
areas or processes.

Risk handling is the process that identifies, evaluates, selects, and implements options in
order to set risk at acceptable levels given project constraints and objectives. This
includes the specifics on what should be done, when it should be accomplished, who is
responsible, and the associated cost and schedule. The most appropriate strategy is
selected from these handling options.

Risk monitoring is the process that systematically tracks and evaluates the performance
of risk-handling strategies throughout the project life cycle and develops further risk-
handling options, as appropriate. It feeds information back to the other risk management
activities of planning, assessment, and handling.

Risk documentation is recording, maintaining, and reporting assessments, handling
analysis and plans, and monitoring results. It includes all plans and reports for the
Program Officer (PO) and NSF and Awardee decision authorities.

Contingency is the amount added to an estimate to allow for items, conditions, or events
for which the state, occurrence, and/or effect is uncertain and that experience shows will
likely result, in aggregate, in additional costs.1
1.3        PURPOSE OF RISK MANAGEMENT

Risk is the degree of exposure to an event that might happen to the detriment or
benefit of a program, project, or activity. It is described by a combination of the
probability that the risk event will occur and the consequence of the extent of loss
or gain from the occurrence. Risk management is a structured, formal, and disciplined
approach, focused on the necessary steps and planning actions to determine and control
risks to an acceptable level. Risk management, along with Earned Value Management, is
one of the most important project management tools available to the PO and the
Awardee.
1
    From the AACE International Recommended Practice No. 10s-90 ―Cost Engineering Terminology‖


                                                                                                 4
Project risk management is the continuing application of the risk management
process throughout the project life cycle. Its purpose is to enhance the probability
of project success by increasing the likelihood of improved project performance,
thereby decreasing the likelihood of unanticipated cost overruns, schedule delays,
and compromises in performance, quality and safety. Risk is an inherent part of all
activities, whether the activity is simple and small, or large and complex. The relative
size and/or complexity of an activity may or may not be an indicator of the potential
degree of risk associated with that activity.

A key output from the risk analysis effort is the establishment of appropriate
contingency/reserves within the project cost estimates and the project schedules at
the confidence levels decided upon. A probabilistic approach is essential since a
simple algebraic addition of best case outcomes underestimates contingency and worst
case outcomes overestimates contingency.

Contingency is added to an estimate to cover ―known unknowns.‖ It includes items such
as planning and estimating errors and omissions, minor price fluctuations, design
developments and changes within the project scope, and variations in market and
environmental conditions. Contingency is generally expected to be expended during the
execution of the project. Contingency usually excludes items such as major scope
changes, (changes in end product specification, capacities, and building sizes),
extraordinary events such as major strikes and natural disasters, and escalation and
currency effects.2 In the Government project environment, contingency is often added to
the base project cost estimate to increase the probability that the project can be completed
within publicized cost and schedule objectives. This practice has been implemented in
part to meet Congressional expectations that cost and schedule baselines for Federal
projects will not be exceeded.

Contingency funds may be held or controlled by the Awardee, the NSF PO, or both. It is
recommended that the NSF PO retain some funds for contingencies that may arise during
project execution, and not immediately make those contingency funds available to the
Awardee. Otherwise, the only alternative to avoiding a cost overrun is to de-scope the
project.


1.4         SCOPE

Risk management is the continuing process of planning, identifying, quantifying,
responding to, and controlling risks to maximize the potential for the success of
an activity. The degree of application of risk management is to be commensurate
with a tailored approach, and is a management tool to maximize the results of
positive events and minimize the consequences of adverse events.



2
    Ibid, AACE International.


                                                                                           5
There are many different types of risks. For NSF purposes, risk management is not
defined as an environmental, safety, hazards, threat, health, business, or OSHA risk
assessment, and consequently, this Guide does not address the conduct of these specific
risk assessments. These independent assessments may, however, provide an input to the
risk management process based upon the potential (or likelihood) of events materializing
as risks that would increase project cost, cause schedule delays, reduce safety margins,
or reduce the performance of the final product.

Risk management can be applied to cost, schedule, technical performance
(i.e., risk associated with evolving a new design or approach), programmatic
performance (i.e., risk associated with obtaining and using resources
that can affect the project), and any other factors important to the management
decision process. Activity success means that the activity is technically feasible,
programmatically feasible, and can be completed within an established budget and
an established schedule. Conversely, activity failure can result from the
failure to meet any of these factors.

Achieving risk reduction is an integral part of setting priorities, sequencing
project work, and responding to the most serious risks first. Thus, the identified risks
must be prioritized. Risk is a dimension of work prioritization and an important (but not
the only) consideration in establishing prioritized sequencing of activities and other
decision-making processes.

The elements of the risk management process are shown in Figure 1-1.




                                        Figure 1-1



Numerous types of risk exist. Some examples of risk in different categories are
shown in Figure 1-2.
.
Risks may be grouped or sorted into different categories. The Department of



                                                                                            6
Defense identifies five facets of risk: technical, programmatic, supportability, cost, and
schedule. The Department of Energy discusses eight facets of risk, but recognizes that
safety, environment, disposition, support, and procurement are all technical risks.

The way one chooses to categorize risks is not important as long as the information
is used properly. In general, when the risks associated with a project are being evaluated,
all aspects of the project should be considered. While there is never a technical risk that
does not have a potential impact on cost and/or schedule, the converse is not true.
There are a number of cost- and schedule-driven administrative or management
factors that do not result from technical issues. While these can also have significant
impacts on cost and schedule, they do not need to address technology or
design issues.




                                        Figure 1-2


Any given risk may belong to more than one risk category. For example, a particular
piece of equipment may pose a technical challenge and have significant
programmatic implications (e.g., not available when needed).

Historically, estimating uncertainties have been included in project cost estimates


                                                                                             7
as ―traditional contingency‖. It primarily represents uncertainties in the project
cost and schedule estimates for the defined work scope that result from things such as,
but not limited to: errors and omissions, inflation, adverse weather, pricing variances,
quantity variances, and technical complexity. Base cost and schedule estimates are
usually developed by the estimator at the 50/50 level of probability of overrun or
underrun. Historically, contingency has often been estimated as a percentage of the base
cost of an item or activity (Work Breakdown Structure element) to cover the uncertainties
listed above and thereby increase the confidence level of the estimate to 80 or 90 percent
probability of underrun. However, this is not the preferred method of developing
estimates for contingency. The percentage was varied based on the maturity of the
design and/or project and the related uncertainty, based on the estimator’s judgment. For
complex projects that involve significant technology development or first-of-a-kind
scope/design uncertainties, the traditional contingency models may not be adequate. For
these projects, a systematic technical programmatic risk analysis methodology may be
used for evaluating needed contingency. This contingency includes the possible impacts
from technical and programmatic types of risk. In addition, the actions resulting from risk
response/risk handling strategies are included in project baseline scope and cost estimate.
Thus, risk assessment and contingency development are integral to each other.


1.5    RISK MANAGEMENT PROCESS STEPS AND METHODOLOGY

The steps involved in the risk management process have been defined variously by
different practitioners. At the NSF, we define the risk management process as
comprising the following five steps:

Risk Management Planning
Risk Identification
Risk Quantification
Risk Handling
Risk Monitoring and Control

Taken together, Risk Identification and Risk Quantification are often called Risk
Assessment. Risk Handling is also sometimes known as Risk Response Planning.

The following sections provide a detailed description of the five steps in the process
and describe at least one approach or methodology..

1.5.1 Risk Management Planning

A risk management plan should be developed at the onset of a project.
This plan is a living document used throughout the life of the project and should
therefore be under configuration management. The plan should identify the project
objectives and technical description, project assumptions, responsibilities for risk
management, and a description of the risk management process that will be followed—



                                                                                         8
including the participants, responsibilities, procedures, criteria, tools, and techniques to
be used to identify, quantify, respond to, and track project risks. Inherent in the project
description should be the identification of issues/exceptions with standardized practices
and procedures, such as:

       Unusual heat stress or exposure to cold situations
       Deviations from standard construction practices
       Requirements that could alter standard job plans or maintenance activities
       Limited access to medical facilities
       Work involving confined spaces, scaffolding, ladders, etc
       Nonstandard methods for compliance with OSHA

These issues should be documented to facilitate the identification of any risks associated
with them, as opposed to identification of tasks that can readily be defined and
costed as part of the project scope and baseline. While all applicable industry and
site safety, operations, and maintenance documents provide input to facilitate risk
identification, subject matter experts are generally the best source of information.
A risk management plan should also identify when during the project life cycle
the risk analysis (identification, quantification, and handling/response) will be performed
and updated. The level of detail in the plan, and the scope, timing, and level of
risk analysis should be commensurate with the complexity of the project. Risks
that are identified and quantified as low should have minimal follow-on activities.
The outline of a typical Risk Management Plan is shown in Table 1-1.

Risk planning is the detailed formulation of a plan of action for the management of risk.
It is the process to:
   Develop and document an organized, comprehensive, and interactive risk
    management strategy
   Determine the methods to be used to execute a PO’s/Awardee’s risk management
    strategy
   Plan for adequate resources.
Risk planning is iterative and includes describing and scheduling the activities and
processes to assess (identify and analyze), handle, monitor, and document the risk
associated with a project. The result is the Risk Management Plan (RMP).




                                                                                               9
Table 1-1 Elements of a Typical Risk Management Plan




                                                       10
                Table 14-9. Sample Format for a Risk Management Plan

INTRODUCTION. This section should address the purpose and objective of the plan, and
provide a brief summary of the project, to include the approach being used to manage the
project, and the acquisition strategy.
PROJECT SUMMARY. This section contains a brief description of the project, including the
acquisition strategy and the project management approach. The acquisition strategy should
address its linkage to the risk management strategy.
DEFINITIONS. Definitions used by the project office should be consistent with DOE definitions
for ease of understanding and consistency. However, the DOE definitions allow project
managers flexibility in constructing their risk management programs. Therefore, each
program’s risk management plan may include definitions that expand the DOE definitions to fit
its particular needs. For example, each plan should include, among other things, definitions for
the ratings used for technical, schedule, and cost risk.
RISK MANAGEMENT STRATEGY AND APPROACH. Provide an overview of the risk
management approach, to include the status of the risk management effort to date, and a
description of the project risk management strategy.
ORGANIZATION. Describe the risk management organization of the project office and list the
responsibilities of each of the risk management participants.
RISK MANAGEMENT PROCESS AND PROCEDURES. Describe the project risk
management process to be employed, i.e., risk planning, assessment, handling, monitoring and
documentation, and a basic explanation of these components. Also provide application
guidance for each of the risk management functions in the process. If possible, the guidance
should be as general as possible to allow the project’s risk management organization flexibility
in managing the project risk, yet specific enough to ensure a common and coordinated
approach to risk management. It should address how the information associated with each
element of the risk management process will be documented and made available to all
participants in the process, and how risks will be tracked to include the identification of specific
metrics if possible.
RISK PLANNING. This section describes the risk planning process, provides guidance on how
it will be accomplished, and describes the relationship between continuous risk planning and
this RMP. Guidance on updates of the RMP and the approval process to be followed should
also be included.
RISK ASSESSMENT. This section of the plan describes the assessment (identification and
analysis) process. It includes procedures for examining the critical risk areas and processes to
identify and document the associated risks. It also summarizes the analyses process for each
of the risk areas leading to the determination of a risk rating. This rating is a reflection of the
potential impact of the risk in terms of its variance from known best practices or probability of
occurrence, its consequence, and its relationship to other risk areas or processes. This section
may include:
• Overview and scope of the assessment process • Sources of information
• Information to be reported and formats                 • Assessment techniques and tools
• Description of how risk information is retained
RISK HANDLING. This section describes the risk handling options, and identifies tools that
can assist in implementing the risk handling process. It also provides guidance on the use of
the various handling options for specific risks.
RISK MONITORING. This section describes the process and procedures that will be followed
to monitor the status of the various risk events identified. It should provide criteria for the



                                                                                                 11
 selection of risks to be reported on, and the frequency of reporting. Guidance on the selection
 of metrics should also be included.
 RISK MANAGEMENT INFORMATION SYSTEM, DOCUMENTATION AND REPORTS. This
 section describes the management information system structure, rules, and procedures that will
 be used to document the results of the risk management process. It also identifies the risk
 management documentation and reports that will be prepared; specifies the format and
 frequency of the reports; and assigns responsibility for their preparation.




The Awardee/PO should periodically review the RMP and revise it, if necessary. Some
events such as: (1) the baselining of a project, (2) preparation for a major decision point,
(3) technical audits and reviews, (4) an update of other project plans, and (5) a change in
major project assumptions may drive the need to update an existing RMP.

Planning begins by developing and documenting a risk management strategy. Early
efforts establish the purpose and objective, assign responsibilities for specific areas,
identify additional technical expertise needed, describe the assessment process and areas
to consider, delineate procedures for consideration of handling options, define a risk
rating scheme, dictate the reporting and documentation needs, and establish report
requirements and monitoring metrics..

The Awardee’s/PO’s strategy to manage risk provides the project team with direction and
basis for planning. Initially formalized during a program’s concept phase and updated for
each subsequent program phase, the strategy should be reflected in the project’s
acquisition strategy. Requirement documents, known risks, and system and project
characteristics are sources of information for the Awardee/PO to devise a risk strategy
and begin developing a RMP. Since the project’s risks are affected by the Government
and Awardee team’s ability to develop and build the system, industry and academia can
provide valuable insight into this area of consideration.

The RMP is the road map that tells the NSF and Awardee team how to get from where
the project is today to where the PO wants it to be in the future. The key to writing a good
plan is to provide the necessary information so the integrated project team (IPT) knows
the objectives, goals, and the risk management process. Since it is a map, it may be
specific in some areas, such as the assignment of responsibilities for NSF and Awardee
participants and definitions, and general in other areas to allow users to choose the most


                                                                                               12
efficient way to proceed. For example, a description of techniques that suggests several
methods for evaluators to use to assess risk is appropriate, since every technique has
advantages and disadvantages depending on the situation.


1.5.2 RISK IDENTIFICATION

For most projects, risk management is not a one-time activity or project event; it is
a continuing process. Risk analyses will occur several times in the project life
cycle. Often a conceptual risk analysis is conducted to facilitate alternative
evaluations, determine the level of project management planning required, and the
level of technical information and development activity appropriate to the project.
Risk analysis for a project is typically performed and updated during each of the
life-cycle phases of the project. Frequent (weekly or monthly) reviews of the risk analysis
should be performed to identify new risks, to evaluate progress in risk handling
strategies, and to evaluate changes during the project concept/development and
implementation cycles.

Awardees treat risk differently from the Government because each views risk from a
different perspective. The PO, in executing his risk management program, needs to
understand the Awardee viewpoint.

Grant awardees/contractors typically divide risks into two basic types: business risks and
project risks. Business risk, in the broadest sense, involves the inherent chance of making
a profit or incurring a loss on any given contract. Project risk involves, among other
things, technical, requirement, and design uncertainties. A contractor’s efforts to
minimize business risks may conflict with a Government PM’s efforts to lower project
risk.

While the Government and awardees/contractors may have different views on specific
cost, schedule, and performance risk levels/ratings, they generally have (or should have)
similar views of the risk management process. One exception may be the requirements
placed by corporate management—that could conflict with the Government view of
project risk.


1.5.2.1 Selection of Assessable Elements

Assessable elements are discrete entities against which an effective risk analysis
is performed and the results evaluated to provide the input needed to make
necessary decisions. Dividing an activity, project, or program into smaller more
manageable elements enables the identification of risks in a structured manner.
For example, in attempting to evaluate the risk associated with two different
alternatives available to baseline a project design, the assessable elements might
be ―Alternative 1‖ and ―Alternative 2‖. Similarly, in evaluating manufacturing a
new widget, assessable elements might be the Product ―Widget‖ and the Process



                                                                                           13
―Manufacturing Facility‖. If the project involves design, construction, and operation
of a facility, the assessable elements can be the various functions or groupings
of functions (i.e., systems, subsystems, or functions). It can also be based on the
various elements in the Work Breakdown Structure (WBS) for the project. Note that
there is no right or wrong selection; some elements are simply
more conducive to future activities than others. In situations where multiple risk
assessments are conducted for the same project, it is not necessary that the same
assessable elements be used each time. In fact, it is most likely that the selection
of assessable elements will change throughout the project’s life cycle.


1.5.2.2 Risk Identification

Risk identification is an organized approach for determining which events are
likely to affect the activity or project, and documenting the characteristics of the
events that may happen with a basis as to why this event is considered a risk.
Identification relies on the skill, experience, and insight of project personnel and
subject matter experts, as well as the project manager and the PO. Risks should be
identified that are both internal (under project control) and external (beyond
project control).

Once risk areas have been identified, risk identification proceeds by clearly
documenting what risks are foreseen in each area. This includes not only the
issue or event, but specifically why this concern is an assessable risk to the
project. Whereas risk is generally considered in terms of negative consequences (e.g.,
harm or loss) in the project context, it is also concerned with opportunities that
result in positive outcomes. Therefore, risk identification may be accomplished
through cause and effect evaluation that indicates whether an outcome should be
avoided or encouraged.

Key sources of input to risk identification include:

       Activity or Project Descriptions (Scope Statements, etc.). The nature of the
        project will have a major effect. For example, a project involving proven
        technology may have significantly less risk when compared to a project
        involving new technology, which may require extensive development and thus
        have a higher risk.

       Other Activity or Project Planning Documents. The WBS may provide
        visibility into new innovations not readily extracted from scope statements,
        statements of work (SOWs), etc. Cost and/or time estimates may provide
        greater risks when developed from early or incomplete information. Procurement
        plans may identify unusual market conditions such as regional sluggishness
        or lack of multiple suppliers. Finally, the end user and the design agency
        may develop hazard lists that identify additional sources of risk. Any
        assumptions listed in a cost or schedule estimate or scope description are


                                                                                         14
        excellent sources for risk identification (what happens if the assumption is
        incorrect?).

       Historical Information—This information can be extracted from previous
        project files, personal remembrances, lessons-learned databases, and commercial
        databases.

Methods and tools for initiating identification of risk can vary, depending upon
the resources (project documentation, experience with similar projects, lessons
learned, knowledgeable personnel, etc) available. Risk identification can be
initiated by using risk source checklists (including categories for both technical
and programmatic risks), process flow charts, risk/activity templates, interviews
with subject matter experts, and team brainstorming. The tools are intended to
both stimulate the thought process of the Risk Analysis Team and supplement
their knowledge regarding potential risks.

Table 1-2 illustrates a typical checklist of risk categories. In using these check-lists,
the Risk Analysis Team evaluates each assessable element, one-by-one,
against each item in the risk category list, to determine whether anything in the
project presents a risk. The process continues until the entire checklist has been
considered. While the use of a template is similar to that of a checklist, using a
process flowchart helps to bring about a better understanding of each step in a
scenario and the interrelationships between steps. This type of evaluation considers
each of the steps involved in the process, one at a time, to determine the
potential that the step includes any risks. This method is most useful when new or
modified process steps are involved.

The results of the risk identification step are clear statements of risk with corresponding
bases. The event that creates the risk will be identified, as well as the affect the event
could have on the project or activity. This information should be documented in the Risk
Assessment Form used for the project. A typical Risk Assessment Form is shown in
Table 1-3, with associated instructions for completion of the form provided in Table 1-4.
Some of the information included in the form will be discussed in later sections of this
Guide.




                                                                                            15
Table 1-2. Risk Identification Checklist



                                           16
Add the attached Risks to the Table:




                                       17
                      Table 1-3. Typical Risk Assessment Form




                    Table 1-4. Risk Assessment Form Instructions


1.5.3 Risk Quantification

Risk quantification involves determining the probability of the occurrence of a risk,
assessing the consequences of this risk, and combining the two (probability and
consequence) to identify a ―risk level.‖ This risk level represents a judgment as to
the relative risk to the project as a whole and is categorized as Low, Moderate, or
High. Based on the risk level, handling strategies are identified to respond to the
risk.

A number of factors complicate this analysis including:

     A single risk event can cause multiple effects on a number of systems (ripple
      effect).
     Opportunities for one project participant may be considered detrimental by
      another.


                                                                                        18
       Mathematical techniques can cause false impressions of precision and reliability,
        i.e., results may only be indicators, not absolute measures.

Risk quantification may be performed quantitatively or qualitatively, depending
upon the project complexity and the preference of the analysis team. The end
result is the same in both cases.

Risk level determination can be done using a variety of techniques. This can be
done by determining the probability of the risk occurring and its consequence(s).
The probability of a risk occurring is usually a number or a grade and has no units
(dimensionless). However, consequences are usually measured in specific units
such as cost, schedule, or performance parameters. In the methods described below,
criteria are defined and used to convert the consequence(s) into a unit-less number
or grade. Later, the impact of risk on a project or activity is defined using units of
cost and schedule.

Table 1-5 shows typical criteria for defining probabilities and Table 1-6 shows
typical criteria for defining consequences. These probability and consequence
tables are used with both the qualitative and quantitative methods of risk quantification
discussed below. The criteria followed by asterisks in these tables must be
calibrated relative to the project. For example, the consequence definitions of
Negligible, Marginal, Significant, Critical, and Crisis may vary considerably from
a small to a large project. These tables should be provided as part of the RMP.




                              Table 1-5. Risk Probabilities



                                                                                            19
20
                              Table 1-6. Risk Consequences

Special attention must be given to first-of-a-kind risks because they are often
associated with project failure. First-of-a-kind risks should receive a critical or
crisis consequence estimate unless there is a compelling argument for a lesser
consequence value determination.

The output of the risk quantification process is a determination of the probability
of occurrence, the consequence of occurrence, and the risk level for each risk.
This information is documented in Sections B, C and D of the Risk Assessment
Form shown in Table 1-3. The risk quantification method chosen must be able to
provide this risk level based upon the judgment exercised in the analysis process
and be consistent with the implementing organization’s procedures. Numerous
methodologies can be employed to quantify risk. Whatever method is used,
documentation of the chosen methodology is recommended. Documentation
creates a record for future use in the event that a new team performs a later review,
revision, or update.

The two methods developed further in this section include:

      Qualitative—based upon the intersection of the qualitative probability and
       consequence values derived from Tables 1-5 and 1-6, respectively, using the
       Risk Level Matrix shown in Figure 1-3.
      Quantitative—based upon the product of the quantitative probability and
       consequence values derived from Tables 1-5 and 1-6, respectively.

1.5.3.1 Qualitative Approach (Risk Level Matrix)

This method begins by assigning qualitative values to event probability and
consequence(s) that will then be used to determine a qualitative risk factor. The
following steps provide the details of the method. The key features of this method
are that it:
     Allows independent assessment of the probability and consequence of a risk
     Provides qualitative definition of basis for the risk and risk level.

The qualitative methodology uses the risk level matrix shown in Figure 1-3.

Steps:
1. Address each risk statement from the risk assessment form individually.
2. Determine the qualitative probability of occurrence value (P) for each risk with
appropriate basis and justification. The probability of occurrence is for the
duration of all project phases or for the activity being assessed. Table 1-5
provides typical criteria for establishing probability values.
3. Determine the qualitative consequence of occurrence value (C) for each risk
with appropriate basis and justification. The consequence of occurrence is for
the duration of all project phases or for the activity being assessed. Table 1-6


                                                                                        21
provides typical criteria for establishing consequence values.
Assign a risk level based upon the intersection of the qualitative P and C values
on the 5x4 risk level matrix in Figure 1-3. Depending upon the activity and the
ability to differentiate the risk levels, other matrices may be chosen by the risk
analysis team.

Probability of Risk




                              Figure 1-3. Risk Level Matrix

1.5.3.2 Quantitative Approach (Probability x Consequence Equation)

This method begins by assigning quantitative values to event probability and
consequence(s) that will then be used to determine a quantitative risk factor. The
details of this method are outlined below. The key features of this method are that
it:

       Provides qualitative definition of basis for the risk, but quantitative inputs for
        risk level.
       Provides finer grading within the risk levels.

This method is useful for prioritization activities, either among alternatives where
numerous risks exist within the individual risk levels, or among risks in determining
where to allocate resources.

The quantitative methodology uses the Probability x Consequence Equation

RF = (P x C), where:

RF = Risk Factor
P = Probability of Occurrence


                                                                                             22
C = Consequence of Occurrence
Steps:
1. Address each risk statement from the risk assessment form individually.
2. Determine the quantitative probability of occurrence (P) for each risk with
appropriate basis and justification. The probability of occurrence is for the
duration of all project phases or for the activity being assessed. The probability
is expressed as a decimal between 0 and 1, where 0 is no probability of occurrence
and 1 is 100% probability of occurrence. Table 1-5 provides typical
criteria for establishing probability values.
3. Determine the quantitative consequence of occurrence (C) for each risk with
appropriate basis and justification. The consequence of occurrence is for the
duration of all project phases or for the activity being assessed. The consequence
is expressed as a decimal between 0 and 1. Table 1-6 provides typical
criteria for establishing consequence values.
4. Using the formula RF = P x C, determine the risk factor for each identified
risk.
5. Based on the following values, determine the risk level for each identified risk.
High Risk - RF is greater than 0.4 1
Moderate Risk - RF is greater than 0.1, but less than or equal to 0.4
Low Risk - RF is less than or equal to 0.1

     threshold ensures that risks with a mid-range (0.6) probability of Likely and a high-end (0.7)
1 This
consequence of Significant (and vice-versa) will be classified as High risks.




1.5.3.3 Other Risk Quantification Methods

Expected monetary value, expert judgment, simulation, and the use of decision
trees are other risk quantification methods that may be used. Expected monetary value is
the product of the risk event probability multiplied by the value of the gain or loss that
will be incurred. Schedule impacts and intangibles (i.e., a loss may put the organization
out of business) must be considered when using this approach.

Expert judgment is often used in lieu of, or in conjunction with, mathematical
techniques. For example, risk events could be described as having a very likely,
likely, unlikely, or very unlikely probability of occurrence and a crisis, critical,
significant, marginal, or negligible impact or consequence. Based on these descriptions,
the risk level matrix shown in Figure 1-3 can be used.

Simulation uses a model of a system process such as the project schedule to
simulate a project using Monte Carlo analysis to ―perform‖ the project many
times so as to provide a statistical distribution of calculated results. The use of
Monte Carlo analysis to estimate the risk cost distribution by statistically combining
risk costs is illustrated in Section 1.5.5.

A decision tree is a diagram depicting key interactions between decisions and
associated events (uncertainties) as understood by the decision-maker. This approach


                                                                                                      23
helps the analyst to divide a problem into a series of smaller, simpler, and more
manageable events that more accurately represent reality to simplify decision-making.


1.5.4 Risk Handling

Risk handling is the identification of the course of action or inaction selected for
the purpose of effectively managing a given risk. All identified risks shall be
handled. Risk-handling methods should be selected after personnel have determined
the probable impact on the project, so that handling strategies are selected
that identify the optimum set of steps to balance risk with other factors, such as
cost and timeliness. Responses to risks generally fall into one of four major
categories (reduce or mitigate, accept, avoid, or transfer) shown in Figure 1-4 and
are described in greater detail in the subsections that follow.




                         Figure 1-4. Risk Handling Strategies

The selected handling strategy, or strategies, should be documented in Sections E,
F, G, and H of the Risk Assessment Form shown in Table 1-3. Costs and schedule
impacts related to the scope of the selected risk handling strategies are added to the
project baseline cost and schedule. Thus, risk handling implementation costs are included
in the baseline cost.

1.5.4.1    Reduce and/or Mitigate



                                                                                        24
This strategy identifies specific steps or actions, which will increase the probability
that an activity will succeed, or, conversely, reduce the probability of the occurrence
of the risk or reduce or mitigate the consequence of a risk. The expected outcome of a
risk event can be reduced by lessening the probability of occurrence, e.g., by using
proven technology to lower the probability that the project will not work, or by
reducing the risk outcome by adding specific mitigation actions and any corresponding
cost implementation and schedule to the project scope. Using this strategy, the risk
remains, but at a reduced level. This reduced level is called the residual risk. This residual
risk will be statistically combined later with other residual risks to develop risk
contingency.

If the strategy is to reduce and/or mitigate the risk, then the cost and duration to
implement that strategy is determined and documented on the risk assessment
form. In addition, the probability, the consequence, and the risk factor and level
of the residual risk (i.e., risk after reduction and/or mitigation) are then determined.
The potential cost and schedule impact of the residual risk is identified
using three types of estimates: the best case (or most optimistic), the most likely,
and the worst case (or most pessimistic) estimate for establishing the cost distribution
probability for Monte Carlo simulations.

1.5.4.2    Accept

Accepting a risk is essentially a ―no action‖ strategy. Selection of this strategy is
based upon the decision that it is more cost effective to continue the project as
planned with no resources specifically dedicated to addressing the risk. However,
the ―no action‖ strategy may be hedged by developing a contingency plan in case
the risk event occurs and then tracking the risk to assure that it does not increase
during project execution. Low risks are typically accepted.

For a handling strategy of accept, the residual risk equals the initial risk because
this strategy does not change the risk level. The residual risk will be statistically
combined with other residual risks to develop contingency. If the risk is accepted
without additional actions, then the cost and duration of implementation is zero,
which is documented on the risk assessment form. The potential cost and schedule
impact of the risk is identified using three types of estimates: the best case (or
most optimistic), the most likely, and the worst case (or most pessimistic) estimate
for establishing the cost distribution probability for Monte Carlo simulations.

1.5.4.3 Avoid

This strategy focuses on totally eliminating the specific risk driving
event usually by eliminating the potential that the risk event can occur. This can
be accomplished through total structure, system, or component redesign, or by
selecting an alternate design approach, that does not include the particular risk.
The project will not be able to eliminate all risks, but specific risk events can
often be eliminated with this strategy.



                                                                                           25
If the strategy is to avoid the risk, the cost and duration of implementation of the
strategies is determined and documented. Once the strategy is implemented, the
risk level for the specific element will be reduced to zero. No residual risk remains
with this strategy. Generally, this method may be done in parallel with the up-front
requirements analysis, supported by cost/requirement trade studies, which can include
cost-as-an-independent-variable trades

1.5.4.4    Transfer

This strategy is used when a project scope with identified risks can be transferred
to another project or entity, especially when this same risk can be more easily
handled within the receiving project or entity. A risk can be transferred to an
outside organization by purchasing services to obtain technology outside of the
project. This in itself is a risky strategy in that the vendor can go out of business
or fail to meet the agreed requirements, leaving the project with the same initial
problem. In any case, the individual or organization receiving the risk must
accept the risk transfer. Another technique used to transfer risk is the use of insurance or
derivatives in the financial markets.

If the strategy is to transfer the risk, the cost and duration of implementation of the
strategies is determined and documented. Once the strategy is implemented, the
risk level for the specific element will be reduced to zero. No residual risk remains
with this strategy.


A sampling is listed below of the types of risk handling actions available to the
Awardee/PO.

      Multiple Development Efforts. Create competing systems in parallel that meet
       the same performance requirements.
      Alternative Design. Create a backup design option that uses a lower risk
       approach.
      Trade Studies. Arrive at a balance of engineering requirements in the design of a
       system.
      Early Prototyping. Build and test prototypes early in the system development.
      Incremental Development. Design with the intent of upgrading system parts in
       the future.
      Technology Maturation Efforts. Normally, technology maturation is used when
       the desired technology will replace an existing technology which is available for
       use in the system.
      Robust Design. This approach, while it could be more costly, uses advanced
       design and manufacturing techniques that promote quality through design.


                                                                                          26
   Reviews, Walk throughs, and Inspections. These three actions can be used to
    reduce the probability/likelihood and potential consequences/impacts of risks
    through timely assessment of actual or planned events.
   Design of Experiments. This engineering tool identifies critical design factors
    that are sensitive, therefore potentially high risk, to achieve a particular user
    requirement.
   Open Systems. Carefully selected commercial specifications and standards whose
    use can result in lower risks.
   Use of Standard Items/Software Reuse. Use of existing and proven hardware
    and software, where applicable, can substantially reduce risks.
   Two-Phase Development. Incorporation of formal risk reduction into project
    research and development. The first part of research and development is where
    prototypes are developed and tested. In the second part, models or pilot plant are
    developed and tested.
   Use of Mock-ups. The use of mock-ups, especially man-machine interface mock-
    ups, can be used to conduct early exploration of design options.
   Modeling/Simulation. Modeling and simulation can be used to investigate
    various design options and system requirement levels.
   Key Parameter Control Boards. The practice of establishing a control board for a
    parameter may be appropriate when a particular feature (such as system weight) is
    crucial to achieving the overall program requirements.
   Manufacturing Screening. For projects in research and development, various
    manufacturing screens can be incorporated into test article production and low
    rate initial production to identify deficient manufacturing processes.
   Insurance. Insurance can be purchased at an additional project cost to cover
    almost any possible risk event. This is an example of risk transfer.

   Derivatives/commodity futures. Various financial instruments can be purchased
    to cover items such as interest rate risks, currency fluctuations, etc. Some of these
    techniques may NOT be available to for use on Federal projects.

   Performance Bonds. Performance bonds may be required from subcontractors to
    ensure timely and acceptable performance of the contracted work scope.

   Fixed Price Contracts. Different contract types may be utilized to change the
    risk allocation between the contractor and the Government. For example, the use
    of fixed price contracts transfers the financial risk from the Government to the
    contractor, albeit at a higher price.




                                                                                        27
      Warranties and Guarantees. System/facility/product performance warranties or
       guarantees can be purchased from the supplier of an item, usually at an additional
       cost.

1.5.5 Risk Impact Determination

Risk impact determination is the process of evaluating and quantifying the effect of
risk(s) on the project. Risk impacts a project in two different ways:

      Handling strategy implementation, which must be reflected in a revised project
       Baseline.
      Residual risk, which must be reflected in project contingency.

The ultimate impact of risk management is to increase the probability of project/
activity success by focusing attention on problem areas early and reducing the
amount of costly rework in the future. For each and every risk, there is potential
cost or schedule impact if the risk occurs. The impacts of these risks on cost and
schedule must be addressed in the project estimates.

1.5.5.1 Handing Strategy Implementation

The first impact is the handling strategy implementation, which must be included
in the project cost and schedule baseline. If the risk is reduced using a risk reduction
or mitigation strategy, there may be a cost and schedule impact associated
with the implementation of that strategy as shown in Figure 1-5. The ―implementation‖
cost and schedule impacts of the risk mitigation strategy must be included
in the baseline project cost and schedule.




                                                                                           28
              Figure 1-5. Risk Handling Strategy Implementation Costs

1.5.5.2 Residual Risks

Even after risk-handling strategies have been implemented, there may be remaining
risk impacts, which are referred to as residual risks. The cost and schedule impacts
of residual risks must be included in the contingency calculations. This is
accomplished by determining a cost and/or schedule impact probability distribution
for each residual risk. These probability distributions are then combined
statistically through a Monte Carlo process (discussed below) to produce the contingency
estimate.

For the example shown in Figure 1-5, the contingency is $82 (at an 80 percent
confidence level), significantly less than the $235 algebraic sum of the worst case
residual risk costs.

Figure 1-6 illustrates the impact of risk handling on cost in another example. The
initial risk cost prior to handling is $48.630 million. The handling implementation
cost is $1.989 million, and the residual risk contribution to the project contingency,
using the Monte Carlo process at an 80% confidence level, is $7.371million.
The remainder of this section provides greater detail on the analysis of cost impacts
from risks and the use of an approach to determine the risk impact on
schedule.




                                                                                         29
                 Figure 1-6. Impact of Risk Handling on Project Cost

Cost Analysis Methods

There are a number of methods available for determining the impact of risk on a
project. One method is to assign a standard, flat percent contingency to the cost
estimate, as determined by the cost estimator and project manager. This method
can be termed the “flat rate contingency” method and is generally useful for
activities where estimating uncertainty is known, based on historical data and
experience. This flat rate calculation is applied individually to each function or
activity such as engineering or construction instead of applying it to the overall
project cost. The sum of the individual components becomes project risk.


                                                                                     30
The second contingency estimation method for projects with a number of moderate
or high risks is termed the “Monte Carlo simulation” method. This is performed
by defining the cost of each activity in terms of a cost profile, namely a
cost probability distribution. Once the profiles are known, they can be statistically
combined using the Monte Carlo simulation method. The result of the simulation will be
a project risk cost profile versus the probability of project success. This method is
extensively used in the insurance industry to determine insurance rates based on mortality
data. There are software tools such as Crystal Ball ® , Risk for Microsoft Project ® , or
Primavera ® Monte Carlo that can be used to do similar modeling. A similar analysis
approach could be used to determine the impact of risk on schedule. This process is sum-
marized below.

Application of the Monte Carlo Method

The Monte Carlo method uses individual cost vs. probability distributions for
each of the residual risks to statistically generate the overall cost vs. probability
profile. The simulation software also generates a sensitivity chart showing the
impact of the various risk-based cost elements on the overall distribution.
As noted above, the process begins with preparation of an input probability
distribution for each of the residual risks. In general, for each residual risk there is
a range of costs with the best case and worst case estimates. One of the distributions
commonly used for cost profiles is the triangular distribution shown in
Figure 1-7. Other distributions, such as normal, exponential, or beta, could be
used based on the available data and user experience/judgment. Figure 1-8
provides examples of some of these additional distribution functions that are
available in Crystal Ball ® .

For a triangular distribution, however, one needs only three data points for each
residual risk element, namely, the most likely or anticipated cost, the best case
cost, and the worst case cost. The most likely value falls between the best and the
worst case values, forming the triangular-shaped distribution, which shows that
the values near the minimum and the maximum are less likely to occur than those
near the most likely value. The various risk elements with their residual cost
versus probability profiles are provided as input to the model.




                                                                                           31
                                   Figures 1-7 and 1-8.

Calculation of the Total Residual Risk Cost Contingency Distribution

Once this data is obtained, the individual residual risk costs can be statistically
combined as shown in Figure 1-9 using Monte Carlo simulation to obtain the
overall project cost vs. probability profile. A total cost distribution is generated
using the random sampling methodology or Monte Carlo method. This is usually
done using a Monte Carlo software tool available from commercial vendors.
Crystal Ball ® software was used to generate the total cost distribution in this
model (see Figure 1-10).




                                                                                       32
                                  Figures 1-9 and 1-10.

Schedule Contingency

The residual risk impact on schedule has at least three effects, as follows:

      It potentially delays the completion of the specific task element(s).
      As a result of the slip, the task element(s) that precede or follow the affected
       element will also be impacted; this can result in a cost impact.
      Additional project cost (in the form of such things as overtime differential pay,
       etc.) may be incurred for delays in schedule completion.




                                                                                           33
For example, resources may have been staged to perform various project activities.
If one activity is delayed, there is a schedule impact. In addition, the resources
to perform the follow-on activities will have to be idled or allocated to other tasks or
activities which can result in demobilization and remobilization of manpower resources.
This results in a cost impact. The term ―hotel load‖ cost is used for the task of
―maintaining a core work group in a standby mode‖ when task element(s) are delayed.

The method to determine the impact on the schedule and establish a schedule
contingency is similar to the contingency analysis and uses the Monte Carlo
method. The schedule impact is determined for each residual risk element in the
form of ―best case,‖ ―most likely case,‖ and ―worst case‖ task duration estimates. Using
project scheduling software such as Primavera ® Monte Carlo or Microsoft Risk Plus®,
the schedule risk profile can be determined. The schedule contingency can be calculated,
based on the amount of risk that one is willing to take.

The ―hotel load‖ costs associated with the schedule contingency are also determined
for each residual risk element and the ―hotel load cost‖ contingency is calculated using
Monte Carlo method. This is termed ―cost of schedule contingency‖ and is added to the
cost estimate contingency.


1.6. Risk Reporting and Tracking

Risk reporting is the documentation of the risk identification, quantification,
handling, and impact determination activities for a project in a risk analysis report.
This report normally becomes a reference in the project’s overall risk management
plan for use in future risk analysis activities.

Risk tracking is the active monitoring of action items developed from risk handling
strategies and the identification of a need to evaluate new risks and /or reevaluate
changes in previously identified risks. Risk tracking can typically monitor the
following types of information:

      Accomplishment of detailed scheduled milestones, specifically as they apply to
       risk handling elements.
      Cost and schedule data including both monthly and periodically generated status
       Information. This would include EVMS data.
      Research and development studies, engineering studies, and science and
       technology roadmaps.
      Test results, especially for risky program elements.
      Project action item lists.

One useful tool is the Risk Register. The risk register lists the major project risks (high
risks), the risk handling strategy, the person to whom each risk has been assigned for
accountability purposes, the current status of the risk handling strategy, and comments.



                                                                                              34
This Risk Register should be reviewed weekly or at least monthly with the Awardee’s
management team and the PO.


Because the types of information and indicators being monitored are so diverse,
appropriate tracking tools will vary widely among projects. A tracking system
and tracking tools should be defined that are commensurate with the size and
complexity of the project. The selection and definition of a tracking system to be
used in a project is normally defined in the project’s RMP. Unfavorable trends from risk
tracking indicate either that risks were not fully or properly defined, or that handling
strategies were not adequate. In such cases, the risk analysis must be re-evaluated.


1.6 RISK DOCUMENTATION
A primary criterion for successful management is formally documenting the on-going
risk management process. This is important because:
      It provides the basis for program assessments and updates as the project
       progresses
      Formal documentation tends to ensure more comprehensive risk assessments than
       undocumented efforts
      It provides a basis for monitoring risk-handling actions and verifying the results
      It provides project background material for new personnel
      It is a management tool for the execution of the project
      It provides the rationale for project decisions.
Documentation should be done by those responsible for planning and collecting and
analyzing data, i.e., the Awardee’s IPT in most cases.

Risk management reports vary depending on the size, nature, and phase of the project.
Examples of some risk management documents and reports that may be useful to a PO
are:
      Risk Management Plan
      Risk Assessment Form
      Risk handling priority list
      Risk handling plan of action
      Risk Register (an example of a risk register template is provided as Table 1-7)
      Risk monitoring documentation:
      Project metrics
      Technical reports



                                                                                            35
      Earned Value reports
      Watch list
      Schedule performance report
      Critical resources reports.
      Test results
      Non-conformance reports

POs/Awardees can devise a list of standard reports that will satisfy their needs most of
the time. However, since there will always be a need for ad hoc reports and briefing and
assessments, on large projects, it may be advisable to store risk information in a
management information system. This allows deriving standard reports and creating of ad
hoc reports, as needed.


1.7 Risk Management and the Procurement Process
The NSF accomplishes most of its project efforts through the award of Cooperative
Agreements to primarily non-profit educational institutions. As such, the primary focus
of the PO is on the Awardee’s acquisition strategy/plan and on the approval of
subcontracts over $100,000 in value.
The Awardee’s acquisition strategy should describe how risk is to be handled and must
identify which risks are to be shared with the sub-contractor and which are to be retained
by the Awardee or the NSF. The key concept here is that the NSF shares the risk with the
Awardee/sub-contractor, but does not transfer risk to the Awardee/sub-contractor. The
PO always has a stewardship responsibility to the taxpayer to develop a capable system
and can never be absolved of that responsibility. Therefore, all project risks, whether
primarily managed by the PO or by the Awardee/sub-contractor, must be assessed and
managed by the PO.

The Awardee determines how much of each risk is to be shared with the sub-contractors,
in the acquisition strategy/plan. The Awardee should not require sub-contractors to
accept financial risks that are inconsistent with their ability to handle them. Financial
risks are driven, in large measure, by the underlying technical and programmatic risks
inherent in a program. The PO should, therefore, ensure that the Awardee selects the
proper type of contract based on an appropriate risk assessment, to ensure a clear
relationship between the selected contract type and project risk. An example would be the
use of cost-reimbursable-type contracts for development projects. The PO should utilize
the PAT for their project to provide assistance in reviewing the Awardee’s proposed
acquisition strategy/plan and any proposed sub-contracts requiring NSF approval.

The PO should also review proposed Awardee sub-contracts to ensure that the NSF does
not assume real or implied liabilities as a result of the contract type, scope, and/or terms



                                                                                          36
and conditions. Issues such as legal liability, property management, intellectual property,
operations and maintenance, and rights of way should be considered.

The intention here is to establish balance between cost, schedule, performance, and risk
early in the acquisition process and to manage to a cost objective.
Whereas the NSF has traditionally managed performance risk, equal emphasis must be
placed on managing cost and schedule risks. An underlying premise is that if costs are too
great, and there are ways to reduce them, then the NSF may reduce performance
requirements to meet cost objectives. Cost control and effective risk management involve
planning and scheduling events and demonstrations to verify solutions to cost, schedule,
performance risk issues.

Trade-off analysis is essential to attain a favorable balance between cost, schedule,
performance, and risk. The PO should identify risk and cost driving requirements during
the concept development phase of the project to know where tradeoffs may be possible.
Risk assessments are critical to the process since they provide managers with essential
data to assist in the cost, schedule, performance, risk trade decisions.

Effective risk management is also essential in performing facilities projects that are
international in scope or have multiple performing organizations; projects of this nature
are inherently more risky. The PO must take steps to identify and quantify the increased
risks and take risk handling actions to mitigate, transfer, or avoid the added risks
wherever possible. International projects will introduce increased risk in areas such as
government-to-government relations, government instability, currency fluctuations,
funding uncertainties, management complexities, project integration, transportation
requirements, import-export, and cultural differences.




                                                                                           37
                                                           Table 1-7. Risk Register Template


Id            Description of Risk              Likelihood      Seriousness       Grade      Change         Mitigation Actions             Responsible            Cost    WBS4
            Identify consequences3                                                                                                          Officer

1.1    Inadequate funding to complete               M                M              B         NEW       Re-scope project,               Project Manager           NA      
       the project                                                                                      focusing on time and
                                                                                                        resourcing
1.2    Lack of technical skills in Client           H                H              A           ↑       Develop training plan              Consultant            $2000
       Business Unit




3
  In larger projects, the consequences of the threat may not be evident, and noting them under each risk, or in a separate column can be useful in identifying
appropriate mitigation actions.
4
  WBS = Work Breakdown Structure, this is to indicate that the identified mitigation action has been included in the WBS (workplan).




                                                                                                                                                                  38
39
                   APPENDIX A: Risk Management Reference Documents
Document                                    Description
PMI Project Management Body of              Provides a high level description of the risk management process,
Knowledge, 2000                             tools, and techniques. Generally accepted as the foremost
                                            authoritative reference on project risk management.
Department of Energy Project                Provides a very detailed description of the project risk management
Management Practices Guide, Practice #      process, tools, procedures, and techniques. Includes a sample
14, February 1, 2003.                       Risk Management Plan.
DoD 4245.7-M, Transition from               Provides a structure for identifying technical risk areas in the
Development to Production, September        transition from a program’s development to production phases. The
1985.                                       structure is geared toward development programs but, with
                                            modifications, could be used for any acquisition program. The
                                            structure identifies a series of templates for each of the
                                            development contractor’s critical engineering processes. The
                                            template includes potential areas of risk and methods for reducing
                                            risk in each area.
Risk Management Concepts and                Devoted to various aspects of risk management.
Guidance, Defense Systems Management
College, March 1989.
Systems Engineering Management Guide,       Devoted to risk analysis and management and provides a good
Defense Acquisition University Press,       overview of the risk management process.
January 2001, Section 15.
Continuous Risk Management Guide,           Provides a risk management methodology similar to the one
Software Engineering Institute, Carnegie    described in the Deskbook. Its value is that is subdivides each
Mellon University, 1996.                    process into a series of steps; this provides useful insights.
                                            Appendix A describes 40 risk-management techniques, the
                                            majority of which are standard management techniques adapted to
                                            risk management. This makes them a useful supplement to the
                                            Deskbook identified techniques.
A Systems Engineering Capability Maturity   Describes one approach to conducting an Industry Capabilities
Model, Version 1.0 Software Engineering     Review. Section PA 10 (pp. 4-72–4-76) discusses software risk
Institute (Carnegie Mellon University),     management. The material presented in this handbook also can be
Handbook SECMM-94-04, December              tailored to apply to system and hardware risk.
1994.
A Software Engineering Capability           Describes an approach to assess the software acquisition
Maturity Model, Version 1.01 Software       processes of the acquiring organization and identifies areas for
Engineering Institute (Carnegie Mellon      improvement.
University), Technical Report, December
1996.
Capability Maturity Model for Software      This is a tool that allows an acquiring organization to assess the
(SM-CMM), Version 1.1,/CMU/SEI-93-TR-       software capability maturity of an organization.
24, February 1993.
Taxonomy-Based Risk Identification,         Describes a method for facilitating the systematic and repeatable
Software Engineering Institute, Carnegie    identification of risks associated with the development of a
Mellon University, CMU/SEI-93-TR-6          software-intensive project. This method has been tested in active
(ESC-TR-93-183, June 1993.                  Government-funded defense and civilian software development




                                                                                                          41
Document                               Description
                                       projects. The report includes macro-level lessons learned from the
                                       field tests.
NAVSO P-6071.                          Navy “best practices” document with recommended
                                       implementations and further discussion on the material in DoD
                                       4245.7-M.
Risk Management, AFMC Pamphlet 63-     An excellent pamphlet on risk management that is intended to
101, July 1997.                        provide PMs and the PM with a basic understanding of the terms,
                                       definitions, and processes associated with effective risk
                                       management. It is very strong on how to perform pre-contract
                                       award risk management.
Defense Acquisition Deskbook           Primary reference tool for defense acquisition work force; contains
                                       over 1,000 mandatory and discretionary publications and
                                       documents which promulgate acquisition policy and guidance.
                                       (http://www.deskbook.osd.mil)

Acquisition Software Development       Describes one approach to conducting an Industry Capabilities
Capability Evaluation, AFMC Pamphlet   Review. This two-volume pamphlet was generated from material
63-103, 15 June 94.                    originated at Aeronautical Systems Center. The concepts support
                                       evaluations during source selection and when requested by IPTs.
                                       The material presented in this pamphlet also can be tailored to
                                       apply to system and hardware risk management.
Risk Management Critical Process       Provides guidance and extensive examples for developing RFP
Assessment Tool, Air Force SMC/AXD,    Sections “L” and “M,” plus source selection standards or risk
Version 2, 9 June 1998.                management. Also includes technical evaluation and review
                                       questions, which are helpful for assessing a risk management
                                       process; and risk trigger questions, which are helpful for risk
                                       identification.
NAVSO P-3686, Top Eleven Ways to       Contains the Navy approach to risk management with baseline
Manage Technical Risk, October 1998.   information, explanations, and best practices that contribute to a
                                       well-founded technical risk management program.
USACE: Cost Risk Analysis Guidance     Useful for traditional construction projects.
Document, 1992.
U.S. EPA: Cost Risk Analysis for HTW   Useful for environmental remediation projects.
Remediation Projects, 1992
Construction Industry Institute:       Provides a very good overview of risk management for traditional
Management of Project Risks and        construction projects.
Uncertainties, 1989




                                                                                                    42
 APPENDIX B: –Combining Traditional and Probabilistic Risk-Assessment-Based
                         (PRA) Contingencies


Once the cost impact of residual risks has been identified, this cost – referred to as the
PRA contingency – may be combined with the traditional contingency and included in
the project cost estimate. There are various methods for accomplishing this, the simplest
being algebraic addition of the PRA contingency estimate and the traditional contingency
estimate. A more accurate reflection of this combined value can be established through
probabilistic addition of the traditional and PRA contingencies. The most thorough
treatment of risk impact is to incorporate the cost associated with each risk
directly into the cost of an identified project ―item,‖ along with the traditional
contingency. For example, assume the estimated cost for procurement and installation of
100 feet of pipe is $1,000. Traditional contingency variables of quantity, unit cost, labor
rates, etc. identify a distributed cost of between 90% and 125% of this value, or $900 to
$1,250. Project risks, such as unexpected radiological conditions encountered in the
construction area, unanticipated underground interferences, lack of integrity of the
existing system, etc., identify an addition to the cost distribution of -$0/+$400. This
results in a new distributed cost for the cost of the installed piping of between 90% and
165% of the estimated cost of $1000. The primary shortcomings of this method are:

      This cannot be applied unless the WBS levels have been identified in the estimate
      Many risks are identified that do not have a one-to-one alignment with a single,
       specific project element/WBS entry.

An alternative method for combining traditional and PRA contingency is to statistically
combine the final distributed base project cost estimate with the final, distributed PRA
contingency calculation of all risks identified for the project. If the
base project cost estimate is not provided as a distribution function model, an appropriate
model is generated to reflect the data. This process is illustrated by the following
example.

Suppose that the output generated by a project cost estimate yields the data in Table B-1
on the following page. Table B-2 shows how the traditional and PRA-based contingency
distributions are ―summed‖ into a combined probability distribution function.




                                                                                         43
Table B-1.



             44
Figure B-1. Combining Traditional and PRA Contingency




                                                        45

								
To top