Docstoc

The Institute of Risk Management Managing Portfolios and

Document Sample
The Institute of Risk Management Managing Portfolios and Powered By Docstoc
					Title Slide

 The Institute of Risk Management
 Managing Portfolios and Programmes
 … a personal perspective from
 David Booth, Head of Risk & Opportunity Management Consultancy,
 BAE Systems Integrated System Technologies
 29th April 2008
About BAE Systems

 BAE Systems is a global company engaged in the
 development, delivery and support of advanced defence
 and aerospace systems in the air, on land and at sea.

•   A global capability with Customers in over 100 countries
•   97,500 highly skilled people
•   No. 3 largest global and No. 6 US defence company
•   Global sales exceed £15bn per annum
•   Order book exceeds £38bn




INTEGRATED SYSTEM TECHNOLOGIES
   The ge
     len
C hal




The business context (and challenge) of programme / portfolio risk
Diversity, and complex synergies:
Customers, Technologies, Solutions, Geography, People & skills,
Approaches, Commercial environments, Timescales, Teams.
   The ge
     len
C hal

About Integrated System Technologies

 Equipped to face the demands of the rapidly evolving
 market in national security & resilience, defence and
 complex, mission critical solutions

•   Outstanding skills and expertise from our business heritage
•   2007 Turnover - £595 million
•   2007 Orderbook - £1.5 billion
•   3800 people - 75% skilled engineers


                     • ~ 20 significant, challenging projects
                     under monthly senior review.
                     • > 500 individual projects, overall
                                                        !
INTEGRATED SYSTEM TECHNOLOGIES
   The ge
     len
C hal

Integrated System Technologies Diverse Footprint
                                         Integrated System Technologies
              Land                          Sea                     Joint and Air                 Security & Resilience




   Commander         Dagger        CVF              T45             CVF         Lightning 2 JSF      Spider     Surveillance




   Land Training     FALCON       NMRR            SAMPSON      Mission System        UAV             DSU      Video Management




                                 Talisman          PAAMS         C4ISTAR             ICE




                                              Naval Training                    Ground Station



INTEGRATED SYSTEM TECHNOLOGIES
   The ge
     len
C hal

Our locations (13 UK sites + overseas activities)
                                                                                     Portfolios & programmes
                                                                                    (multiple projects against
                                                                                      collective objectives)
                                                                                      often run over several
                                                                                               sites
                                                                    Dunfermline




                                             Warton


                                                                                               Waterlooville

                                                                                       Sunbury
                                 Cwmbran
                                                                                         Chelmsford
                                    Filton                                             New Malden

                                                                                                    Frimley
                                                                                  Portsmouth
                                                                        Cowes
                                                 Dorchester
                                                              Christchurch



INTEGRATED SYSTEM TECHNOLOGIES
   The ge
     len
C hal

Meeting our customers’ changing needs

In Home markets and across the world
• Governments and Agencies
• Prime contractors & End Users

  • We deliver solutions: Studies, Visualisations,
  Experimentation, Consultancy, custom & COTS Hardware
  & Software products, System Integration, turnkey Military
  Platform & Civil Installations, Partner & Subcontractor
  Management, Training, Support.
  • Contracted against: Deliverable items, Performance,
  Through-Life Capability, Availability, ‘Results’ (e.g. Trained
  Crews), Savings against legacy benchmarks
  • Measured by: Reputation, Delivery & Financial
  performance (our Integrated Business Plan), Shareholders

INTEGRATED SYSTEM TECHNOLOGIES
   The ge
     len
C hal

Meeting our customers’ changing needs

In Home markets and across the world Risks to
                                individual project
• Governments and Agencies         solutions …
• Prime contractors & End Users

  • We deliver solutions: Studies, Visualisations,
                                            … can have rolled-
                                              COTS Hardware
  Experimentation, Consultancy, custom &up consequences
                                            across contracts
  & Software products, System Integration, turnkey Military…
  Platform & Civil Installations, Partner & Subcontractor
  Management, Training, Support.
                                       items, Performance,
  • Contracted against: Deliverable … and ultimately distil
  Through-Life Capability, Availability, ‘Results’ (e.g. level.
                                       to the business
  Trained Crews), Savings against budget benchmarks
  • Measured by: Reputation, Delivery & Financial
  performance (our Integrated Business Plan), Shareholders

INTEGRATED SYSTEM TECHNOLOGIES
   The ge
     len
C hal

… and Partnerships: multiple relationships   ,   and




INTEGRATED SYSTEM TECHNOLOGIES
   The ge
     len
C hal

Some other factors we need to contend with …

• Multiple, overlapping lifecycles – start dates & Milestones set by customers.
     • Every quarter, the business looks (and feels) ‘different’…
• Competitive bidding: uncertainty in the ‘balance’ of the Orderbook
• External constraints and drivers
     • Time windows for installations; e.g. within ship refit periods
     • Customer Budget (payment) profiles vs. ‘shape’ of work to be done
• International market considerations
     • Political, currency, procurement priorities & ‘seasons’ – drought and flood
• A single ‘problem’ can ripple through a number of projects
     • From the ‘top’ or other external source, or from within the supply chain / business
• Significant and dynamic Change in requirements and our Offerings
     • History needs to be continually re-calibrated: “project B is only 65% like project A”
• Firm business resource planning + a ‘shadow’ forward load profile, related to
  possible risk occurrences. [Contracts from a few months to 10’s of years]

INTEGRATED SYSTEM TECHNOLOGIES
When concentrating on the immediate task, have you stopped to
wonder how risks ‘down the line’ might affect your success?




INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp




Responding to the Challenge
The environment, process, people and tools
… and some new thinking
   The se
     on
R esp

Portfolio / Programme Risk: Essential Dimensions


• Robustness in the business, but with managed flex
     • Frameworks
     • Reviews
     • Accountability                   The following material considers
• Cross – domain working                some of the specific aspects of
     • Organisation                     building a portfolio / programme
     • Planning                           risk perspective, touching all
                                                   three areas.
     • Working styles
• Management of Individual Projects
     • Information from data
     • Information Management
     • Analysis and decision - making

INTEGRATED SYSTEM TECHNOLOGIES
The Business Infrastructure
   The se
     on
R esp

The BAE Systems environment

• Strong - & simple - Operational Framework
It “sets out the way we do business and what it means to
   be part of our Company”, and includes
     • Mandated Policies and Processes
     • Core business Processes – Lifecycle Management (LCM)
         • Mandating a systematic Project Risk & Opportunity Management process
         • A substantial body of Good Practice Guidance
         • Training, Maturity Model and Competence Framework
         • Tailoring to meet individual Business needs
           A common approach to managing risk is a powerful asset
                       Meeting Industry Benchmarks
                 Fully integrated into the Business model
                      Not an add-on, but a real add-in
INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

Key Operational Features 1/2

• The Lord Weinstock legacy: effectively a profit and loss account
  accountability ‘down’ at the project level – the PM is on point
           • Rigorous challenge and containment – no £‘bleed’ between projects
       On the face of it contrary to portfolio principles, but in reality the
                 foundation of knowing exactly where we are.
   Groups of projects are owned in hierarchical management structures
• Comprehensive Performance and Gate Review structures (5 are mandated)
           • Well defined risk footprint, with independent validation
           • Strong at both the pre-contract and delivery stages
        Transparency in debating and setting risk appetite at the outset
  Includes a drumbeat review of forecast outturn in the Contract Review,
   in which related projects pass under scrutiny within sight of each other

INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

Key Operational Features 2/2

• Mature and comprehensive Management Accounting regime, with robust
  Assurance and Control measures
     A critical foil against potential over-focus on risk pseudo science (!)
   These stakeholders have turned out to be critical to progress made to
    date. Constructive tension in developing approaches. Key advocates
                           when making progress.


• Developing common language
          Fundamental that the stakeholder groups calibrate their local
                                  terminology.
                                 Harder than it looks.


INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

 Project Risk Visibility & Action Hierarchy


   Escalation for                               Embodied
  senior attention                               within
   (exceptional)                                Corporate
                                               Governance

   Awareness &                                   Cross –
   Action within                                business &
   Business Risk                                functional
       (rare)                                 representation

  Managed within                                 Strong
  the Project Risk                             discipline,
  process domain                                regular
      (routine)                                monitoring



INTEGRATED SYSTEM TECHNOLOGIES
Internal appreciation is OK, but could it blunt your perception of the
forward challenge?




INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

The context of Project Risk and Opportunity Management
                                                                                 The Insyte Risk
The big ‘risk’ picture                                                            & Opportunity
                                                Project
                                                                                   Management
                                                 Risk
                                 Supply
                                                                                   Plan (ROMP)
                                                                Insurance           actively ties
                                 Chain
                                                                                 together all the
                                                                                   risk domains
                      Business                                            Fraud
                      Continuity                Insyte                  Prevention
                                                ROMP                           This apparently
                                                                                    simple
                           Business                                  Bonds &
                             Risk                                   Guarantees    (& obvious)
                                                                               development is
                                                          OAS
                                                                               a major factor in
                                          IBP             QBR
                                                          ARB                  being joined-up
                                                                               in our approach
INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

The context of Project Risk and Opportunity Management

Key project
                                                Project
risk & opp.                                      Risk
domain                           Supply
                                                                Insurance
relationships:                   Chain

•Mainly with
the Supply            Business                                            Fraud
                      Continuity                Insyte                  Prevention
Chain
                                                ROMP
•Calibrating
with BCM,                  Business                                  Bonds &
Business                     Risk                                   Guarantees
Risk &                                                    OAS
Insurance, as                             IBP             QBR
                                                          ARB
required.
INTEGRATED SYSTEM TECHNOLOGIES
Wasn’t it explained to you what being a subcontractor in a much
larger programme would involve?




INTEGRATED SYSTEM TECHNOLOGIES
The Quality of Information in Projects
   The se
     on
R esp

Insyte Project Commitments

• Insyte takes on commitments it can meet
• We deal in a high expectation market place
• Customers expect us to do what we say
• However, in competition, we also need to know how to cover all our
   exposures efficiently
• (When we are open-book customers also expect this)


• Striking the balance is never easy (!)



INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

Insyte Delivery – the confidence view

• Our risk appetite to win is balanced by the need to deliver in contract

• We pitch high confidence solutions and plans at the outset
     • Simple treatments consider all reasonable consequences in the work-up
     • Probabilistic models suggest a range of outcomes with associated
       probabilities of achievement – we use values towards the higher confidence
       ‘end’ of the scale


• In delivering, we account for the confidence level provided by the current
  understanding of our position
     • It can be an experience – based view of the project / review community
       (necessary when only a single-point assessment is available)
     • When a probabilistic assessment is in place, supported by a computer
       simulation, the output can be very useful, and supports the judgement

INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

About Project Risk and Opportunity Management. It:
• Is a well-established business process

• Is a project responsibility

• Applies throughout the lifecycle: continuously from bid to delivery

• Has numerous interfaces: data, people and other processes

• Helps us plan to handle events that might spoil our project success

• Helps us find and exploit improvements in our projects

• Is key to modelling confidence in projects’ outturn

• Addresses Cost + Schedule and Performance

INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

Definitions
• Risk - An event or a series of events which, on occurring, would damage
  a project objective in terms of performance, functionality, time of delivery,
  acceptance, or cost.

• Uncertainty - The range of outcomes associated with estimating work;
  an attribute describing the possible spread of cost or duration, due to the
  availability of information on the task itself or the estimating method used.

• Opportunity - An event or series of events which, on occurring, would
  offer benefit to the project in terms of performance, functionality, time of
  delivery, acceptance, or cost.




INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

Definitions (footnote)
• Risk - An event or a series of events which, on occurring, would damage
  a project objective in terms of performance, functionality, time of delivery,
  acceptance, or cost.

• Uncertainty - The range of outcomes associated with estimating work;
  an attribute describing the possible spread of cost or duration, due to the
  availability of information on the task itself or the estimating method used.

• Opportunity - An event or series of events which, on occurring, would
  offer benefit to the project in terms of performance, functionality, time of
  delivery, acceptance, or cost.

• Issue – A ‘live’ concern or matter which poses an immediate or definite
  threat, and which (simply) requires action to resolve it.
                Issues are not taken onto the risk register.


INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

 Uncertainty vs. Risk

• Uncertainties: we know that they will happen, but we are
  not exactly sure how large / long / costly they will be,
  within a range.

• Risks: we are not sure if they will happen or not, and we
  may also not know about their effect(s), other than that
  they lie within a range.




INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

 Uncertainty or Risk?
          The factors affecting your drive to work will affect your
          journey track record in contrasting ways…
     •   Traffic conditions                •   Puncture or breakdown
     •   Unlucky traffic light sequences   •   Run out of petrol
     •   Weather conditions                •   Stoppage due to accident ahead
     •   Demisting time                    •   Collision
     •   Time of departure                 •   Burst water main
     •   School holidays                   •   Road closure due to VIP visit

     •Looking back over a year, the uncertainties will be
     responsible for, say, a 10 minute variability in your journey.

     Any single risk occurrence is likely to have added half an hour
     to one of the times, if it happens.
INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

 Uncertainty vs. Risk

                       Uncertainty and Risk need different treatment

• Uncertainty is covered in confidence in the baseline model …
• Risk event effects are carried on the register, and are an additional factor
  in the model …
• We always try to improve the clarity with which we:
    • Account for both, avoiding overlap & underlap
    • Understand their collective effect(s)

• We need to be clear: is project outturn range driven by uncertainty or
  risk???


                           Widget factory vs. Software development
INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

 Uncertainty vs. Risk                 •Software Office struck by
                                      747/Hurricane/Fire overnight.
                                      •24 hour recovery plan
   •Normally widgets cost and Risk need different treatmentswings into
                 Uncertainty
   5p +/- 0.01p to make.              action; data recovery + new PCs =
                                      only 2 days lost.
 • •Widget machine fails =
   Uncertainty is covered in confidence in the baseline model …
   three weeks down-time              •6 mths later later… the project is
 • Risk event effects are carried on the register, and are an additional factor
   waiting for sprangle
   in the model …                     declaring completion at the latest
                                      planned date, due
   We always try to improve the clarity with which we: to requirements
 • flange replacement.
                                      ‘now understood’, low productivity
     • Account for both, avoiding overlap & underlap
     Performance utterly              and higher complexity + ‘as many
     • Understand their collective effect(s)
      dominated by risk               defects as we thought at worst’…
 • We need to be clear: is project outturn range driven by uncertainty or
   risk???                                  Performance driven by
                                                        uncertainty

                           Widget factory vs. Software development
INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

What the process does for the project and the business
At the outset…
• Determines how much schedule float the project may need
• Establishes how much £contingency should be carried in project costings
• Provides information to drive spend to save investment judgements
     • Investing in mitigating actions to bring down – and hold down – the levels of
       contingency and float required

                                                  Portfolio / programme issues.
As the project proceeds…
• Provides information to the project team members
  – and a stimulus for action that really makes a difference
• Provides a picture of confidence in project outturn for formal Review, and
  whether Risk or Uncertainty is the main driver – and drives improvements
• Captures the amount of potential threat still ‘hanging over’ the project, so
  profit is taken safely – a key shareholder requirement

INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

The Focus of Risk: where interests overlap
                                 Customer Stakeholders
                                                            The Risk Focus




          Project                                           Functions
     Stakeholders                                           (Commercial,
                                                            Finance etc.)

                                                         Beware risk at overlaps!
                                                         Good communications
                                                         are essential, and need
                                                         to be worked at
                                     Supply Chain
                                                         through the lifecycle.
INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

Adjacent process links: must define & work the interfaces
                                 Customer Stakeholders




                                                           g
                                                       atin
                                                               The Risk Focus
                     Pla




                                                   im
                    sch nning                            &      n
                                                        g ctio




                                                Est
                       ed u &
                                                     in
                           ling
                                                   am ele
                                                Te ier S
                                 Project          pl
          Project                Risk &       S up     Functions
     Stakeholders              Opportunity             (Commercial,
                               Management              Finance etc.)
                             e                Pro
                         a ng               Rep ject
                      Ch g’                     orti Fina
                 n ts, rin                          ng      n
                                          Make / buy

             em e n -e e                               & T ce:
          uir lutio
         q o                                               ra d
                                                               ing
       Re ‘S
          &
                               Supply Chain

INTEGRATED SYSTEM TECHNOLOGIES
Has your interest in brilliant process implementation reduced your
view of what really matters – i.e. the risks & actions?




INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

A Portfolio Approach to Risk Registers 1/2

• Investment in common proprietary toolset for all projects
• Training and live support are important
• All data held in a ‘business-shaped’ tree (like windows explorer)

1. Predict! Risk Controller
  – Risk Register
  – Qualitative Risk Assessment
  – Action management
  – Reporting
2. Predict! Risk Analyser
  - Quantitative Risk Assessment
  - Monte Carlo
3. Home-grown data audit tool
  – Supports Risk Controller use
  – An aid to Project Governance

INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

A Portfolio Approach to Risk Registers 2/2

Consistent data standards are key.
Must derive information from many entries.
With this data, we can investigate, for example….
• How often does Supplier X appear in the individual
   registers? Across all or sub-groups?
• What is the aggregated value of risk related to them?
• Over what time frame may it impact (incl. FY view)?
• Is there a common cause of risk across a number of projects?
• How well co-ordinated is mitigation investment to any common cause of
   risk? Are projects doing their own thing? Would a strategic (senior)
   intervention be helpful?
• What can be done to ease common risk effects, e.g. by improving
   resilience in resource planning?

INTEGRATED SYSTEM TECHNOLOGIES
   The se
     on
R esp

Using Portfolio / Programme Risk Information

• Recognise action routes which can benefit more than one project
     • Root cause analysis will distil common risk causes, Action taken here is
       against these is always good value.
• Mitigation Investment strategies to the common good
     • Direct at causes to reduce probabilities
     • Direct at reducing the extent of multiple impact effects, the benefit of which
       may be greater than the sum of the parts


• Reflect the benefit achieved in the individual risk Assessments – project
  Governance must hold. Accurate records are essential.

• Be prepared to find that things are worse than they appear from a single
  project’s perspective, until the Action delivers…


INTEGRATED SYSTEM TECHNOLOGIES
Are your mitigation investments sensible, and good value in
protecting against the potential loss?




INTEGRATED SYSTEM TECHNOLOGIES
Confidence Models in Decision Support
THERE IS NO SUBSTITUTE FOR JUDGEMENT, THOUGH
The context and build up of ‘confidence’ in outcome
                                                           Confidence is usually
                                                            Confidence is usually
          Status of:                                       derived via an experience –
                                                            derived via an experience –
    • Requirements                                         based assessment of
                                                            based assessment of
                                               Track
                                                Track
    • Solution Maturity                       Record       project data. It may also be
                                                            project data. It may also be
                                               Record
    • Acceptance                                           supplemented by figures
                                                            supplemented by figures
                                                           from an outcome simulation
                                                            from an outcome simulation
                                                           model (Monte Carlo).
                                                            model (Monte Carlo).
                                 Schedule
                                  Schedule                    Risk
                                                                Risk
                                 and Cost
                                  and Cost   Confidence
                                              Confidence     position
                                                              position
                                 position
                                  position


A definition of confidence:
 A definition of confidence:
“A statistical term describing
 “A statistical term describing              Contract
                                              Contract
                                                                         Status of:
the belief we place, from a                  Review                  • Schedule
 the belief we place, from a                   Review                • Resources
probabilistic point of view, on               debate
                                               debate
 probabilistic point of view, on                                     • Supply Chain
the likelihood of an outcome”
 the likelihood of an outcome”
INTEGRATED SYSTEM TECHNOLOGIES
Simulation or modelling outputs:
The distribution profile represents the population of possible outcomes


• The ‘Confidence’ figure is a parameter extracted from a probabilistic
   simulation of a project. Its integrity depends on the quality of the model
   used to generate it, and of the input data.
• The Monte Carlo model simulates project outcome by taking random
   samples of the input cost data ranges and generates multiple outcome
   scenarios, each representing one way the project could turn out.
• The simulation is predicting many possible cost outturn positions, each
   with a likelihood of happening – which can be translated into a
   ‘confidence’ figure



INTEGRATED SYSTEM TECHNOLOGIES
  Simulation or modelling outputs:
  The distribution profile represents the population of possible outcomes

 Number of times                  Full Range of Cost Outturn
  the simulation
 comes up with a
  particular cost                                           The Most Likely
     outturn                                             outcome is represented
                                                           by the tallest stack
      ≡ Relative                                             (most commonly
                                                            occurring answer)
     Likelihood




                                       ML                          Predicted
                                                                   Project Cost
INTEGRATED SYSTEM TECHNOLOGIES
  Simulation or modelling outputs:
  Confidence and project cost

 Number of times                        Full Range of Cost Outturn
  the simulation                  LowerConfidence           Higher Confidence
 comes up with a
  particular cost                                                The Most Likely
     outcome                                                  outcome is represented
                                                                by the tallest stack
      ≡ Relative                                                  (most commonly
                                                                 occurring answer)
     Likelihood




                                                                        Project Cost


                                               ML                        Predicted
                                 Lower Confidence          Higher Confidence
                                                                         Project Cost
INTEGRATED SYSTEM TECHNOLOGIES
  Simulation or modelling outputs:
  Introducing the language of confidence

      Relative
     Likelihood                                       50% point: half of the
                                                     population lies below this
                                                       cost, and half above



                                                              90% point: 90%
                                                              of the population
                                                               lies below this
                                                               cost, and 10%
   10% point: 10%                                                   above
   of the population
    lies below this
    cost, and 90%
         above

                                 (10%)   (ML)(50%)   (90%)     Predicted
                                                               Project Cost
INTEGRATED SYSTEM TECHNOLOGIES
Simulation or modelling outputs:
 Confidence and project cost


• Confidence is represented by a percentage on a sliding (non linear)
   ‘cursor’ scale against the range of cost outturn predictions.
• In principle;
     • For a given cost + contingency, a confidence %age can be read off to indicate
        the prospect of maintaining margin in full.
     • Conversely, by selecting a confidence %age, the level of cost + contingency
        needed to maintain margin at that confidence level can be read off




INTEGRATED SYSTEM TECHNOLOGIES
  Simulation or modelling outputs:
  Confidence levels associated with cost

    Relative
   Likelihood




                                                             Confidence level
                                                             scale, overlying
                                                              linear Cost +
                                                            Contingencies axis.




                                 10%   ML 50% 60% 70% 80%   90%    Predicted
                                                                   Project Cost
INTEGRATED SYSTEM TECHNOLOGIES
Simulation or modelling outputs:
  Confidence and project cost

• Confidence represents the number of simulations that have been
  run that fall below the cost figure. i.e.


     “if I ran this project 100 times in the same conditions and used the

        ‘80% cost figure’ to calculate margin, then:

     • in 20 instances there would be some margin erosion,

     • but in 80 instances there would be some margin improvement.”




INTEGRATED SYSTEM TECHNOLOGIES
Simulation or modelling outputs:
  Confidence levels associated with cost

• The relationship between cost and confidence, over the ranges of interest, is
  rarely linear. It is likely to vary in character markedly for different projects
  and to vary through the lifecycle.
• The key characteristic is the relative cost of ‘buying’ increased confidence
  or, conversely, the relative confidence penalty arising from a cost reduction
• Commonly, the confidence per £ ratio decreases towards the top end of the
  scale, but depends strongly on the project type
   • For a low risk project, if the ‘journey’ from 60% to 70% costs £100k, it
      could cost another £200k to achieve 80% (twice as much for the second
      decade)
   • For a high risk project, the figures could be £100k and £600k resp. (six
      times as much for the second decade). 90% could cost ~£1.2M (!)


INTEGRATED SYSTEM TECHNOLOGIES
Simulation or modelling outputs:
The cost of confidence – a lower risk project

    Relative
                                               Increasing cost of confidence
   Likelihood




                                 10%   ML 50% 60% 70% 80%      90%        Predicted
                                                                        Project Cost
INTEGRATED SYSTEM TECHNOLOGIES
  Simulation or modelling outputs:
  The cost of confidence – a higher risk project

    Relative
   Likelihood                                  Increasing cost of confidence




                                 10%    ML 50% 60% 70%     80%       90% Predicted
                                                                       Project Cost
INTEGRATED SYSTEM TECHNOLOGIES
  Simulation or modelling outputs:
   The cost of confidence – lower and higher risk projects compared
 Relative                                                    The beginnings of a
Likelihood
                                                              portfolio model...

                                                                       Higher Risk




                                 10%    ML 50% 60% 70%   80%     90%         Cost
    Relative
 Relative
   Likelihood
Likelihood
                                                                        Lower Risk




                                  10%   ML 50% 60% 70% 80%     90%   Cost
INTEGRATED SYSTEM TECHNOLOGIES
What modelled “Confidence” means to margin

• The confidence percentage is an indicator of how successful the project
  is likely to be at each cost figure within the predicted range.
• The confidence level at the baseline cost marks the point at which the
  erosion of As Contracted margin starts to grow as costs increase, and
  where margin improvement starts to grow, as costs decrease.
• The shape of the distribution is indicative of the incremental erosion that
  may occur should corrective action not be successful.
  This will vary project by project.
• Generally, dropping below a notional confidence threshold is rarely a
  ‘cliff-edge’ catastrophe.
• Understanding the relationship with margin is key to making the most of
  confidence data.

INTEGRATED SYSTEM TECHNOLOGIES
What modelled “Confidence” means to margin
At the start…

                      Margin
                 Improvement


 As Contracted Margin


                       Margin
                 Deterioration

   For any set of projects bid at
       Relative                                             For a project bid at, say,
     over 50%, the collective
     Likelihood
    probabilities are higher for                            the 80% confidence cost
        improvements than                                     figure, an outturn at a
    deteriorations. (There is a                            higher figure will result in
      greater population with                                Margin erosion, while a
   outcomes which have more                                cost reduction will give an
      margin than less). This                                     improvement.
   comprises a ‘portfolio effect’
   where the majority do better
  than the As Contracted figure,
      and we have a smaller
     population with reduced        10%   ML   50%   80%   90%      Predicted Cost
              margin.
INTEGRATED SYSTEM TECHNOLOGIES
What modelled “Confidence” means to margin
If there is cost growth…

                      Margin
                 Improvement


 As Contracted Margin


                       Margin
                 Deterioration

      Relative
       Relative
     Relative
     Likelihood
      Likelihood
    Likelihood




                                                               Confidence in achieving
                                                               the As Contracted
                                                               Margin falls




                                  10%
                                 10%10%    ML 50%50%
                                          MLML 50%      80%
                                                          80%     90%
                                                            80% 90%90% Predicted Cost Cost
                                                                           Predicted
                                                                         Predicted Cost


INTEGRATED SYSTEM TECHNOLOGIES                         70%
What modelled “Confidence” means to margin
                     Margin
   These figures are
     only indicative.
                                     In many cases, the numerical ratio between
                                     confidence %age variation and margin threat is high
   Significant work is
  needed to determine                                  Confidence Range
    this trade space.                                    Typ. +/- 10%?


 As Contracted Margin            Margin Variance:                                  Cost with
                                 +/- 5% of margin?                                 associated
                                                                                   confidence
                                                                                   level.
                                                     Confidence Range
                                                       Typ. +/- 6%?
          Contract
        Review Trade
                                      In higher risk projects, the ratio between confidence
           Space
                                      %age variation and margin threat will reduce, as the
                                      margin deterioration curve rolls off more steeply



INTEGRATED SYSTEM TECHNOLOGIES
So, what would you do, with this information?

• Data quality is critical – how sensitive are the models to any
  inaccuracy?
• What is driving the middle section of the distribution? After all
  this is the ‘most likely’ region…
• What is driving the tail shape. Long tails are killers. What can
  be done to wind it in? (Max values of 3-point input estimates
  need scrutiny…)
• Does your management system allow ‘bets’ based on a
  modelled population? Can you say that I expect (plan for) x%
  of my projects to “fail”?
• The reality depends on maturity & proving the principles,
  over time.

INTEGRATED SYSTEM TECHNOLOGIES
Have you really thought about how big a risk you may be taking?




INTEGRATED SYSTEM TECHNOLOGIES
Now, the real world
Moving on from the theoretical analysis.
‘Traditional’ single point retirement model: net exposure
£



          Net Exposure
       “Gross less factored”               Gross (Baseline)




                                 Factored (Baseline)
                                                              Time




INTEGRATED SYSTEM TECHNOLOGIES
‘Traditional’ single point retirement model: net exposure
£
                                                              Required to be
                                                              accounted for
                                                               in the Margin
      Net Exposure profile                                     Trading Plan
                                           Gross (Baseline)




                                 Factored (Baseline)
                                                                               Time




INTEGRATED SYSTEM TECHNOLOGIES
‘Traditional’ single point retirement model: profile I
£                            Simple (mechanistic) representation of
                                  Contingency holding tracking the
                                 predicted risk retirement baseline
     NB: Net Exposure profile
                                                     factored values

        Factored Baseline
                                         Gross Baseline



                                                               Time




INTEGRATED SYSTEM TECHNOLOGIES
‘Traditional’ single point retirement model: profile II
£
                                        A more practical representation of
                                        Contingency holding following risk
     NB: Net Exposure profile
                                         retirement factored values at the
                                                     latest forecast values
                                                                   & dates.
    Factored Baseline
                                                  Gross Baseline


                                   TC
                                                                     Time
                             now
     Factored Actual                           Factored Forecast



INTEGRATED SYSTEM TECHNOLOGIES
‘Traditional’ single point retirement model: profile III
£                         Typical example of a judgement to retain
                              a higher Contingency holding against
                              late-occurring big hitting risk, or other
     NB: Net Exposure profile
                                  latent concern, with a consequent
                                                   effect on Trading.
    Factored Baseline
                                           Gross Baseline


                                   TC
                                                                 Time
                             now
     Factored Actual                    Factored Forecast



INTEGRATED SYSTEM TECHNOLOGIES
Single versus three-point risk cost estimates
• Single-point values tend to represent a conservative (safe) expected
  position, with confidence in the numbers generated through robust review
  processes.
• When risk impact assessments are 3-point, then the estimates are:
    • middle value ≡ Most Likely (ML), min value ≡ Best, max value ≡ Worst.
• The values are inputs to the Monte Carlo model which provides an additional
  perspective, including an indication of the contingency needed to support a
  given Confidence level.

• Generally, the ML’s would tend to be estimated lower than the ‘traditional’
  single point values, as the estimator covers the poorer outcomes when
  setting the maximum value.
                                 Single-point
                     Min.        ML                       Max
INTEGRATED SYSTEM TECHNOLOGIES
CR Risk Report Set for a three-point risk treatment
• Risk Report including register and confidence components:
     • Most Likely Gross Impact estimates & probabilities for each
     • Factored Values (ML Impact X Prob)
     • Confidence Funding to achieve chosen confidence level, derived
       from the Confidence Graph
• Confidence Graph: Monte Carlo cost model output
     • Shows confidence levels supported by contingency (pre & post
       mit)
     • Confidence supported by contingency can be read off, or
     • Contingency required to give a given confidence can be read off
• Retirement Profile
     • Shows current Confidence track record                                                                                                Post Mit: 30/06/2008


                                                                                                  9000                                                                                                        100




     • Shows Gross & Factored Baselines, Actuals + Forecast lines,                                8000



                                                                                                  7000
                                                                                                                                                                                                              80




       based on Register and Confidence Funding component                                         6000




                                                                                                                                                                                                                    % Confidence level
                                                                                                                                                                                                              60




                                                                                 Contingency £K
                                                                                                  5000



                                                                                                  4000
                                                                                                                                                                                                              40




     • Expectation that contingency will follow Factored values but also allow                    3000



                                                                                                  2000
                                                                                                                                                                                                              20


                                                                                                  1000




       for the confidence funding component                                                         0                                                                                                         0




                                                                                                         Nov-07




                                                                                                         Nov-08




                                                                                                         Nov-09




                                                                                                         Nov-10
                                                                                                         Jun-07
                                                                                                          Jul-07




                                                                                                         Jan-08




                                                                                                         Jun-08
                                                                                                          Jul-08




                                                                                                         Jan-09




                                                                                                         Jun-09
                                                                                                          Jul-09




                                                                                                         Jan-10




                                                                                                         Jun-10
                                                                                                          Jul-10




                                                                                                         Jan-11




                                                                                                         Jun-11
                                                                                                          Jul-11
                                                                                                         Mar-07
                                                                                                         Apr-07
                                                                                                         May-07


                                                                                                         Aug-07
                                                                                                         Sep-07
                                                                                                         Oct-07

                                                                                                         Dec-07

                                                                                                         Feb-08
                                                                                                         Mar-08
                                                                                                         Apr-08
                                                                                                         May-08


                                                                                                         Aug-08
                                                                                                         Sep-08
                                                                                                         Oct-08

                                                                                                         Dec-08

                                                                                                         Feb-09
                                                                                                         Mar-09
                                                                                                         Apr-09
                                                                                                         May-09


                                                                                                         Aug-09
                                                                                                         Sep-09
                                                                                                         Oct-09

                                                                                                         Dec-09

                                                                                                         Feb-10
                                                                                                         Mar-10
                                                                                                         Apr-10
                                                                                                         May-10


                                                                                                         Aug-10
                                                                                                         Sep-10
                                                                                                         Oct-10

                                                                                                         Dec-10

                                                                                                         Feb-11
                                                                                                         Mar-11
                                                                                                         Apr-11
                                                                                                         May-11
                                                                                                                                                  Expiry Dates

                                                                                                         Total TC (CSR figure)              Total MC (CSR figure)                 Current Month
                                                                                                         Gross Exposure (Actual / F'cast)   Factored Exposure (Actual / F'cast)   Gross Exposure (Baseline)
                                                                                                         Factored Exposure (Baseline)       TC Confidence                         TC+MC Confidence




INTEGRATED SYSTEM TECHNOLOGIES
 Three-point retirement model (I)
£                           Simple representation of Contingency
                              holding, based on the ‘ML’ factored
                                 register values, with a (tapering)
   NB: Net Exposure profile
                                       confidence funding overlay

     Factored ‘ML’ Baseline
                                             Gross Baseline

       Supported by Register
                                                              Time



                                 Confidence Funding


INTEGRATED SYSTEM TECHNOLOGIES
 Three-point retirement model (II)
£                           Simple representation of Contingency
                              holding, based on the ‘ML’ factored
                                 register values, with a (tapering)
   NB: Net Exposure profile
                                       confidence funding overlay

     Factored ‘ML’ Baseline
                                         Gross Baseline

      Supported by Register
                                                             Time
                             now

         Confidence Funding          Confidence Funding
     (Actual) indicated by current        (Forecast)
        Monte Carlo analyses.
INTEGRATED SYSTEM TECHNOLOGIES
Forecasting the confidence funding requirement

• The overall Contingency level is likely to be similar to that for the single-
  point analysis, but the supporting model is a better representation of the
  project’s circumstances. Risk retirements still drive the profile, but are
  likely to be more accurately modelled in context.
• Forecasting the confidence funding layer for Trading Planning requires
  similar judgements as are required for the single-point forecast, but the
  process is supported by a higher quality model, in some respects.
     • Starting point can be a proportionate uplift on the Most Likely component
     • ‘Manual’ judgements are still needed to account for significant project events,
       latent high net exposure (killer risks) when contingency is reducing, and
       experience – based assessments
     • What-if simulations can simulate likely future conditions to provide deeper
       insights
     • Regular re-modelling builds confidence in the Confidence analyses

INTEGRATED SYSTEM TECHNOLOGIES
 Three-point retirement model (III)
£                      Illustrating manual adjustments to both the
                        register-based and the confidence funding
                         layers in the Contingency forecast, due to
   NB: Net Exposure profile   judgements on maintaining sufficient
                                         coverage of late-occurring
   Factored ‘ML’ Baseline                                ‘killer’ risk.
                                         Gross Baseline

      Supported by Register
                                                                Time
                             now

         Confidence Funding          Confidence Funding
     (Actual) indicated by current        (Forecast)
        Monte Carlo analyses.
INTEGRATED SYSTEM TECHNOLOGIES
A continuing journey to portfolio / programme assessment

• The basic toolkit is there
• Data quality must be proven
• A supplement to greybeard review
• Comparative review methods
  will evolve as familiarity with the
  confidence analysis takes root
• Stakeholder management and
  education is key.




INTEGRATED SYSTEM TECHNOLOGIES
BAE Systems Integrated System Technologies Limited
Victory Point
Lyon Way, Frimley, Camberley
Surrey, GU16 7EX
United Kingdom
Telephone +44 (0) 1276 603000
Fax +44 (0) 1276 603001                              Questions?
email insyte@baesystems.com
www.baesystems.com/insyte           david.i.booth@baesystems.com

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:10/10/2013
language:English
pages:73