Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Overview of Department of Defense (DoD) Software Engineering by xqo30826


									                  Overview of Department of Defense
                 (DoD) Software Engineering Initiatives
                                       Mr. Scott Lucero
                                 Deputy Director, Software Engineering
                                    Systems Engineering Directorate
                       Office of the Director, Defense Research and Engineering
                          12th Annual NDIA Systems Engineering Conference
                                            October 29, 2009
NDIA SE Conference-SW Initiatives
10/29/09 Page-1
                                    Elements of a DoD Strategy for
                                        Software Engineering
               • Support Acquisition Success
                          – Ensure effective and efficient software solutions across the
                            acquisition spectrum of systems, SoS and capability portfolios
               • Improve the State-of-the-Practice of Software
                          – Advocate and lead software initiatives to improve the state-of-the-
                            practices through transition of tools, techniques, etc.
               • Leadership, Outreach and Advocacy
                          – Implement at Department and National levels, a strategic plan for
                            meeting Defense software requirements
               • Foster Software Resources to meet DoD needs
                          – Enable the US and global capability to meet Department software
                            needs, in an assured and responsive manner

        Promote World-Class Leadership for Defense Software Engineering
NDIA SE Conference-SW Initiatives
10/29/09 Page-2
                                    NDIA Top Software Issues
                                        September 2006
               1. The impact of requirements upon software is not consistently quantified
                  and managed in development or sustainment. “SW Requirements”
               2. Fundamental system engineering decisions are made without full
                  participation of software engineering. “SE/SW Integration”
               3. Software life-cycle planning and management by acquirers and
                  suppliers is ineffective. “SW Sustainment”
               4. The quantity and quality of software engineering expertise is
                  insufficient to meet the demands of government and defense industry.
                  “Human Capital”
               5. Traditional software verification techniques are costly and ineffective
                  for dealing with the scale and complexity of modern systems.
                  “SW Testing”
               6. There is a failure to assure correct, predictable, safe, secure execution
                  of complex software in distributed environments. “SW Assurance”
               7. Inadequate attention is given to total lifecycle issues for COTS/NDI
                  impacts on lifecycle cost and risk. “SW COTS / NDI / Reuse”

NDIA SE Conference-SW Initiatives
10/29/09 Page-3
                            Top Software Issues - 2006 vs.
                          Software Systemic Findings - 2008
 National lDefense Industrial Association (NDIA)IA )
 N a tio n a D e fe n se In d u stria l A sso cia tio n (N D                      D U S D (A Systemic Analysis te
                                                                              ODDRE/SE & T ) S S E D ire cto raof
                  T o 7 Software Issues
                Top p 7 S o ftw a re Issu e s                                 Program S o ftw a reReview Findings
                                                                  P ro g ra m R e vie w Support S yste m ic A n a lysis F in d in g s
                        A u g u st 2006
                    September 2 0 0 6

                     S o ftw are H u m an C ap ital              S o ftw are H u m an C ap ital
                                                                                                  � R e so u rce s
                                                                                                  � Q u a lity L e ve l
                                                                                                  � E n g in e e rin g
                     S o ftw are R eq u irem en ts
                                                                  S o ftw are R eq u irem en ts   � M anagem ent
                                                                                                  � A cq u isitio n S tra te g y

                         S ystem s /S o ftw are                      S ystem s /S o ftw are
                                                                                                  � S yste m s o f S yste m s
                             In teg ratio n                              In teg ratio n           � In te ro p e ra b ility
                                                                                                  � T e ch R e fre sh

                       S o ftw are A ssu ran ce                    S o ftw are A ssu ran ce

                         S o ftw are T estin g                                                     �   S o ftw a re T e stin g *
                                                                  S o ftw are D evelo p m en t     �   S o ftw a re S u sta in m e n t/M a in te n a n ce*
                                                                                                   �   S o ftw a re C O T S /N D I*
                      S o ftw are S u stain m en t
                                                                                                   �   T e ch n o lo g y R e a d in e ss
                                                                                                   �   S o ftw a re A rch ite ctu re
                       S o ftw are C O T S /N D I
                                                                     S o ftw are M etrics         � S o ftw a re M e trics
                                                                                                  � EVM

                                                                   S o ftw are E n g in eerin g
                                                                                                  � P ro je ct P la n n in g
                                                                         M an ag em en t          � M a n a g e m e n t O ve rsig h t
                                                                                                  � S o ftw a re C o n fig u ra tio n M a n a g e m e n t
                                                                    K n o w led g e S h arin g    � P ro ce ss
                                                                                                  � R e p o rtin g

NDIA SE Conference-SW Initiatives
10/29/09 Page-4
                                    Current Software Engineering
               • Program Support
                          – Provide software support for acquisition program reviews.
                            Develop independent schedule and defect estimates.
               • Human Capital
                          – Software Acquisition Training and Education Workgroup:
                            Establish SW competencies across the acquisition career fields
                          – Reference Curriculum for Graduate Study of Software Engineering:
                            Version 1.0 completed this month, to be sustained by IEEE and ACM.
               • Advance the State of the Practice
                          – Software Sustainment, NDIA Software T&E Summit/Workshop
               • Policy and Guidance
                          – Earned Value Management, Military Handbook for Work Breakdown
                            Structures: MIL-HDBK-881.
                          – Oversight of Services’ SW Acquisition Process Improvement
NDIA SE Conference-SW Initiatives
10/29/09 Page-5
                                        Notional Example of
                                    Schedule Feasibility Analysis

   Current Plan (Dec 2013)
                                End Date   Cum Probability (%)
                               1/26/2014          1
                               3/17/2014          3
                                5/6/2014          8
                               6/25/2014         13
                               8/14/2014         22
                               10/3/2014         31
                             11/22/2014          41
                               1/11/2015         54
                                3/2/2015         65
  Review Team Projection       4/21/2015         75
  (50th percentile - Dec 2014) 7/30/2015
                                                 83                  Review Team Projection
                               9/18/2015         95                  (50th percentile Dec 2014)
                               11/7/2015         99
                             12/27/2015         100

            Likelihood of delivery to current
            schedule: less than 1%
                                                                     Current Plan
                                                                     (Dec 2013)

                                                                                12/7/2013   6/25/204   1/11/2015   7/30/2015

NDIA SE Conference-SW Initiatives
10/29/09 Page-6
                              Software and Systems Reliability

               • DoD has renewed emphasis on systems reliability and
                 lifecycle costs of shortfalls
                          – DDRE effort underway to consolidate software reliability guidance
               • Starting to use parametric models to project numbers of
                 latent software defects and discovery rates
                          –     Used to support:
                          –     Development of satellite launch plans
                                                                              Software Block Maturity – Nov 2008
                          –     Aircraft production decisions
                          –     Operational test readiness reviews
               • Gauging software reliability
                 using Mean Time to Defect
                 (MTTD) discovery

                                                                           System Testing DRs / Test Hr
                                                                                (DT events to IOT&E)

NDIA SE Conference-SW Initiatives
10/29/09 Page-7
                                    Software Human Capital Efforts

                   Software Industrial Base Study – July 2007
              There is a choke-point in availability of top-tier software
                      managers, architects, and domain experts.
      Supply of sufficiently trained SW developers is inadequate near-term.
       • Software Acquisition Training and Education (SATEWG)
                    – Chartered February 2008 by USD(AT&L) to add software competencies to
                      DoD’s 13 acquisition career fields
                    – Recent accomplishments:
                       − Developed software competency framework,
                       − Established SPRDE software competencies
                       − Gap analysis of SATEWG competency framework and DAU’s
                         Software Acquisition Management courses
                       − Current focus is on PM, Contracting and Test career fields
          • Graduate Software Engineering Reference Curriculum (GSwERC)
                    – Partnership with Industry and Academia
                    – Version 1.0 completed September 2009
                    – Transitioned to IEEE and ACM for sustainment
NDIA SE Conference-SW Initiatives
10/29/09 Page-8
                           Software Sustainment Challenges

               • Software intensive systems encourage*:
                          – Build-a-little, test-a-little, field-a-little risk reduction
                          – Incremental and spiral development efforts
                          – Concurrent planning, development and sustainment activities
               • No longer a natural ‘break point’ where software
                 development can be transitioned to a
                 sustainment organization
                          – Technical capability of Government sustainment organizations
                            reduced due to acquisition reform
               • Planning for software sustainment now a lost art
                          – Acquisition programs no longer produce MIL-HDBK-347 Computer
                            Resource Life Cycle Management Plans

                              Better planning needed to partition software work
                             among multiple developers and increase competition
NDIA SE Conference-SW Initiatives
10/29/09 Page-9
                                                     UNCLASSIFIED   *AIAA Infotech Conference 2009, Software Sustainment Challenges in Defense Acquisition
                         NDIA Software Test and Evaluation
                           Summit/Workshop – Sep 2009
               • Purpose: “Recommend policy and guidance
                 changes to emphasize robust software T&E
                 approaches in Defense acquisition.”
               • Speakers from Government, Industry and
               • Conducted workshops on:
                          – How much software T&E is enough
                          – Software T&E involvement across the lifecycle
                          – Emerging paradigms: SOA, SoS, Security
               • Workshops specifically addressed:
                          – Policy & guidance, Human capital, RFP language,
                            SW T&E tools
               • NDIA Software Experts and DT&E sub-committee
                 to produce white paper by December 2009
NDIA SE Conference-SW Initiatives
10/29/09 Page-10
                               Software Measurement and Analysis
                                       Improvement Areas

                                                                       Find best Earned Value
                                                Generate software
                          Determine better                              Management (EVM)
                                                 appropriate WBS
                             methods of                                   practices for SW
                            obtaining cost
                           estimating data      Improve estimation
                                                                       Link quality indicators
                                                tools, techniques, &
                                                                              to EVM

                         Concepts - Requirements - Arch/Design - Development - Maintenance

                                    Integrate software guidance into proven
                                            management techniques

NDIA SE Conference-SW Initiatives
10/29/09 Page-11
                                       Software Earned Value
                                    Management (EVM) Study/Pilot
               • Develop methods to combine EVM and software
                 metrics to predict cost and schedule overruns
               • Piloted on a 5-year ACAT 1D software
                 development program
               • Pilot indicator shows estimate-at-completion
                 (EAC) forecasts for:
                          – Existing program management plans
                          – Milestone-based EVM measures
                          – Software metrics, i.e, growth profile of size, effort, defects

                        Equivalent EAC forecasts provide an increased
                                 confidence in project plans
NDIA SE Conference-SW Initiatives
10/29/09 Page-12
                                   Estimates at Completion (EAC) for
                                  Metrics, Earned Value, Program Plans
                                                                                                                                                                                                      EV EAC
                                                                                  SW Metric Spiral - Sample Summary
                                                                                      Performance Measurement Baseline

                               $1,600                                                                                                                                                             Program
                                          BAC = $1420
                                                                                                                                                     $1,559    $1,579                               Plan
                               $1,400                                                                                             $1,318    $1,471                                               Completion
                               $1,200                                                                                  $1,251
                                                                                                                                                                  EV Metrics
                 DOLLARS ($K

                                                                                               $941                                                               Calculated Schedule Range:
                                                                                                             $1,032                                               09/08/08 - 10/12/08
                                $800                               $706                                                                                           Calculated Estim ate at Com plete
                                                                                               715                                                                $1,797K - $2,515K
                                $600                 $478                                             $565
                                                                                                                                                                  Cum ulative CPI = 0.79
                                                                                                                                                                  SW Metrics
                                $400                                                                                                                              Calculated Schedule Range:
                                                                          $291                       Data Date                                                    05/14/08 - 06/18/08
                                        $71                                                                                                                       Calculated Estim ate at Com plete
                                  $0                                                                                                                              $1 570K $1 710K

                                   Oct07            Nov07 Dec07                  Jan08     Feb08         Mar08         Apr08     May08      Jun08    Jul08     Aug08 Sep08         Oct08     Nov08    Dec08

                                                                                                                            TIME (Months)
                                                     BAC                                BCWS (PV)                               BCWP (EV)                     ACWP (AC)                      ETC

                                              Confidence increases as EACs overlap
                                         Multiple measures reaching the same conclusion
NDIA SE Conference-SW Initiatives
10/29/09 Page-13

                                                          Contact Information:
                                                          Don Scott Lucero
                                                 Deputy Director, Software Engineering
                                    Systems Engineering Directorate, Defense Research and Engineering

NDIA SE Conference-SW Initiatives
10/29/09 Page-14
                                                              UNCLASSIFIED                              14

To top