EIAANSI 632 as a Standardized WB

Document Sample
EIAANSI 632 as a Standardized WB Powered By Docstoc
					        University of Southern California
       Center for Software Engineering




        EIA/ANSI 632 AS A
       STANDARDIZED WBS
         FOR COSYSMO
                                            2005 NASA Cost Analysis Symposium
                                                                  April 13, 2005


Marilee Wheaton, The Aerospace Corporation
Ricardo Valerdi, The Aerospace Corporation
                 & University of Southern California

                                                                                   1
        University of Southern California
       Center for Software Engineering




                                            Agenda
•   Background on COSYSMO & SE Standards
•   Use of EIA/ANSI 632
•   Effort Distribution Across EIA 632 Processes
•   Lessons Learned
•   Preliminary Results
•   Value of Systems Engineering




                                                     2
           University of Southern California
          Center for Software Engineering




            USC-CSE Affiliates (34)
•   Commercial Industry (15)
     – Daimler Chrysler, Freshwater
       Partners, Galorath, GroupSystems.Com, Hughes, IBM, Cost Xpert
       Group, Microsoft, Motorola, Price Systems, Rational, Reuters
       Consulting, Sun, Telcordia, Xerox
•   Aerospace Industry (7)
     – BAE Systems, Boeing, General Dynamics, Lockheed
       Martin, Northrop Grumman, Raytheon, SAIC
•   Government (8)
     – DARPA, DISA, FAA, NASA-Ames, NSF, OSD/ARA/SIS, US Army
       Research Labs, US Army TACOM
•   FFRDC’s and Consortia (4)
     – Aerospace, JPL, SEI, SPC


                                                                       3
        University of Southern California
       Center for Software Engineering




               COSYSMO Overview
• Parametric model to estimate system engineering
  costs
• Includes 4 size & 14 cost drivers
• Covers full system engineering lifecycle
• Developed with USC-CSE Corporate
  Affiliate, ISPA, SCEA, SSCAG, PSM, and INCOSE
  participation

                                                                   Operate, Ma
                                                                               Replace
                                            Oper Test Transition   intain,
  Conceptualize          Develop                         to                     or
                                            & Eval                 or
                                                      Operation                Dismantle
                                                                   Enhance



                                                                                           4
            University of Southern California
           Center for Software Engineering



           COSYSMO Data Sources
Raytheon                             Intelligence & Information Systems (Garland, TX)
Northrop Grumman                     Mission Systems (Redondo Beach, CA)
Lockheed Martin                      Transportation & Security Solutions (Rockville, MD)
                                     Integrated Systems & Solutions (Valley Forge, PA)
                                     Systems Integration (Owego, NY)
                                     Aeronautics (Marietta, GA)
                                     Maritime Systems & Sensors (Manassas, VA)
General Dynamics                     Maritime Digital Systems/AIS (Pittsfield, MA)
                                     Surveillance & Reconnaissance Systems/AIS
                                     (Bloomington, MN)
BAE Systems                          National Security Solutions/ISS (San Diego, CA)
                                     Information & Electronic Warfare Systems (Nashua,
                                     NH)
SAIC                                 Army Transformation (Orlando, FL)
                                     Integrated Data Solutions & Analysis (McLean, VA)
           University of Southern California
          Center for Software Engineering




COSYSMO Operational Concept
# Requirements
# Interfaces
# Scenarios                 Size
# Algorithms                Drivers
                                               COSYSMO       Effort
                            Effort
                            Multipliers
         - Application factors
             -8 factors                        Calibration
         - Team factors
             -6 factors
                                                      WBS guided by
                                                      EIA/ANSI 632
                                                                      6
        University of Southern California
       Center for Software Engineering



  COSYSMO Effort Multipliers
• Application Factors                       • Team Factors
   – Requirements understanding               – Stakeholder team
   – Architecture understanding                 cohesion
   – Level of service requirements            – Personnel/team
   – Migration complexity                       capability
   – Technology Maturity                      – Personnel
                                                experience/continuity
   – Documentation Match to Life
     Cycle Needs                              – Process maturity
   – # and Diversity of                       – Multisite coordination
     Installations/Platforms                  – Tool support
   – # of Recursive Levels in the
     Design

                                                                         7
               University of Southern California
              Center for Software Engineering



       7-step Modeling Methodology
         Analyze Existing
         literature

         1          Perform
                    Behavioral Analysis

                    2               Identify Relative
                                    Significance

                                    3              Perform Expert-
                                                   Judgement, Delphi
                                                   Assessment
                                                   4        Gather Project Data


                                                            5      Determine Bayesian
                                                                   A-Posteriori Update
                                                                         Gather more data;
                                                                  6      refine model
Determine statistical significance

                                                                         7
                                                                                             8
             University of Southern California
            Center for Software Engineering


Software Cost Models                                DBA COCOMO
                                                                        Other Independent
                                                        2004            Estimation Models
COCOMO 81              COCOMO II                                     COCOTS           COSYSMO
   1981                  2000                       COINCOMO          2000              2002
                                                       2004
                                                                     COSoSIMO      Costing Secure
                                                                       2004         System 2004




                       iDAVE                     COPLIMO        COPSEMO            Security
COQUALMO
                        2003                       2003           1998          Extension 2004
  1998


Software Extensions                                        COPROMO     CORADMO
                                                             1998        1999


      Legend:
      Model has been calibrated with historical project data and expert (Delphi) data
      Model is derived from COCOMO II
      Model has been calibrated with expert (Delphi) data
           Dates indicate the time that the first paper was published for the model                 9
            University of Southern California
           Center for Software Engineering




 Conceptual View of Unified Model: Initial Efforts


                COSOSIMO

                COSYSMO
                                                COCOMOII extensions
COSIZEMO                                        •RAD, security
                COCOMOII/                       •Incremental,               Overlap
                                                phase/activity
                COQUALMO                        •Agile, risk, Monte Carlo   Resolver
                                                •ROI (product line,
                COCOTS                          dependability)
                                                •Maintenance




                                                                                       10
           University of Southern California
          Center for Software Engineering



EIA/ANSI 632 Processes for Engineering Systems
      – Developed in 1998 by ANSI, INCOSE, et al
      – 5 major categories
          • 13 processes
                  – 33 activities

•   Acquisition and Supply             •       Product Realization
     – Supply Process                           – Implementation Process
     – Acquisition Process                      – Transition to Use Process
•   Technical Management               •       Technical Evaluation
     – Planning Process                         – Systems Analysis Process
     – Assessment Process                       – Requirements Validation Process
     – Control Process
•   System Design                               – System Verification Process
     – Requirements Definition Process          – End Products Validation Process
     – Solution Definition Process

                                                                                    11
                  University of Southern California
                 Center for Software Engineering



  Heritage of Standards for Systems Engineering
                                                                                 2002             2002
                                                                               ISO/IEC         ISO/IEC
                                                                                15504           19760
                                                                                (FDIS)           (PDTR)

                                                                                 1998
                                                                1994             EIA              2002
                                                              EIA / IS           632           ISO/IEC
                                                                632                             15288
                                                                               (Full Std)
                                               1994
                                                          (Interim Standard)                     (FDIS)
                        1974                Mil-Std-
                                                                 1994
  1969                Mil-Std-               499B                                                1998
                                                                IEEE             1998         EIA/IS 731
Mil-Std-               499A              (Not Released)
                                                                1220            IEEE            SE CM
  499
                                                              (Trial Use)       1220        (Interim Standard)
                                                                               (Full Std)


                                                                                                  2002
                                                                                              CMMIsm
    Legend                                                                                   SE/SW/IPPD
         Supersedes
         Source for      Source : Standards for Systems Engineering, Jerry Lake, 2002         (Version 1.1)
                         Reprinted courtesy of Jerry Lake                                                        12
                               University of Southern California
                           Center for Software Engineering



                  Breadth and Depth of Key SE Standards
                                                                               System life
                   Process
                  description                                                  ISO/IEC 15288
Level of detail




                  High level                        EIA/ANSI 632
                  practices


                                                                   IEEE 1220
                                                                                        Input to 632/1220

                   Detailed                                                                     Operate,
                   practices
                                                                               Transition to    Maintain,     Replace
                               Conceptualize                Develop             Operation      or Enhance   or Dismantle

         Purpose of the Standards:
                   ISO/IEC 15288 - Establish a common framework for describing the life cycle of
                      systems
                   EIA/ANSI 632 - Provide an integrated set of fundamental processes to aid a
                      developer in the engineering or re-engineering of a system
                   IEEE 1220 - Provide a standard for managing systems engineering
                                              Source : Draft Report ISO Study Group May 2, 2000
                                                                                                                           13
                                              Reprinted courtesy of ISO Study Group
     University of Southern California
    Center for Software Engineering




Effort Distribution Across EIA 632
     Fundamental Processes
                                         N = 18
        EIA 632 Fundamental
                                           Average   Standard Deviation
              Process
       Acquisition &
       Supply                               7%             3.5
       Technical
       Management                          17%             4.5
       System Design                       30%             6.1
       Product Realization                 15%             8.7
       Technical
       Evaluation                          31%             8.7




                                    Total = 100%                          14
                  University of Southern California
                 Center for Software Engineering




                Effort Profiling mini-Delphi
                                                                                          Operate,
                                                           Operational   Transition to                Replace or
                                                                                          Maintain,
                                 Conceptualize   Develop     Test &       Operation                   Dismantle
                                                                                         or Enhance
                                                            Evaluation


                                                            ISO/IEC 15288

Acquisition &
   Supply
                  EIA/ANSI 632




  Technical
 Management

   System
   Design

 Product
Realization

 Technical
 Evaluation



                                                                                                                   15
                  University of Southern California
                 Center for Software Engineering



              Effort Distribution of EIA 632 Fundamental
                 Processes Across ISO 15288 Phases
                                                       N = 15
                                                                                     Operate,    Replace or
                                                       Operational                               Dismantle
                                                                     Transition to   Maintain,
                 Conceptualize              Develop       Test
                                                                      Operation      or
                                                         & Eval.                                              (checks
                                                                                     Enhance                    um)

Acquisition
and Supply
                    28 (12.3)              51 (18.6)    13 (11.3)      8 (5.0)                                 100

Technical
Management
                    22 (10.0)               38 (9.9)    25 (7.4)       15 (6.4)                                100

System Design       34 (12.4)              40 (19.4)    17 (9.6)       9 (6.2)                                 100

Product
Realization
                    13 (14.1)              30 (24.3)   32 (16.0)      25 (20.4)                                100

Technical
Evaluation
                    18 (11.4)              27 (11.0)   40 (17.7)       15 (8.5)                                100


                               In each cell: Average (Standard Deviation)                                            16
       University of Southern California
      Center for Software Engineering




Challenges & Lessons Learned
Challenges
• Few companies follow standard WBS
   – Many tailored WBS based on domain or product
• Key was to map diverse set of WBS to EIA/ANSI
  632 activities
Lessons Learned
• Using standards provided a common baseline for
   – Defining Systems Engineering
   – Establishing scope of the model
• Data collection yielded typical activity profile for
  systems engineering
                                                         17
   University of Southern California
  Center for Software Engineering




Value of Systems Engineering:
Minimize Total Program Overrun




          Source: Werner Gruhl, NASA Comptroller’s Office
          Reprinted courtesy of NASA                        18
                                    University of Southern California
                                   Center for Software Engineering




   Value of Systems Engineering (cont.)
                        1.2
(Cost/Schedule Based)
 Development Quality




                        1.0




                        0.8
                                                                                               10.0

                                                                                                9.0
                        0.6

                                                                         Comparative Success
                                                                                                8.0

                                                                                                7.0

                        0.4                                                                     6.0
                              0%   2%     4%      6%      8%     10%    12%                     14%        16%   18%
                                                                                                5.0
                                                         SE Effort
                                                                                                4.0

                                                                                                3.0

                                                                                                2.0

                                                                                                1.0

                                                                                                0.0
                                                                                                      0%               5%        10%    15%

                                                                                                                            SE Effort
                                           Source: SECOE 01-03, INCOSE 2002, Eric Honour                                                      19
                                           Reprinted courtesy of INCOSE
          University of Southern California
         Center for Software Engineering




                                          Summary
• Insufficient systems engineering resources has been cited as
  one root cause for space system development problems in
  several recent studies
• COSYSMO has been developed to provide a model for
  estimating systems engineering resources parametrically from
  system level technical parameters and associated cost drivers
   – Model development included USC-CSE Corporate
     Affiliates, ISPA, SCEA, SSCAG, PSM, and INCOSE participation
• Depth and breadth of model anchored in foundation of
  systems engineering industry standards
   – WBS definition guided by EIA/ANSI 632
   – Life cycle definition guided by ISO/IEC 15288
• Current effort includes calibration of model with effort and
  technical parameter actuals
                                                                    20
            University of Southern California
           Center for Software Engineering



                                        References
Gruhl, W. NASA Comptroller’s Office, 1992.
Honour, E., Toward an Understanding of the Values of Systems Engineering, 2nd
     Conference on Systems Engineering Research, March 2004.
Lake, J., Standards for Systems Engineering, INCOSE Report, 2002.
Roedler, G., Draft Report ISO Study Group, May 2, 2000

Valerdi, R., Rieff, J., Roedler, G., Wheaton, M., Lessons Learned for Collecting
      Systems Engineering Data, 2nd Conference on Systems Engineering Research,
      April 2004, Los Angeles, CA.

Valerdi, R., Technical Size Via Requirements, ISPA Southern California Winter
      Workshop, NASA JPL, March 2004.

Valerdi, R., Ernstoff, M., Mohlman, P., Reifer, D., Stump, E., Systems Engineering
      Sizing in the Age of Acquisition Reform, 18th Forum on COCOMO and
      Software Cost Models, October 2003, Los Angeles, CA.

Valerdi, R., Boehm, B., Reifer, D., COSYSMO: A Constructive Systems Engineering
      Cost Model Coming Age, 13th INCOSE Symposium, July 2003, Crystal City, VA.

COSYSMO prototype v1.22 available at: http://www.valerdi.com/cosysmo                 21
     University of Southern California
    Center for Software Engineering




Marilee Wheaton
{Marilee.wheaton@aero.org}

Ricardo Valerdi
{rvalerdi@usc.edu}

COSYSMO Website:
 www.valerdi.com/cosysmo

                                         22

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:73
posted:7/8/2010
language:English
pages:22