Docstoc

NRO CAIG Society of Cost Estimating and Analysis

Document Sample
NRO CAIG Society of Cost Estimating and Analysis Powered By Docstoc
					 Non-Recurring Space Vehicle Hardware
Cost Estimating Relationship Development
         SCEA Annual Conference June 26-29, 2012

             Ryan Timm (Booz Allen Hamilton)
                  James Smirnoff (Wyle)
           Eileen DeVillier (Booz Allen Hamilton)
            Michelle Dye (Booz Allen Hamilton)
                  Mitch Robinson (Wyle)




                                                    1
                            About the NRO

     The National Reconnaissance Office (NRO) is the
     national program to meet the U.S. Government's
     intelligence needs through spaceborne
     reconnaissance

     A Department of Defense (DoD) agency and an
     element of the Intelligence Community

     Funded through the National Intelligence Program
     and the Military Intelligence Program portions of the
     federal budget

     The NRO’s existence was declassified by the Deputy
     Secretary of Defense on September 18, 1992
            NRO and DoD acquisition processes are closely aligned
NRO CAIG                                           2
                                          Overview
      How does the NRO CAIG estimate Space Vehicle Non-Recurring
      costs?
           Box-level estimates
           NR CER functional form
               Scale variables
               Complexity variables



      Why is Non-Recurring CER development more difficult than
      recurring CER development?

      What issues have we encountered developing NR CERs and how
      are we working to resolve them?
           Selecting an optimal scale variable
           Balancing data homogeneity and data quantity
           Segregating cost of NR engineering effort from cost of development units



NRO CAIG                                                                              3
                  Expenditures for Actual NRO Satellite Program


                                                                                          Nonrecurring Costs:
                                                                                          Requirements definition
                                                                      46%     Total NR
                                                                                          Engineering design & analysis
                                                         54%
                                                                              Total REC   Manufacturing tooling
                                                                                          Development units
                                                                                          Simulators
                                                                                          Development and acceptance test procedures
                                                                                          Redesign, rework & retest to correct design flaws

                                                                                          Recurring Costs:
                                                                                          Production unit parts & materials
                                                                                          Production unit fabrication, assembly & testing
                                                                                          Spare parts production units
                                                                                          Rework due to workmanship problems
       SV 1     PDR       CDR
                                                SV1                          SV2
       ATP                                     Launch                       Launch




         Non-Recurring costs are the vast majority of costs early in a program lifecycle
         The NRO budgets to the Independent Cost Estimate (ICE)*
         An accurate budget is important to program success

 *Budget is based on the ODNI Independent Cost Estimate (ICE) or NRO/CAIG Agency Cost Position (ACP)
               NR costs can be a significant portion of total SV acquisition costs
NRO CAIG                                                                                                                                    4
                              NRO CAIG Cost Estimates:
                              Detailed – Product/Box Level


                                         Bus and Payload   Satellite-Level
                                            SEIT/PM          SEIT/PM
               Box 1
                Box 1                                        Payload
                 Box 1
                  Box m
                    Box m                 Subsystem B
                 Box Level
                     Box m
                      Box n
                                           Subsystem B
                                            Subsystem B
                                              Subsystem          Bus
                                                                              Total Space-
                                             HW, SW, STE
                 Estimates                                                   Segment Cost




               Nonrecurring: Design/Develop/Qualify
               Recurring: Build/Integrate/Test



              CAIG
                              Summing
            Estimating
                              Elements
           Touch Points




    Process Begins by Defining and Estimating Equipment at the Product/Box Level

NRO CAIG                                                                                     5
                 Why Box-Level Parametric Estimates?
  Box Level
      Link to Work Breakdown Structure
      Support design trades
      Demonstrate detailed understanding of space vehicle
      Can “tune” the cost estimate to the technical baseline

  Parametric
      Used throughout the program lifecycle
           Pre-acquisition
           Proposal evaluation
           Milestone reviews
      Unbiased
      Repeatable
      Provides statistically quantifiable uncertainty
      Conducive to sensitivity and affordability analysis

       Provides the most utility to support acquisition and program execution

NRO CAIG                                                                        6
                      Current NRO CAIG Dataset
    Database of over 2000+ space hardware data points

    Data comes directly from contractors, multiple program types
           NRO, Commercial, NASA, Air Force

    Coordination with Space and Missile Systems Center and industry
           Unmanned Space Vehicle Cost Model (USCM) Data
               Standard WBS and Normalization is the same / model development is separate
           Industry collaboration via regularly occurring cost IPT’s

    Data behind each CER is available for easy reference
           For direct analogy estimating
           For calibrating CER up or down due to other criteria

                                                                        Unit Wt     AUC
                  CER             Program                Box                               % Error
                                                                         (lbs)    (FY00$K)
            Booms and Towers   Program A    XYZ Support Tower            105.0     2,780      59%
            Booms and Towers   Program B    Solar Array Boom Assembly    92.0       285      -83%
            Booms and Towers   Program C    PDQ Antenna Boom             57.0      1,056     -41%

NRO CAIG                                                                                             7
                         NRO CAIG Product/Box Level CER Inventory

    6 Non-Recurring CERs                                                                 78 Recurring CERs
     RF Equipment                                           Att. Control Elex (ACE)               Helix antenna                       Solid Rocket Motors
     Digital Equipment                                      Back-End RF Electronics               Dipole/Other antenna                Solid-State Transponders
     Antennas                                               Power Monitors                        Nutation Dampers                    Solid-State Transmitters
     Misc. Electronics
                                                            BAPTAs                                Comm Data Processing Electronics    Star Trackers
     Structures, Electromechanical Devices, & ACS Sensors
                                                            Li batteries                          SIG or EO Processing Electronics    Solar-Array Booms
     Optics
                                                            NiCd batteries                        Positioner assemblies               Other Deployable Structure
                                                            NiH batteries                         Positioner motors                   Secondary Structures
                                                            Booster Adapters                      DC power converters                 Trusses and Towers
                                                            Command Receivers                     AC power converters                 Equipment Compartments
                                                            GPS Digital                           Power & Coax Harnesses              Optical Payload structures
                                                            Comm Front-End RF Electronics         Propulsion Plumbing                 Analog sun sensors
                                                            Comm LNAs                             Pressurant Tanks                    Digital sun sensors
                                                            DC Power Harnesses                    Propellant Tanks                    Bus and RF Payload thermal H/W
                                                            Deployment Drives                     Pyro Driver Electronics             EO Payload Thermal H/W
                                                            Driver Control & Data Rounting Elex   RF Coax Harnesses                   Thermal Blankets
                                                            Earth Sensors                         Shunts, Dissipators and Capcitors   Thermal Heaters and Sensors
                                                            EPS Electronics                       SIGINT Feed Equipment Groups        Thermal Heat Pipes & Radiators
                                                            Flight Computers                      SIGINT Feeds                        Thermal Shields/Barriers/Louvers
                                                            IRUs                                  SIGINT Front End RF Electronics     Thrusters
                                                            Accelerometers                        SIGINT Preamplifiers                Oscillators
                                                            Large Deployable Reflectors           Small Parabolic Antennas            Timers/Clocks
                                                            Magnetic Torquers                     GaAs, deployable arrays             TT&C Digital Electronics
                                                            Magnetometers                         GaAs, not deployable arrays         TWTAs
                                                            Downlink MW Plumbing                  Silicon, deployable arrays          Waveguide Assemblies
                                                            TT&C MW Plumbing                      Silicon, not deployable arrays      Reaction Wheels
                                                            Horn antenna                          Solar Array Drives                  CMGs


               There are recurring CERs for most Equipment Groups
               There are far fewer NR CERs
NRO CAIG                                                                                                                                                        8
             Why are NR CER’s Harder to Develop?
      Less data available for NR CER development than REC CERs
           All units have recurring cost but not all units have significant NR cost
           Not all organizations collect data on NRO CAIG preferred cost drivers
           Data Issues
               Low confidence in NR / REC split
               Estimated cost of development unit hardware exceeds total NR



      Intuitively, there are more cost drivers in play
                    Heritage
                    Technology Readiness
                    Number of Development Units
                    Scale of the Project
                    (e.g., weight, production unit cost)
                    Complexity                                          Also Recurring-Cost Drivers
                    Production Quantity



    Functional form may be selected to enable more data points to be used
NRO CAIG                                                                                              9
                              NR CER Functional Form

   Typical CER form:                          $ NR  a Scaleb Complexity 1..n c

    Dependent variable: NR Engineering (NReng)... Does not include
    development hardware
               $NR Eng  $ NR  (T 1  DevUnits )
    Scale Variables                                               Complexity Variables
           Weight                                                       % New Design
           Theoretical First Production Unit Cost (T1)                  % Unique Design
           Average Unit Cost (AUC) of Production Units                  Quantity
                                                                        Less Common
                                                                             SV Design Life
                                                                             SV Mission Type
                                                                             Orbital Regime
                                                                             HW Specific
   *NReng separates development unit costs from NR engineering costs:
   Dev Units= Engineering Units/ 2 + Qualification Units
NRO CAIG
                              Scale Variables
    Purpose: Calibrate the cost estimate to the size of end item you are estimating
    Square footage would be an example of a scale variable for housing




    Common space hardware scale variables: Weight, Average Unit Cost (AUC),
    and Theoretical First Unit Cost (T1)
    Each scale variable has pro’s and con’s which we will discuss later in the brief


                        All things being equal, more costs more....
NRO CAIG                                                                          11
                                          Complexity Variables
         %New Design: the relative amount of design effort
         required to design unique components of a box
                100% : Completely new design
                0%: Existing design that requires no modification



         %Unique Design: a measure of how much design
         repetition exists within a unit

         Production Quantity
                Higher production quantity → higher development costs
                Phenomenon first uncovered in 2005 study of box-level costs*




*41st Annual DODCAS presentation: “Satellite-Subsystem Development Costs”, Burgess, E. and Menton, N., February 2008


NRO CAIG                                                                                                               12
           Issues encountered during NR CER Development

      Selecting an optimal scale variable

      Striking a balance between data homogeneity and data
      quantity for CER development

      Segregating cost of NR engineering effort from cost of
      development units




NRO CAIG                                                       13
                             Selecting a Scale Variable
    Weight – Due to the high cost and limited capacities of launch vehicles,
    weight is a good predictor of the scale of specific SV commodities. All else
    being equal, the larger / heavier box will require more NR effort
           (+) Weight can be predicted relatively accurately early in program life
           (-) Dissimilar commodities may not scale in the same manner


    Production Cost – Generally, we have found that the more expensive a box
    is to produce, the more NR effort dedicated to it
           AUC
                 (+) Dissimilar commodities may scale more uniformly with a cost based scale variable
                 than weight
                 (-) AUC can be predicted with existing REC CERs however uncertainty is relatively high
                 for many commodities
                 (-) Higher production quantities would have the effect of decreasing the scale variable
                 unless a quantity variable is included in the CER
           T1
                 (+) Higher production quantities have no effect on the scale variable
                 (+) Dissimilar commodities may scale more uniformly with a cost based scale variable
                 than weight
                 (-)The Cost Improvement Curve (CIC) must be estimated
                 (-) T1 can be predicted with existing REC CERs however uncertainty is relatively high for
NRO CAIG         many commodities                                                                         14
           Example: AUC vs. Weight Scale Variable
     Suppose we have two Candidate CER’s to estimate the cost of
     Electromechanical Devices:

     Model with AUC Scale Variable:                 Model with Weight Scale Variable:
    NR  a(AUC) b (Qty) c (%ND) d                   NR  a(Weight) b (Qty) c (%ND) d
     The models are similar and all things being equal, we might choose the
     AUC based model due to the lower Standard Percent Error (SPE):

                                  AUC                       Weight
                         NR= a(AUC)^b(Q)^c(%ND)^d   NR = a(Wt)^b(Q)^c(%ND)^d
                     a             1.53                       0.42
                     b             0.66                       0.72
                     c             0.90                       0.88
                     d             0.74                       0.60
                     n              57                         57
                   dof              53                         53
                  SPE              55%                        60%




    The AUC scale variable seems preferable but now factor in input uncertainty…
NRO CAIG                                                                                15
           Example: AUC vs. Weight Scale Variable
      Given the following inputs, we can apply                                  Independent Variable           Value
                                                                                      Weight (lbs)             20.0
      either CER to estimate NR cost:                                                 AUC ($M)                  3.5
                                                                                       Quantity                  2
                                                                                        %ND                    50%

      Assuming the inputs are deterministic, we                                    Deterministic Inputs
      get the following results:                                                AUC Model      Weight Model
                                                                  Mean ($M)           3.9                4.4
                                                                Std Deviation         2.2                2.6
                                                                          CV         55%                60%
      But consider that the inputs are estimates
      themselves and they are uncertain.*
                                                                                    Uncertain Inputs
      And assume that we know weight (+/15%)                                    AUC Model    Weight Model
                                               Mean ($M)                             3.9                 4.4
      with less uncertainty than we know AUC Std Deviation                           2.5                 2.7
      (+/-50%) when we estimate:                       CV                           64%                 61%




                  Now the weight scale variable seems comparable….
NRO CAIG                                                                                                               16
           *“Errors in Variables Regression for CER Development”, Covert, R, Spring 2008 SCEA Journal
                    Data Homogeneity vs. Quantity
      Where data is sufficient, NR box level CERs can be developed that
      correspond directly to the REC CER equipment groups

      Often there are too few NR data points to develop compelling CERs
      at the box level (~700 NR data points, 80 equipment groups)

      NRO CAIG uses two primary strategies to address issues of
      insufficient data:
           Grouping like items at a higher level to develop a more robust CER
           Using dummy variables to stratify a CER based on subgroups:


                 NR  a(AUC) b (Qty) c (%ND) d e drivesf wheels
                 Drives = 1 if box is a positioner, actuator or deployment drive
                 Wheels= 1 if box is a reaction wheel or control moment gyro
                 Dummy variables= 0 otherwise


NRO CAIG                                                                           17
                              Grouping Example

             Equipment groups                Data points
   All Electro Mechanical Devices                57

       Positioners                               26
       Deployment Drives                         14          Investigated grouping
       Actuators                                  2
       Magnetic Torquers                          1
       BAPTAs                                     2
       Control Moment Gyros                       5          Investigated grouping
       Reaction Wheels                            5
       Nutation Dampeners                         2




            Analyst must balance data homogeneity and quantity of data
NRO CAIG                                                                      18
                                                           Stratifying Example
      Models must have similar behavior over the range of expected
      values for both scale and complexity variables

                                         Good                                                                   Bad
                               Model Sensitivity to %ND                                           Model Sensitivity to Scale
                          4                                                                               Variable
                         3.5                                                                  7


                          3                                                                   6
       Estimated NR $M




                         2.5                                                                  5




                                                                            Estimated NR $M
                          2                                                                   4

                         1.5
                                                                                              3

                          1
                                                                                              2
                         0.5
                                                                                              1
                          0
                                                                                              0
                                                                                                  0.1 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 5.5
                                             %New Design                                                              AUC $M

                           All EM   Drives & Positioners   Wheels & Gyros                         All EM   Drives & Positioners   Wheels & Gyros



                                          NR  a(AUC) b (Qty) c (%ND) d e drivesf wheels
NRO CAIG                                                                                                                                           19
             Segregating cost of NR engineering from Dev Units

      We want our cost estimates to be sensitive to the planned number of
      Development Units

      To do this, we assume the cost of each Development Unit is equal
      to the cost of a Theoretical First Unit (T1) and subtract this cost from
      the total NR Cost:
                  $NR Eng  Historical $ NR  (DevUnit  T1)
      Then, we develop a CER for NReng:

                  Estimated $ NR Eng  a Scaleb Complexity 1..n c
      Finally, we add back the Development Unit cost to arrive at the total
      estimated NR cost:
               $NR  Estimated $ NR Eng  Assumed DevUnit Cost

    Problem: For some data points, the remaining NReng cost is less than zero!
NRO CAIG                                                                     20
                 Segregating cost of NR engineering from Dev Units

     Development Units consist of Engineering (E) and Traditional Qual Units (TQ):
           Engineering Units (E):
                Units built to conform to basic design elements of a box during its development.
                May not be built exactly to flight configuration or include flight quality parts
                Assumed to cost 0.5∙T1*
           Traditional Qual Units (TQ):
                Design articles built to flight configuration standards and subjected to qual testing.
                These may later be refurbished and used as flight hardware
                Assumed to be equal to the cost of a T1 *


     Potential Solution:
     Estimate the cost of development units directly by regression

$ NR  a(Scale Variable ) b (Complexity Variable 1...n ) c (Qty ) d  e( E  T 1)  f (TQ  T 1)


If our assumptions are correct, the coefficient of the E and TQ terms will be 0.5 and 1

NRO CAIG    *Assumed values of E= 0.5∙T1 and TQ=T1 are longstanding assumptions that trace back to PRICE-H
                                                                                                             21
              Potential Solution: Estimate Dev Units Directly by Regression


      Utilizing RF Equipment data points as a test case, we
      directly estimated the development unit costs as a
      fraction of T1:

      $ NRRF  a ( ScaleVariable) b (ComplexityVariable1...n ) c (Qty ) d  e( E  T 1)  f (TQ  T 1)


        Model           Constant         %ND       %UD        QTY       AUC E∙T1           TQ∙T1

        E and TQ            2.00         0.47       0.06      0.51      0.83     0.44       0.53


      Results inconclusive
           Coefficient of Engineering Unit (E) term close to expectations (0.5)
           Coefficient of Traditional Qual Unit (TQ) term significantly less than expectations (1.0)

                    Further study needed to understand development unit costs

NRO CAIG                                                                                               22
                Next steps to improve CAIG NR CERs
     Issue: Selecting an Optimal Scale Variable
           Continue to experiment with different scale variables

     Issue: Balancing Data Homogeneity and Data Quantity
           %ND is not available for many of our data points
             Investigate reclaiming lost data points by redefining %ND as a more
             simplified metric (Existing Design, Minor modification, Major Modification,
             New Design)
           %UD is not available for many of our data points
             Identify equipment subgroups where it is most beneficial to have %UD
             Eliminate %UD complexity variable from equipment group CERs in which
             the additional data points are more valuable than the descriptive power of
             the %UD complexity variable

     Issue: Segregating cost of NR engineering effort from cost of
     development units
           Continue to explore regression analysis solutions
           Revisit and explore historical data

NRO CAIG                                                                                   23
                        Questions?
  Ryan Timm
  NRO CAIG Support
  Booz Allen Hamilton
  Timm_Ryan@bah.com
  (703) 633-2151

  James Smirnoff
  NRO CAIG Support
  Wyle
  James.Smirnoff@Wyle.com
  (703) 633-2110

NRO CAIG                             24
25
             NRO Cost Analysis Improvement Group (CAIG)

      At the program level, the CAIG provides objective pre-investment
      decision analysis and in-process program management
      decision support capabilities
           Independent cost and schedule analyses for all major program milestones

           Supports annual NRO budget build
           Supports source selections and proposal assessments for competitive awards and should
           cost analysis of sole-source procurements
           Develops “Best in Class” processes and models
                                   Program-Level Execution



                                   1                 2                     3                      4                       5                         6                       7
                                                                                                                                                          Program
                                                                                                                               Baseline                                           Milestone
                                           ICE             Budget               Source                   Contract                                       Performance
                                                                                                                              Established                                       Reviews (PDR
                                       Development       Established           Selection                Negotiation                                     Assessment
                                                                                                                                 (IBR)                                             / CDRs)
                                                                                                                                                           (PMB)
                                   0
                                                                           CAIG Core Infrastructure: People, Processes, Tools, Technology, Data



                    Pre-Acquisition                                                                   Acquisition / Execution                                                             Operations

                                                                                                                                                               (Fabrication, Assembly,
       Concept Studies
       Concept Studies          Concept Development
                                Concept Development                    Preliminary Design
                                                                       Preliminary Design                Complete Design
                                                                                                         Complete Design                Manufacturing
                                                                                                                                        Manufacturing          Integration, Testing)       Operations
                                                                                                                                                                                           Operations


                      ATP                                   ATP IBR                               PDR                         CDR                                                        IOC     FOC
                                                                                                                                                  Space System Lifecycle
                    (Phase A)                            (Phase B)



NRO CAIG                                                                                                                                  26
                  NRO CAIG Estimating Process
    (Contractor’s) Cost &                                                                     Schedule
    Tech Data Sheets                                                                          Assessment
       Space Hardware                      SPACE ESTIMATE                          Risk
       Data Sheets
                            Analyze        Parametric, analogy, bottom-up          Analysis
       Space Software       Model                                                  Monte        Develop
       Data Sheets          Inputs                                                 Carlo,       Budget
                                           GROUND ESTIMATE                         Discrete     Spread
      Ground Hardware                      BOM analysis, parametric, analogy       Risks
      Data Sheets

       Ground Software         • By Work Breakdown Structure (WBS) element
                               • Consistency                                                  Cross-check
       Data Sheets
                               • Software complexity, software lines of code (SLOC)
         Operations &          • Space hardware complexity, heritage, weight, etc.
         Maintenance           • Ground hardware complexity, quantities, capability, etc.

     Other Program Info:                                                                       Document
     • Proposals                                                                                   &
     • CDRLs                Discussions      Program
                                                              NCG              Independent    Brief Results
                            with Program     Office
     • CDR packages                                           Technical        Technical
                            Office &         Technical
     • RFIs                                                   Assessment       Assessment
                            Industry         Assessment
     • Briefing charts

     • etc., etc., etc.
                                               Government Assessment




NRO CAIG                                                                                                      27
                    NRO CAIG Estimating Process
(Contractor’s) Cost &                                                                     Schedule
Tech Data Sheets                                                                          Assessment
   Space Hardware                      SPACE ESTIMATE
   Data Sheets                                                                 Risk
                        Analyze        Parametric, analogy, bottom-up          Analysis
   Space Software       Model                                                  Monte          Develop
   Data Sheets          Inputs                                                 Carlo,         Budget
                                       GROUND ESTIMATE                         Discrete       Spread
  Ground Hardware                      BOM analysis, parametric, analogy       Risks
  Data Sheets

   Ground Software         • By Work Breakdown Structure (WBS) element
                           • Consistency                                                   Cross-check
   Data Sheets
                           • Software complexity, software lines of code (SLOC)
     Operations &          • Space hardware complexity, heritage, weight, etc.
     Maintenance           • Ground hardware complexity, quantities, capability, etc.
                                                                                          Document
Other Program Info:                                                                       &
• Proposals
                                                                                          Brief Results
• CDRLs                 Discussions      Program
                                                          NCG              Independent
                        with Program     Office
• CDR packages                                            Technical        Technical
                        Office &         Technical
• RFIs                                                    Assessment       Assessment
                        Industry         Assessment
• Briefing charts

• etc., etc., etc.
                                           Government Assessment
NRO CAIG                                                                                          28
                NRO CAIG CER Management Process

                                                                                    CER
  •Existing CER Data
  •Latest NRO Data                  Build                                         Database
  •USCM Data                       Dataset
  •Other Data
                       •Costs
                       •Number of units
                       •Weights                                          DRAFT CER
                       •Tech descriptions
                       •Reconcile Multiple
                                                           40+ Hours
                        sources
                                              Review
                                             with CER    Document
                                               Lead


                                                                                 APPROVED CER


                                                        CER Working
 NRO CAIG CER Development is:                           Group Review
 • Continuous
 • Time Intensive                                       •Peer Review Process
                                                               •Review Dataset
 • A Peer Review Process                                       •Review Candidate CER’s
                                                               •Assign Future Actions


NRO CAIG                                                                                    29
           Evolution of NRO CAIG NR CER Modeling Efforts

                  1990s         2005                2008                         2012

 Models
                              •RF Equipment
                              •Digital Equipment
                •One model
                              •Antennas                            •Align to REC CER categories
                for all box
                              •Misc. Electronics                    where feasible
                types
                •4 Strata     •Structures, Electromechanical
                               Devices, & ACS Sensors
                              •Optics


 Data Points
                    146           577                  577                         701


 Updates                                           •Fixed Data     •Continued to fix data errors
                                                      Errors       •Addition of new data points
                              •Data scrub
                                                •Updates for NRO   •Increase homogeneity in
                                                     Inflation      categories




               2012 NR CER work incorporates updated and new data and
                     explores new NR CER forms and categories
NRO CAIG                                                                                           30
                  CER Functional Forms: Recurring

   Dependent variable: Average Unit Cost (AUC)

                                         total recurring cost
              AUC 
                      # of protoqual units + flight units + refurb units + spares

   Independent variables:
         Unit weight (in almost every CER we have finalized)
         Quantity (boxes per contract)
         Technical variables (frequency, power, etc.)

   Typical functional form (parameters a, b, c, and d estimated by regression)


                            AUC  a  wt b  q c  TechVariabled
                                                                             These are equivalent
                        AUC / wt  a  wt b1  q c  TechVariabled


           Other functional forms always attempted, sometimes chosen.
                This form has proven most reliable/robust to date.
NRO CAIG                                                                                            31
                         Percent New Design (%ND)
      Percent New Design is the relative amount of design effort required
      to design unique components of a box
           100% : Completely new design
           0%: Existing design that requires no modification


      %ND is an important element of nonrecurring hardware cost
      estimation
           It is significant in the CAIG’s box-level NR CERs; and is used by development contractors
           Can be rolled up to the spacecraft level to summarize the overall relative amount of new
           development


      Ambiguities remain in the definition and application of percent new
      design:
           Contractors using “gut-feel” vice consistent application of the definition
           Inconsistencies across different organizations or contractors
           Each contractor has a different tool for calculating %ND



NRO CAIG                                                                                               32
                     Percent Unique Design(%UD)
      %UD is a measure of how much design repetition exists within a unit
      In determining %UD, the contribution of each sub-element, or portion, of the unit
      should be weighted by some measure of its relative contribution to the overall unit
      design and development effort such as:
           Production cost
           Weight
           Internal part/module count
           Drawing count
      %UD is calculated separately for both the mechanical and electrical characteristics of
      the assembly or box
      For example if an electrical box has an identical A and B side for simple redundancy
      and a backplane and motherboard weighted at 1.5 circuit boards

               Unique Boards   1  1.5         2.5
     %UDELEC                          100       71%
                Total Boards   2  1.5         3.5


NRO CAIG                                                                                   33
                                                     NR Cost is
                                    Correlated with Recurring Production Quantity

             Higher production quantity → higher development costs
                     Phenomenon first uncovered in 2005 study of box-level costs*
                     Subject of on-going discussions/investigations with industry
                     Investigation led to theories about why this relationship exists…

                                             Boxes That Are Rated as 100% New
                    Average NR/T1




                                     n=142            n=65                n=33                 n=22                    n=6


                                      1-2              3-4                  5-6                7-14                    15+
                                                    Total Production Quantity
NRO CAIG
*41st Annual DODCAS presentation: “Satellite-Subsystem Development Costs”, Burgess, E. and Menton, N., February 2008
                                                                                                                             34
                                 NR Cost is
                Correlated with Recurring Production Quantity
           There are real underlying Reasons for this:
              Design for producibility
              Investments (test equipment, tooling, training) to improve
              producibility
              More GSE needed to handle quantities in parallel
              Every unit is different—requires some NRE each time

           Quantity is a surrogate for other factors
              Schedule stretch-outs encourage innovation on later units
              Obsolescence over long schedule/multiple satellites
              Performance of first unit on orbit motivates changes to subsequent
              units on same contract

           Cost accounting/normalization takes some of the blame
              NR/REC segregation issues
                  CDR-split method
                  Setup costs booked as NR
                  Protoflight engineering booked as NR
NRO CAIG                                                                           35
                                           UNCLASSIFIED



                       Determining NR / REC Split
      Ideally the contractor and all subcontractors have properly
      segregated NR / REC cost

      In many cases the analyst must work with the contractor to correct
      or determine the split:
           Use contractor internal data
           Examine definitions of specific accounts and map / allocate effort
           Use time phased data and program milestones (e.g. CDR)
           Request subcontract details


      Data points in which the NR / REC costs cannot be confidently
      segregated cannot be used for either REC or NR CERs




NRO CAIG                                                                        36
            Typical Problems With Normalized Cost Data

  •   Contractor used a different WBS
  •   Provenance of data sometimes unknown
  •   Data collected before end of contract – EAC typically grows at end
  •   Number of end-items produced is not known (spares, engineering units,
      refurbishments)
  •   Costs not accumulated by “tail number”
  •   Technical scope and cost scope are misaligned. For example…
                          Solar Array?                  Solar array technical documentation and mass
                                                        properties may include all panels, cells,
                                                        substrates, hinges, and drive positioner.
                                         Part of
                                         Solar Array?
                                                        Cost data often includes only panels, only
                                                        cells, etc. Other items booked to structures or
                                                        mechanisms.

                                                        Every program and contractor may be
                                                        different.

           Solar Array?



            Obtaining High Quality Data for CER Development is Difficult
NRO CAIG                                                                                          37
                           Performance Alone is not a Good Predictor of Cost


 Example: Estimating the Cost of a television
    Performance (screen size) is a cost driver at the store today
    Used by itself as a price forecasting tool, it fails:
                 $2673                                                                      $2599
                 In today’s dollars                                                         Today

               1960 RCA 21” Color TV                       Same money,
                                                      Better performance




      60 lbs

                                  $127/inch
                                                                                                        50” 1080p plasma
                                                                $52/inch                            Internet/streaming ready
                $765
                Today
                 Samsung 23”                                               60 lbs + stand
                                      $33/inch

                                       Same performance class
                                       Much less money                                      NOTE:         Cost/lb
                                                                                            RCA:          $42.50
     18 lbs                                                                                 Panasonic:    $42.62
                                                                                            Samsung:      $44.55

NRO CAIG                                                                                                                       38
                                     Both Performance and Scale/Size Variables are needed
                                               Multivariate Example: Traveling Wave Tube Amplifiers (TWTAs)
                                                        Actual unit cost data from NRO, NASA , and AF programs

                                 R² = 0.411


                                                                         Neither
                                                                        weight nor
      Unit Cost




                                                                                                 Unit Cost
                                                                        power are                               R² = 0.370
                                                                        very good
                                                                        predictors
                                                                          alone




                                               Weight                                                                         Power

                                                                                                TWTA unit cost = 72.6 x (WT)^0.632 × (Q)^-0.293 × (Power)^0.38
                                                                           Using the
   Error in Weight-Based Model




                                   R² = 0.56                                                                 R² = 0.787
                                                                         weight-based
                                                                         model, errors
                                                                         are associated

                                                                                          Actual Unit Cost
                                                                           with higher
                                                                         power TWTAs                                                            Model using
                                                                                                                                               weight & power
                                                                                                                                               performs much
                                                                                                                                                    better




                                               Power                                                              Estimated Cost (weight & power)
NRO CAIG
                                        And Since the CER’s are Multivariate,
                                    Heavier doesn’t always = Higher Estimated Cost
TWTA unit cost = 72.6 x (WT)^0.632 × (Q)^-0.293 × (Power)^0.38
TWTA dev cost = unit cost x (%new)^0.4
Assumes no engineering units or spares                                  #1: All-new 40W, 10lb
                                                                        TWTA is $5056

                         6000                                                                     #2: Existing but
                                                                                                  heavier 40W, 20lb
                         5000                                                                     TWTA is $3404
        TOTAL COST($K)




                         4000

                         3000

                         2000

                         1000

                            0
                                0           5            10            15           20            25
                                                       WEIGHT
                         • Cheaper to use heritage hardware (#2) for single-unit buy.
                         • New design might pay off for higher quantities (not usually realized for NRO systems).
  NRO CAIG
           Many Criteria for choosing the “best” CER

      Cost Relationships consistent with engineering expectations

      Quality and performance metrics
           SPE (lower is better)
           R^2 (higher is better)
           Bias (lower is better, typically driven to zero)

      Other factors to consider
           Degrees of freedom (more is better)
           Quality of sample data
           Applicability and Ease of Use
           Sensitivity to influential data points
           Balanced performance vs. time and vs. scale variable




NRO CAIG                                                            41

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:22
posted:10/24/2012
language:English
pages:41