2005 NASA Cost Analysis Symposium Technology Estimating ...

2005 NASA Cost Analysis Symposium Technology Estimating: Organizing Chaos Bob Sefcik Glenn Research Center April 13, 2005 Glenn Research Center at Lewis Field What is CHAOS? Cleveland Having Another Off-season Snowstorm April 3, 2005 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 2 Agenda • • • • • Problem Approach NASA Technology Inventory Database GRC Technology Test Resource Estimating Next Steps 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 3 Problem • Asked to do independent cost estimate for Vehicle Systems Program • No current GRC database for estimating technology development projects • Data to populate such a database is spread out amongst many people or in long term storage or missing/discarded due to personnel turnover or …. • No known method to estimate the cost of TRL advancement that is supported by actual data 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 4 Approach • Break technology work down to the major task level – Hardware and Software Task Resources • NASA Technology Inventory Database (NTIDB) – Documents Center technology tasks since 1998 – Tracks planned resources and TRL advancement – Provides previous years and future year planned activities – Not actuals! Very difficult to collect actuals on each task after the fact • Center data if available – Functional Task Resources • Used to break down tasks captured at a higher level in the NTIDB • Center data if available 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 5 NASA Technology Inventory Database Content Task Name: Aerospace Flywheel Technology Task ID 5832 Center GRC Sponsor AST UPN 755 Objective & Description: Flywheel energy systems provide storage of energy and momentum using high speed rotating masses. They offer the potential…. Benefit: Flywheels promise order of magnitude increases in performance… Technology Metrics: Watts/kg… 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 6 NASA Technology Inventory Database Content Task Name: Aerospace Flywheel Technology Milestones TRL 4 4 5 5 6 Year 2004 2004 2005 2005 2007 Description Complete design of advanced technology flywheel module Test counter-rotating flywheel modules on air-bearing table, full scale Conduct multi-degree-of freedom flywheel system demonstration, breadboard level Conduct characterization tests of advanced technology flywheel module TRL 6/7 estimate 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 7 NASA Technology Inventory Database Content Task Name: Aerospace Flywheel Technology Resources Profile FY NASA Profile in K$ Civil Service FTE 1999 2000 2001 2002 2003 2004 2005 2006 2007 1260 0.0 990 0.0 1250 9.0 1200 8.5 1300 9.0 2533 8.5 1900 6.5 1900 6.5 1900 6.5 Distribution of NASA Funding Contracts With Whom Grants With Whom NASA Field Center Other (Space Act, etc.) QSS SSC, Aerospace Corp., small vendors University of Toledo, Penn State U., Auburn U. GRC FY04 k$ 318 329 1886 Partnerships (included within Distribution) NASA Contributions Other Contributions 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field GRC AFRL 50 296 8 4/13/05 Subsystem Technology Example • Power Subsystem – Power subsystem modeling – Power subsystem testing – Power Generation - Photovoltaics • • • • Solar cell development Lightweight array technology Inflatable structures Array deployment – Energy Storage • Batteries • Flywheels – Power Management and Distribution • • • • High-temperature electronics High-rad parts Packaging Component technology – power control unit 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 9 Component/Single Task Estimating (Using the NASA Technology Inventory Database) • Power Subsystem – Power Generation - Photovoltaics • Lightweight array technology TASK ID 1448 $K Plan TRL FTE Plan $K Plan Advanced Thin Film Solar Cell and Ultralight Array TRL Technology FTE Plan $K Plan Advanced High Efficiency Solar Cell and Array TRL Technology (Power FTE Plan Generation) $K Plan Advanced Thin Film Solar Cell and Ultra-Lightweight TRL Array Technology (Power FTE Plan Generation) $K Plan Adv, High Efficiency Solar Cell & Array Technology TRL FTE Plan Adv Thin Film Solar Cell and $K Plan Ultra-Light Array Tech TRL FTE Plan $K Plan 40% Efficient Solar Cell for Concentrator Arrays TRL FTE Plan $K Plan Ultra-light, Ultra-highefficiency Solar Concentrator TRL Arrays for Space Power FTE Plan Task Name Advanced, High Efficiency Solar Cell and Array Technology 1998 $750 3 10 $400 2 6.6 1999 2000 2001 2002 2003 2004 1450 2250 2251 $863 2 11.2 $775 2 7.4 $720 2 0 $690 2 0 $535 n/a 4.7 $750 n/a 1 $502 n/a 0.1 $687 n/a 1.9 $500 3 5.1 $500 1 1 $472 2 0.2 $883 4 0.2 $500 4 3.3 $544 1 1.8 $433 2 0.1 $124 5 0.3 5826 5827 $515 2 2 6893 6895 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 10 Assembly/Multi-Task Estimating (Using the NASA Technology Inventory Database) • Power Subsystem – Energy Storage • Flywheels – component technology, rim technology, bearing technology, increased reliability, use studies TASK ID 1916 Task Name $K Plan TRL FTE Plan $K Plan ACESE Flywheel Energy Storage Demonstration on TRL ISS FTE Plan $K Plan Attitude Control for Satellite Flywheel Energy Storage TRL Systems FTE Plan $K Plan Aerospace Flywheel Technology TRL FTE Plan $K Plan Flywheel Energy Storage System for Small- and TRL Medium-Size Spacecraft FTE Plan $K Plan Integrated Composite Arbor and Flywheel Rim TRL Technology Development FTE Plan $K Plan New concepts in low cost, higher reliability and less TRL complex flywheel systems FTE Plan $K Plan Demonstration of Coordinated Momentum and TRL Energy Transfer (COMET) FTE Plan using Flywheels $K Plan Low Power Loss, Fail Safe Magnetic Suspensions for TRL Energy Storage and Attitude FTE Plan Control Flywheels Hubless Flywheel with Null-E $K Plan Magnetic Bearings TRL FTE Plan Self-Bearing Motor-Generator $K Plan for Flywheels TRL FTE Plan Aerospace Flywheel Technology 1998 $1,265 4 9.5 $4,000 4 15 $7,800 4 32 $70 2 0 $990 2 0 1999 2000 2001 2002 2003 2004 3996 5081 5832 6508 6930 7196 6910 6914 $1,250 n/a 9 $70 n/a 0 $421 n/a 0 $60 n/a 0.4 $699 n/a 0 $394 n/a 0 $1,200 4 8.5 $1,300 4 9 $2,533 4 8.5 $222 2 0.1 $50 3 0 $677 3 0.2 $320 3 0.2 $460 3 0.1 $1,069 3 1.2 $210 4 0.5 10096 10226 $12 4 0 $100 2 0 $70 2 0 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 11 Functional Task Estimating • Facility Testing – GRC facility testing is documented in Project Summary Reports • Reports for multiple projects available for certain facilities (9X15, 10X10, 8X6, & 1X1) • Description of test installation available to determine complexity • Test statistics – planned/actual # of runs and run time, facility occupancy time, schedules • Resources – Labor hours by org., power usage, some purchases • Question: Can the data be parametrically modeled? 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 12 Measuring Complexity of the Test Installation • Assigned a complexity weight to each facility system – Sum of all facility system weights equals 1.00 or 100% without considering test complexity – Facility system weights provided by facilities personnel • Calculated the test setup complexity – Multiplied the system weights times a complexity factor associated with the planned use of the facility system for the current test as compared to past tests: 0-Not Used, 1-Less Complex, 2-Average, 3-More Complex – Historical project complexity factors determined by facilities personnel 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 13 Document Test Complexity (9X15 Example) A llis o n U ltra -H ig h CM PLX W e ig h t F a c ility S u p p o rt S ys te m s M o d e l S u p p o rt S ys te m s M o d e l P re p a ra tio n B u ild in g M o d e l In te g ra te d S u p p o rt S ys te m G ro u n d P la n e E x h a u s t S u p p o rt S ys te m T u rn ta b le P e d e s ta l M o u n t S ys te m s In le t P re s s u re B le e d A ir S ys te m In le t V a c u u m B le e d A ir S ys te m U H B D rive R ig R o ta tin g R a k e S ys te m J e t E x it R ig A ir P re s s u re S ys te m s H ig h P re s s u re A ir S e rvic e A ir C o m b u s tio n A ir H yd ra u lic S ys te m N itro g e n S ys te m G a s e o u s H yd ro g e n S ys te m In fra re d S ys te m L a s e r-D o p p le r V e lo c im e te r (L D V ) S ys te m L a s e r S h e e t (F lo w V is u a liza tio n ) S ys te m F o rc e B a la n c e S ys te m A ltitu d e E x h a u s t S ys te m P h o to g ra p h ic S ys te m N e w o r E x te rn a lly S u p p lie d S ys te m s In s tru m e n ta tio n T h e rm o c o u p le s A c tu a to rs & P o s itio n In d ic a to rs M ic ro p h o n e s D a ta A c q u is tio n & P ro c e s s in g E S P S ys te m E s c o rt D C e n tra l A n a lo g R e c o rd e r T R A D A R -3 T ra n s ie n t D a ta A c q u is itio n S ys te m F a c ility T a p e R e c o rd e rs W e ig h te d T e s t S c o re GE F a n /O G V B yp a s s R a tio F a n GE UPS Fan H S R N o z z le P&W E je c to r N o z z le JSF S u b s o n ic In le t STOVL 2 D In le t JSF Low S p e e d In le t 0 .0 2 0 .0 8 0 .0 8 0 .0 3 0 .0 3 0 .0 3 0 .0 3 0 .0 7 0 .0 3 0 .0 4 0 .0 2 0 .0 1 0 .0 3 0 .0 2 0 .0 2 0 .0 8 0 .0 2 0 .0 2 0 .0 2 0 .0 4 0 .0 3 0 .0 1 0 .0 5 0 .0 2 0 .0 1 0 .0 2 0 .0 3 0 .0 3 0 .0 2 0 .0 2 0 .0 2 0 .0 2 1 .0 0 1 0 0 0 2 0 0 3 3 0 0 0 2 2 0 0 0 2 0 2 0 1 1 2 1 2 2 3 0 0 2 2 0 .9 8 1 0 0 0 2 0 0 3 3 0 0 0 2 2 0 0 0 2 0 2 0 1 0 2 1 2 2 3 0 0 2 2 0 .9 3 1 0 0 0 3 0 0 3 0 0 0 0 2 2 0 0 0 0 0 0 0 1 3 2 1 2 2 3 0 0 2 2 0 .9 0 1 0 0 0 0 0 0 0 0 2 0 0 2 1 0 3 2 0 2 3 0 1 3 2 2 2 2 3 0 0 2 2 1 .1 1 1 0 0 0 0 0 0 0 0 2 0 0 2 1 0 0 0 0 0 0 0 1 2 2 1 2 2 2 0 0 0 0 0 .5 0 1 0 0 2 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 2 1 1 0 2 2 0 0 2 2 0 .5 3 1 0 0 0 2 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 2 1 1 0 2 2 0 0 0 0 0 .4 2 1 0 0 2 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 1 1 1 1 0 2 2 0 0 2 2 0 .4 8 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 14 Collect Historical Test Data (9X15 Example) A llis o n U ltra -H ig h GE F a n /O G V R e s o u rc e R e q u ire m e n ts T o ta l L a b o r H o u rs C ivil S e rva n t L a b o r H o u rs P ro je c t O ffic e (P ) R e s e a rc h C u s to m e r (R ) C o m p u tin g S c ie n c e D ivis io n (V C ) T e s t E n g in e e rin g (D R ) T e s t T e c h n ic ia n (D R ) E n g in e e rin g (D D /D S ) F a b ric a tio n (D D M /P ) C e n tra l T e s t O p e ra tio n s (D F ) P B C L a b o r H o u rs P ro je c t O ffic e (P ) R e s e a rc h C u s to m e r (R ) C o m p u tin g S c ie n c e D ivis io n (V C ) T e s t E n g in e e rin g (D R ) T e s t T e c h n ic ia n (D R ) E n g in e e rin g (D D /D S ) F a b ric a tio n (D D M /P ) C e n tra l T e s t O p e ra tio n s (D F ) P u rc h a s e s (K $ ) M o d e l H a rd w a re M o d e l D e s ig n /D ra ftin g In s tru m e n ta tio n F a c ility H a rd w a re O th e r F is c a l Y e a r D o lla rs 55346 49960 0 3745 596 4418 15435 4844 9640 11282 5386 0 0 0 3112 2274 0 0 0 $ 1 ,0 4 0 .2 $ 6 9 1 .2 $ 0 .0 $ 1 4 0 .2 $ 9 8 .6 $ 1 1 0 .2 1999 B yp a s s R a tio F a n 27936 27936 0 2736 540 5958 12420 882 3240 2160 0 0 0 0 0 0 0 0 0 $ 3 ,4 9 0 .0 $ 3 ,3 0 0 .0 $ 0 .0 $ 0 .0 $ 1 2 0 .0 $ 7 0 .0 1996 GE UPS Fan 30476 26804 0 5958 0 5868 14508 0 470 0 3672 0 0 0 0 3672 0 0 0 $ 3 0 0 .0 $ 0 .0 $ 0 .0 $ 7 4 .0 $ 1 9 0 .0 $ 3 6 .0 1994 H S R N o z z le 24357 24357 0 4346 898 5216 9586 1192 1329 1790 0 0 0 0 0 0 0 0 0 $ 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 1991 P&W E je c to r N o z z le 13076 13076 7 5816 314 2348 3161 400 237 793 0 0 0 0 0 0 0 0 0 $ 5 0 .1 $ 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 5 0 .1 1989 JSF S u b s o n ic In le t 5205 3199 22 300 81 1034 1762 0 0 0 2006 0 0 0 1056 950 0 0 0 $ 1 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 1 0 .0 2003 STOVL 2 D In le t 15553 15553 0 3521 506 2555 5391 148 1425 2007 0 0 0 0 0 0 0 0 0 $ 8 0 .4 $ 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 8 0 .4 1988 JSF Low S p e e d In le t 9 2 1 8 .7 6 9 2 4 .5 0 326 1 1 6 .5 2006 2932 397 391 756 2 2 9 4 .2 0 0 0 9 6 0 .5 7 6 1 .7 0 0 572 $ 2 7 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 0 .0 $ 2 7 .0 1997 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 15 Collect Historical Test Data (9X15 Example) GE Fan/OGV Schedule Summary Planned Work Plan Completed Model Installation Started Testing Started Activity Complete Total Duration (months) Actual Work Plan Completed Model Installation Started Testing Started Activity Complete Total Duration (months) Facility Utilization Facility Occupation Time (months) Scheduled Test Runs Test Hours Actual Test Runs Test Hours Test Hour Variance Consumables Mach Range Test Power Usage (MW) Peak Off-Peak Average Power/Test Hour (MWH) Fuel Consumption Total Labor Breakdown % Engineering % Technician Allison Ultra-High Bypass Ratio Fan GE UPS Fan HSR Nozzle P&W Ejector Nozzle JSF Subsonic Inlet STOVL 2D Inlet JSF Low Speed Inlet Feb-99 Mar-99 May-99 Mar-00 13 Mar-99 Jun-99 Jul-99 May-00 14 11.0 Apr-95 Nov-95 Dec-95 Jun-96 14 May-95 Mar-96 Apr-96 Sep-96 16 6.0 58 572 Jun-93 Feb-94 Apr-94 Aug-94 14 Jul-93 Feb-94 Apr-94 Aug-94 13 6.0 53 Jul-90 May-91 Jun-91 Dec-91 17 Jul-90 Jun-91 Aug-91 Feb-92 19 6.0 Jul-89 0 Oct-02 Nov-02 Dec-02 Jan-03 3 Oct-02 Nov-02 Dec-02 Jan-03 3 2.0 20 191 17 93 (98) 0-.2 2824 58% 42% 30.37 0 48% 52% Oct-87 Feb-89 May-89 19 Oct-87 Feb-89 May-89 19 4.0 38 210 24 68 (142) 0-.2 2647 50% 50% 38.93 0 43% 57% Mar-97 Jul-97 Aug-97 Aug-97 5 Apr-97 Jul-97 Aug-97 Aug-97 4 1.0 16 217 16 124 (93) 0-.2 4100 50% 50% 33.06 0 41% 59% Jul-89 0 3.0 38 227 19 75 (152) 192 715 715 0-.1 16294 50% 50% 22.79 0 30% 70% 42 387 (185) 0-.1 4852 50% 50% 12.54 0 36% 64% 34 173 173 0-.2 5500 50% 50% 31.79 0 39% 61% 0 .1-.2 4514 50% 50% #DIV/0! 0 48% 52% 2521 50% 50% 33.61 0 68% 32% 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 16 Using the Available Data • Identified desired dependent variables – Labor Hours – Megawatt Hours – Facility Occupancy • Identified independent variables that provide the best predictive results within the range of the data in the database 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 17 Independent Variables Labor Hour Prediction Facility Variable CMPLX Factor Occupancy Time Test Runs Test Hours P P P P P 9 X 15 10 X 10 8X6 P P P P 1X1 Megawatt Hour Prediction CMPLX Factor Occupancy Time Test Runs Test Hours P P P P P P P Occupancy Month Prediction CMPLX Factor Test Runs Test Hours 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field P P P P P P P P 4/13/05 18 Test Facility Survey • Request selected test planning data 9 X 1 5 L o w S p e e d W in d T u n n e l T e s t R e s o u rc e s R e q u ire d F ie ld s W B S /P ro je c t: M o d e l T yp e : M ach R ange: P la n n e d T e s t R u n s : P la n n e d T e s t H o u rs : O c c u p a n c y (m /y-m /y): M odel M ods: F a c ility M o d s : P a id fo r b y P ro je c t M od el D esig n M od el M od s M od el F ab O ther D e s c rib e O th e r E x p e n s e s 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 19 Test Facility Survey • Request test setup data based on use of facility systems identified in the facility user manual T u n n e l R e s o u rc e s (C h e c k a ll th a t a p p ly) M o d e l S u p p o rt S ys te m s M o d e l P re p a ra tio n B u ild in g M o d e l In te g ra te d S u p p o rt S ys te m G ro u n d P la n e E x h a u s t S u p p o rt S ys te m T u rn ta b le P e d e s ta l M o u n t S ys te m s In le t P re s s u re B le e d A ir S ys te m In le t V a c u u m B le e d A ir S ys te m U H B D rive R ig R o ta tin g R a k e S ys te m J e t E x it R ig Air P re s s u re S ys te m s H ig h P re s s u re A ir S e rvic e A ir C o m b u s tio n A ir H yd ra u lic S ys te m N itro g e n S ys te m G a s e o u s H yd ro g e n S ys te m In fra re d S ys te m L a s e r-D o p p le r V e lo c im e te r (L D V ) S ys te m L a s e r S h e e t (F lo w V is u a liz a tio n ) S ys te m F o rc e B a la n c e S ys te m Altitu d e E x h a u s t S ys te m P h o to g ra p h ic S ys te m E x te rn a lly S u p p lie d S ys te m s L is t: In s tru m e n ta tio n T h e rm o c o u p le s (e s tim a te d q u a n tity) A c tu a to rs & P o s itio n In d ic a to rs (e s tim a te d q u a n tity) M ic ro p h o n e s D a ta Ac q u is tio n & P ro c e s s in g E le c tro n ic a lly S c a n n e d P re s s u re (E S P ) S ys te m E s c o rt D P lu s D yn a m ic D a ta A c q u is tio n C e n tra l A n a lo g R e c o rd e r T R A D A R -3 T ra n s ie n t D a ta A c q u is itio n S ys te m F a c ility T a p e R e c o rd e rs Le ss Le ss Le ss Le ss A ve ra g e A ve ra g e A ve ra g e A ve ra g e M ore M ore M ore M ore Le ss Le ss A ve ra g e A ve ra g e M ore M ore Le ss Le ss Le ss A ve ra g e A ve ra g e A ve ra g e M ore M ore M ore Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e M ore M ore M ore M ore M ore M ore M ore M ore M ore M ore M ore M ore M ore U sed Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss Le ss C o m p le x ity A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e A ve ra g e M ore M ore M ore M ore M ore M ore M ore M ore M ore M ore 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 20 Model Inputs 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 21 Model Outputs Predicted Labor Requirements Estimated Labor Hours Civil Servant Labor Hours Project Office (P) Research Customer (R) Computing Science Division (VC) Test Engineering (DR) Test Technician (DR) Engineering (DD/DS) Fabrication (DDM/P) Central Test Operations (DF) PBC Labor Hours Direct PBC Test Engineering Test Technicians Fabrication Predicted Test Requirements Estimated Power (MWH) Estimated Occupancy (Months) Results 55,459 120 7,359 1,038 8,964 19,500 2,343 5,655 5,683 0 2,275 2,521 0 Results 16,380 11.22 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 22 Next Steps – Technology Inventory Database • Recommend advocating at HQs to use it as the technology side of the ONCE database – currently under Exploration Systems – decision for future direction to be issued in May • Define a formal, top-down approach to ensure all technology tasks are included • Coordinate with ESMD to change input requirements to incorporate actual versus planned data, ensure all and only needed data is collected, and minimize the task manager’s work to input the data • Develop multi-year reporting capability to include desired fields • Current POCs: Doug Craig/HQ, Mary Reph/GSFC (301) 286-1917 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 23 Next Steps – In-House Tools • Continue to ‘tweak’ the test model to incorporate recommended changes/enhancements. • Model, as configured, uses planned Occupancy, Test Runs and Test Hours for resource calculations. Based on historical changes in plans, develop expected numbers for all three factors to use to get a potential range. • Collect more test article procurement costs and technical data to enable cost estimation of the hardware/software. • Work other ‘low hanging fruit’ – groupings of technology tasks with easily obtainable data - as it is identified. 2005 NASA Cost Analysis Symposium Glenn Research Center at Lewis Field 4/13/05 24