DOT E Observations Priorities U S Air Force T E

DOT&E Observations, Priorities U. S. Air Force T&E Days Aligning the T&E Enterprise ________ Destin, Florida Dr. Charles McQueary Director, Operational Test and Evaluation February 13, 2007 1. OT&E Observations 1. OT&E should be a period of confirmation, not a period of discovery. 2. T&E cycle time must be consistent with the needs of the Warfighter. 3. Suitability trend needs to be reversed—Reliability, Availability, and Maintainability (RAM). – During the past 3 years, 9 of 26 systems evaluated in BLRIP (Beyond Low-Rate Initial Production) reports to Congress were evaluated as not operationally suitable. 2. DoD IOT&E Results Air Force Programs Program Service ACAT IOT&E Result FY 2001 F-15 TEWS (Tactical Electronic Warfare System) Joint Direct Attack Munitions (JDAM) USAF II Effective Effective only with legacy fuses Not Suitable Not Suitable Reliability, Maintainability, Availability Integration with delivery platforms Reason USAF 1C FY 2002 Joint Primary Aircraft Training System (JPATS) USAF 1C Effective with deficiencies Not Suitable RAM, Safety, Human Factors FY 2003 B-1B Block E Mission Upgrade Program USAF 1D Not Effective Not Suitable 16% decrease in weapons release rate, reduction in accuracy of Mark 82 low drag weapons, 14% hit rate on moving targets RAM; needed more maintenance resources and spare parts; BIT FY 2005 F/A-22 USAF 1D Effective Not Suitable Army IOT&E Results: 7 Systems Submitted to IOT&E, 5 Found Effective and Suitable Navy IOT&E Results: 10 Systems Submitted to IOT&E, 5 Found Effective and Suitable 3. DOT&E Priority Enhance Operational Realism Early, Including in DT • Need to incorporate more operational realism into DT to gain operational insights, identify failure modes early. Early identification and correction is the only way to make OT&E a confirmation, and not a venue for late discovery. – Reduces development costs • • DOT&E will work with DoD and Services to better synchronize OT and DT, enhance discovery process during DT, and eliminate surprises in OT. T&E must respond and adapt to acquisition trends focused on developing and fielding systems on shorter timelines, by better integrating DT and OT to provide operational insights. 4. DOT&E Priority Improve Suitability • Suitability of systems requires additional focus. – Effectiveness and suitability are not conflicting concepts if both are considered and incorporated early in the design and developmental processes. • DOT&E must reemphasize the test community's efforts to identify failure modes and impacts early. DOT&E is sponsoring several studies to determine the actual costs of unsuitability, and the optimum place in the development cycle to insert resources. • “It is far more important for a system to be effective when it is needed than when it is available.” 5. DOT&E Priority Provide Timely Performance Info to the Warfighter • New Title Ten authority (via FY 07 NDAA) requires DOT&E to: – Provide additional information on operational capabilities • At the conclusion of OT&E, or; • When a decision for operational use or to use procurement funds is made before a formal decision to proceed beyond low-rate production • Assessments must support full-rate production decisions, AND informed rapid fielding decisions. – DOT&E will assist OTAs in ensuring joint warfighters are aware of system capabilities and limitations when fielded early • Requires a mission-focused, realistic OT • If full-scale OT not possible, some realism must be infused during DT to provide operational insights The key is early engagement, careful following of all testing, and greater use of modern techniques for collecting, sharing, evaluating info. 6. DOT&E Priority Support the Testing of Force Protection Equipment • New Title Ten authority (via FY 07 NDAA) requires DOT&E to provide guidance and consultation on the testing of force protection equipment. – Based on past successful DOT&E efforts to support T&E of personnel body armor, armor protection of wheeled vehicles, and IEDs • To fulfill this new mission, DOT&E will recommend a new DoD policy on testing of force protection equipment. – DOT&E staff will work with the OTAs to provide informed counsel on operational and survivability T&E of force protection equipment – Will need to broaden DOT&E expertise to look at different measures against which force protection and non-lethal systems are evaluated • Ex: Non-lethal systems have a safety threshold they cannot exceed to ensure they are “non-lethal” 7. DOT&E Priority Examine the Allocation of OT Resources • What are the resource implications of greater and earlier involvement in T&E and our current approach to OT&E and LFT&E? Two hundred and ninety-three acquisition programs currently under DOT&E oversight. – What are the right criteria to determine when to apply oversight? – What is the right level of oversight to be applied based upon varying acquisition approaches/strategies? • • Additional responsibilities and new acquisition paradigms may require additional or different training and expertise. 8. DOT&E Priority Assist in the Review of Test and Evaluation Policy • FY 07 NDAA requires USD(AT&L) and DOT&E to review T&E policies and practices, and: – Reaffirm T&E principles that should guide traditional acquisition – Determine how best to apply appropriate T&E principles to emerging acquisition approaches • 2006 National Research Council study of T&E in the new acquisition environment (Testing of Defense Systems in an Evolutionary Acquisition Environment), serves as a useful starting point. – Recommendations: • Require that developmental tests have an operational perspective; • Require contractors to share all relevant data on system performance and the results of M&S developed under government contracts; • Require that all technologies demonstrate sufficient technological maturity before being inserted in a later stage of development – http://books.nap.edu/catalog/11575.html#toc 9. DOT&E Organization Dr. Charles E. McQueary DIRECTOR Mr. David Duma - 703- 697- 4818 Principal Deputy Director Deputy Director- MDA Dr. Ernest Seglie - 703- 697- 3655 Science Advisor Mr. Steve Daly Deputy Director Land Warfare 703-697-3891 Land Combat Systems Rotary & Tilt Rotor Aircraft Land Munitions Land Combat Tactical UAVs Chem-Bio-Rad-Nuc Systems --------------------------------Integrated Resource Analysis Team Threat Systems Office Mr. Mike Crisp Deputy Director Air Warfare 703-692-9929 Air Combat Systems Strategic Air Combat Sys. New Aircraft Carriers Air Launched Munitions Strategic, Operational UAVs --------------------------------Joint T&E Program Center for Countermeasures Joint Test Roadmap Mr. Tom Blann Deputy Director Naval Warfare & Net-Centric Systems 703-681-1440 Surface Combatants Amphibious Ships Submarines Naval Munitions Space & Strategic Sys. Strategic C4ISR Sys. Major Automated Info Sys. --------------------------------Information Operations Information Assurance Mr. Richard Sayre Deputy Director Live Fire T&E 703-614-3991 All LFT&E (Lethality & Survivability) --------------------------------Joint Live Fire Program Joint Aircraft Survivability Program Joint Technical Coordinating Group— Munitions Effectiveness 10.