Tutorial A: Cost Risk Analysis/Tim Anderson, The Aerospace Corporation October 25, 2005 8 a.m. to noon
This tutorial is for acquisition managers, systems engineers, risk managers, and engineering specialists and will focus on how to work with cost risk analysts to develop realistic cost probability distributions and the probabilistic nature of each program element of cost in a cost estimate. It will identify how risk information is used to create a probabilistic cost distribution used for decision-making while integrating program technical uncertainty and cost modeling variation into a probabilistic cost estimate using state-ofthe-art statistical methods. The risk-adjusted cost distribution can be used to determine the confidence of meeting budget constraints and also identify the budget necessary to have a specific level of confidence. The tutorial will show how to organize and display the probabilistic nature of the cost estimate in a way that is explainable to decision-makers with a range of possible costs as well as their likelihoods. Examples will illustrate how to set up and run the simulations and then properly interpret the results. Attendees will receive the Aerospace Corporation’s Cost-Risk Reference Card to use in their day-to-day work. Tutorial Instructor Background Tim Anderson is a Senior Engineering Specialist for The Aerospace Corporation. As a member of the Engineering and Technology Group, he provides cost estimating and operations research services to the National Reconnaissance Office. He has supported numerous government space programs. Retired in 2001, Mr. Anderson served for 20 years in the U.S. Navy and began working in the cost estimating field in 1994 while assigned to the Naval Center for Cost Analysis. From 1997 to 2001, he was a military professor at the Naval Postgraduate School, teaching cost estimation, operations research, and other technical courses. Mr. Anderson has an M.S. in Operations Research from the Naval Postgraduate School and a B.S. in Industrial and Operations Engineering from the University of Michigan. He is a SCEA certified cost estimator/analyst, an instructor with The Aerospace Institute, and a frequent presenter of topics related to cost estimating and cost risk analysis at forums including the Society of Cost Estimating and Analysis (SCEA), the Military Operations Research Society (MORS), the DoD Cost Analysis Symposium (DoDCAS) and the Space Systems Cost Analysis Group (SSCAG). Mr. Anderson also lectures at military, civil and commercial organizations.
Updated 8/29/05
Tutorial B: Best Practices for Acquiring Software-Intensive Space Systems/Suellen Eslinger, The Aerospace Corporation October 25, 2005 8 a.m. to noon
This seminar will present a practical, comprehensive set of software acquisition best practices identified as being significant contributors to the successful acquisition of software-intensive space systems. Developed by an Aerospace Corporation software acquisition research project, these best practices are based on experiences with numerous software-intensive space system acquisitions over the past 20 years. This seminar will cover the best practices through each phase of the National Security Space program acquisition life cycle in chronological order, including both pre-contract award and post-contract award activities. Practical information will be provided to the attendees to enable the immediate application of the best practices to their programs. Other sources of software acquisition best practices (e.g., from the DoD and associated organizations) will be discussed. The target audience for this seminar is everyone responsible for acquiring, or supporting the acquisition of, space systems containing complex, mission-critical software. Note that it is not targeted solely to software specialists. Tutorial Instructor Background Suellen Eslinger is a Distinguished Engineer at The Aerospace Corporation working on a variety of corporate level initiatives to set the benchmark for evaluating contractor performance on US government space programs. She has over 35 years experience in development and acquisition of complex/critical software-intensive DoD and NASA space and ground systems; full life cycle software/systems engineering and software/systems engineering management; and software engineering and software acquisition process definition and improvement. Ms. Eslinger has worked a range of space programs; satellite, ground and launch systems including Space-Based Infrared System (SBIRS) High and its predecessor programs; SBIRS Low and its predecessor programs; Global Positioning System (GPS); Space Based Radar (SBR); Air Force Satellite Control Network (AFSCN); Consolidated Space Operations Center (CSOC); Milstar; Evolved Expendable Launch Vehicle (EELV); Future Imagery Architecture (FIA) and several other classified programs. Suellen is a key instructor for courses at The Aerospace Institute, the Aerospace Corporation’s University. Her contributions include Space Systems Software Program Management, Software Engineering, Software Acquisition, Software Risk Management and Software Architecture.
Ms. Eslinger holds a M.S. in Mathematics from the University of Arizona and a B.S. in Mathematics from Goucher College in Maryland.
Updated 8/29/05
Tutorial C: PMI Risk Management Approach and Advanced Schedule Risk Analysis/Keith D. Hornbacher, M.B.A.Hulett & Associates, LLC, Representing the Project Management Institute RiskSIG October 25, 2005 8 a.m. to noon
This workshop has two components: (1) introduction to the Risk Management approach of the Project Management Institute as found in the Guide to the Project Management Body of Knowledge (PMBOK® Guide), and (2) detailed discussion of project schedule risk analysis, an outstanding example of what PMI calls “quantitative risk analysis.” The PMI discussion will explain the definition of risk that PMI uses, including opportunities as well as threats, and describe the six project risk management processes. The quantitative schedule risk analysis presentation will show that relying on CPM schedules for accurate estimates of project completion and identity of the critical path is risky. It shows how better estimates can be made using risk analysis. The results include: (1) an estimate of the likelihood of overrun, (2) the degree of overrun or schedule contingency reserve that is needed, and (3) which activities and paths are most likely to delay the project, which indicates where risk responses should be developed and implemented. We present the impact of correlation between activity durations and the use of probabilistic branching on the schedule results. Conditional branching provides a way to look at contingency plan trigger points. The use of constraints, resources and schedule status within a risk analysis concept will be discussed. Some software that performs simulations will be discussed and demonstrated. Tutorial Instructor Background Keith Hornbacher has held senior project risk management and analysis positions as a practitioner in assignments across North America and Europe. His experience spans a wide range from complex multibillion dollar programs to small innovative product development projects. Mr. Hornbacher was vice president and senior analyst in the firm that developed quantitative risk analysis methodologies and Monte Carlo simulation software. He currently consults and trains in project risk management, scheduling, and Project Management Office development through the firm, Hulett & Associates, LLC of Los Angeles. His clients have included the US Government and companies in diverse industries: aerospace and defense, energy, pharmaceuticals, transportation, construction, large science, medical technology and healthcare. He focuses on qualitative and quantitative technical, cost and schedule risk analysis and on project scheduling. In addition, he presents seminars and workshops in advanced project management topics as an adjunct faculty member of the University of Minnesota. Mr. Hornbacher has been active in the Project Management Institute (PMI) for more than two decades and contributed to revisions of the Project Management Body of Knowledge (PMBOK®) Guide. He is the PMI RiskSIG’s project manager developing a Project Risk Management Handbook and is Director, Project Risk Management, of PMI’s HealthCare SIG. He is an incumbent member of an M.B.A. Advisory Council and is a past director of INCOSE’s Northstar (Minnesota) Chapter. Mr. Hornbacher has presented papers on cost and schedule risk analysis to professional societies including components of PMI, INCOSE, and the conference of the College of Performance Management (CPM).
Updated 8/29/05
Tutorial D: The Systems Engineering/Risk Management Relationship/George Friedman, The University of Southern California October 25, 2005 8 a.m. to noon
Recent experience and disappointments in the development of new complex systems has increased the emphasis on systems engineering and risk management. Both disciplines have been emerging but only recently has work been done to truly integrate them. Risk management is not only as a recognized discipline within systems engineering, but as a crucial path to systems engineering’s improved rigor, academic acceptance, generation of alternative risk mitigation paths, enhanced employment of testing, and effective decision making. This tutorial will provide a quick refresher on the basics of risk analysis and risk management, will discuss qualitative and quantitative methods, cover rationality and decision theory, examine probabilistic logic of imperfect testing and provide examples. Tutorial information was integrated from a variety of sources: academic course work, research, working with the space industry, and professional society involvement. USC has extensive systems engineering and risk management education and this tutorial has been designed to provide the proven information on integrating risk management with systems engineering. Tutorial Instructor Background Dr. Friedman is presently Adjunct Professor in USC’s Viterbi School of Engineering and Associate Director of their Systems Architecture and Engineering Program, responsible for research in Systems Engineering. He has developed and teaches the graduate course “Advanced Topics in Systems Engineering.” Over 300 students from industry and government have taken his courses toward their master’s degree and he has sat on a dozen PhD committees. He serves on the advisory committee for AFIT’s system engineering program and has given guest lectures for INFORMS, INCOSE, MIT, and the Aerospace Corporation. Previously, he has had over 40 years of industrial experience on a variety of complex programs, primarily for DoD. He retired from Northrop as their Corporate Vice President of Engineering and Technology, where he was responsible for all engineering processes, including systems engineering and risk management. He is a founder of INCOSE, served as its 3rd president, was elected a Fellow, initiated a risk management special interest group, and presently serves as an editor on the peer-reviewed journal, Systems Engineering. He is a Fellow of the IEEE and served as VP Publications of the AES Society. He served on several advisory boards for the USAF, NSF, NASA, CIA, and NATO Education: BS, UC Berkeley; MS and PhD UCLA
Updated 8/29/05
Tutorial E: Principles and Practical Implementation of Space System Risk Assessment and Risk Management/Sergio Guarro and Karin Feldman, The Aerospace Corporation October 25, 2005 1 to 5 p.m.
This tutorial is suited to those that have the accountability of running a successful risk management program, planning the effort, identifying and assessing risks, managing risk information, working with others in this endeavor across the enterprise and reporting results to management. Attendees will receive the Aerospace Corporation Risk Management Reference Card used throughout the tutorial and to be used as a guide on the job. This tutorial will cover both the principles and underpinnings of space system risk assessment (RA) and risk management (RM) and the practical implementation issues that arise in complex, multi-stakeholder programmatic environments. The principles covered include basic definitions of RA/RM processes and models, and both qualitative and quantitative techniques for risk classification and handling. The relationship of programmatic RA/RM models to “classic PRA” (Probabilistic Risk Assessment) is explored and discussed as well. Practical RM implementation issues covered in the course include: multi-stakeholder environments; integration of Government and Contractor RM processes; calibration of risk classification / quantification scales; vertical integration of RM processes in “program of programs” hierarchical environments; RA/RM support of programmatic decision-making. The course discussion is complemented by real-life examples and mini-case-studies. Tutorial Instructor Background Dr. Sergio Guarro is the Director of the Office of Risk Planning and Assessment organization within The Aerospace Corporation. He has Over 30 years of professional experience, with early work in nuclear power plant simulation and control and nuclear materials research, and the last twenty years dedicated to systems effectiveness and programmatic decision support disciplines, including risk and reliability assessment and software safety analysis. Dr. Guarro is an internationally recognized expert in risk assessment and management and the programmatic application of systems engineering, reliability and safety analysis techniques in support of space system programs. He was an International Fullbright Scholar at UC Berkeley and a Fellow of the Advisory Committee on Nuclear Safeguards of the U.S. Nuclear Regulatory Commission. Before joining Aerospace he was a Project Leader with the Nuclear Systems Safety Program at the Lawrence Livermore National Laboratory. Dr. Guarro has published a scientific textbook and more that fifty technical papers and articles. He has taught courses on probability and risk assessment in the Department of Mechanical and Aerospace Engineering at UCLA and also on software safety and quality assurance at the California State University, Long Beach. He has earned Ph.D., E.D. and M.S. degrees in Nuclear Engineering, University of California Los Angeles as well as a Diploma di Laurea, Magna cum Laude, in Nuclear Engineering, University of Bologna, Italy. Karin Feldman is a Senior Member of the Technical Staff in Aerospace’s Office of Risk Planning and Assessment. Ms. Feldman supports the development and deployment of the Continuous Aerospace Risk Management and Assessment (CARMA) risk management tool. Ms. Feldman has also been involved in probabilistic risk assessments, reliability trade studies, and other reliability analysis projects on various space programs. She is an instructor for Risk Management at The Aerospace Institute and was a teaching assistant for Engineering Risk Benefit Analysis at MIT. Ms. Feldman has earned a M.S. in Nuclear Engineering from the Massachusetts Institute of Technology and a B.S.E. in Nuclear Engineering and Radiological Sciences from the University of Michigan, Ann Arbor.
Updated 8/29/05
Tutorial F: Program Executability Risk/Jack Wormington, The Aerospace Corporation October 25, 2005 1 to 5 p.m.
Program management, systems engineering management, and risk management are the basis for the acquisition program office defining the scope of its work and that of its mission partners and contractors. The government acquisition community has noted problems and deficiencies in acquisitions that have led to mission failures and acquisition delays. Acquisition professionals will benefit from this tutorial by gaining an understanding of how acquisition risks can be identified and handled. The workshop will include current problems in space acquisition, appropriately accepting a program after baselining, risks to the program, and approaches to handling program risks. Program examples, along with findings from recent independent program reviews at AF/SMC, will help participants benchmark their program. This presentation builds on the 5th SSERMS tutorial. It provides information on how to evaluate a program for risk and identifies common areas of risk across programs. This tutorial is well suited to those that manage, measure, report and need to correct program risks. Tutorial Instructor Background John R. “Jack” Wormington is Vice President of Program Assessment at The Aerospace Corporation. Wormington's responsibilities include working with senior leaders of the company's customers to establish independent review processes for national security space programs, conducting reviews of major acquisition programs, and making recommendations to improve the executability of programs. Wormington held senior leadership positions at Boeing Satellite Systems and Hughes Space and Communications before joining The Aerospace Corporation in 2002. His assignments in commercial space programs included leadership of the successful development of XM Satellite Radio and other commercial satellite programs, including all HS702 programs at Hughes Space and Communications. Wormington served in the U.S. Air Force from 1966 to 1995, retiring as a brigadier general. As a leader of military space programs, his responsibilities included direction of launch and range operations at Cape Canaveral Air Force Station and the Eastern Test Range at Patrick Air Force Base in Florida and leadership of the Advanced Launch System Program. He also created and led the Space Surveillance and Tracking System Program, among other achievements. In the Pentagon at Headquarters U.S. Air Force he was planning and program officer for space defense, Deputy Chief of Staff for Operations and Plans, and the space shuttle program element monitor, Deputy Chief of Staff for Research, Development and Acquisition. Wormington has Bachelor of Science degrees in astronautical engineering, physics, and engineering science from the U.S. Air Force Academy, a master's degree in astronautical sciences from Stanford University, an MBA from Auburn University and a master's in aerospace operations management from the University of Southern California.
Updated 8/29/05
Tutorial G: Continuous Risk Management/Barney Roberts, Futron Corporation Representing The INCOSE Risk Management Working Group October 25, 2005 1 to 5 p.m.
This tutorial is derived from activities of the members of the Risk Management Working Group (RMWG) of the International Council of System Engineering (INCOSE). The RMWG is investing energies into how the risk management process is integrated into the system engineering process, and how does one make decisions based on the risk analysis results. The RMWG is also devoting energies to the understanding how to measure the performance of the process. Finally, the RMWG is collecting case studies and other resources for the practitioner to use. The attendees will be provided with a CD of electronic resources. This tutorial provides the attendee with the benefits of a formal risk management process. The attendee will receive an overview of risk-based decision support products that have been proven to deliver high value to space system projects. The formal risk management process of planning, identifying, analyzing, handling and tracking are covered as well as specialized analytical approaches to assessing risk, both in the project management arena as well as the system engineering arena. Tutorial Instructor Background Mr. Barney Roberts is the chairman of the INCOSE Risk Management Working Group (RMWG). He has served as an officer and director prior to taking the chairmanship of the RMWG. He is well published in professional literature contributing over 70 papers and has won a Best Paper award three times for INCOSE presentations alone. Currently Mr. Roberts is the Director for the Futron Risk Management Center of Excellence. His role in that position is to formalize the risk management practices, procedures and tools that he has successfully developed and utilized in assisting customers in understanding and managing risk since 1994. He has developed advanced, integrated, quantitative techniques for cost-technical-schedule risk management assessments. He has provided risk management training and workshops at most NASA centers and many of their support contractors’ facilities. Barney has pushed the state-of-the-art in many diverse areas such as project management, system engineering, hardware and software development and certification, crew training, aerodynamic test and analysis, flight mechanics, flight control design and analysis, rocket engine analysis and test, computational fluid mechanics, heat transfer, and thermal systems design. Barney’s career includes over 30-years at the NASA Johnson Space Center (JSC) serving in a role of system engineering and project management for most of the agency’s manned spacecraft development activities. He served as the Manager of the Planet Surface Systems Office coordinating the strategic planning and development of concepts and operational strategies for human outposts on the Moon and Mars was a significant contributor to the development of the Space Shuttle. Among his accomplishments he has received the NASA Exceptional Service Medal and holds a patent for an aerobraking orbit transfer vehicle.
Updated 8/29/05
Tutorial H: Smarter Buyer: Mitigating Space Acquisition Risks/Pat Maloney and Frank Wong, The Aerospace Corporation October 25, 2005 1 to 5 p.m.
The government and industry have been seeking means to improve acquisitions so that they meet operational needs, are on schedule and on budget, and nurture the organizations involved. This tutorial, a condensed version of The Aerospace Corporation’s Smarter Buyer course, is a must for acquisition professionals. It will provide an understanding of the business climate, industry’s need to perform, government-industry interaction, and alternative means to incentivize proper corporate behaviors. Tutorial material, based on interviews with top industry and government officials, will cover acquisition risks and their implications and potential remedies. Topics will include a brief introduction to the space business environment, the need for industry programs to generate earnings, concerns of the CEO and space sector manager and how they impact the program, the industry business development process, how government decisions impact industry financials, and how to improve acquisitions through better industry program manager/government program director contract structuring and communications. Tutorial Instructor Background: Patricia A. Maloney is the Director of the Aerospace Corporation’s Economic and Market Analysis Center (EMAC). The EMAC analyzes the space business market conditions, market, and industrial base to advise government customers on how to incentivize industry. Previously, Ms. Maloney was with Mobil Corporation for 16 years. She held a variety of positions, including: CFO for the US Western Region, Torrance, CA; Associate Director of Project Finance for Asia Pacific, with responsibility for Indonesia, Singapore and Jakarta, Indonesia; Wholesale Marketing Manager and Strategic Planning Associate for US Marketing and Refining, Fairfax, VA, and Manager, Crude Oil Operations, Gulf and Far East, New York, NY. She also worked for ARCO Solar, Inc. on the negative electrode for NiH battery and the electrochemical deposition of copper indium disulfide solar cells Pat has earned an M.B.A. in Operations Research and International Finance from the Amos Tuck School of Business Administration at Dartmouth College and a B.S. in Chemistry from the California State University at Northridge. Frank Wong is a Senior Engineering Specialist at The Aerospace Corporation. He has worked in the satellite industry since 1990 as an electrical engineer, financial analyst, and market research analyst. Since joining The Aerospace Corporation in 2000, he has researched several industries including remote sensing, hyperspectral imagery (HSI), launch vehicles, TWTA, and GPS. He is also co-author and teacher of a “Smarter Buyer” course, designed to help government acquisition personnel understand how corporations operate from a financial perspective. Prior to Aerospace, he worked at Hughes Space and Communications, designing payload hardware, creating satellite proposals and supporting the Thuraya Satellite System Program Office. Mr. Wong has a BS and MS in electrical engineering and an MBA from UCLA. His MSEE focused on microwave electronics and antennas. His MBA was in finance and strategic marketing.
Updated 8/29/05