Docstoc

Slide 1 - DAU Home Page

Document Sample
Slide 1 - DAU Home Page Powered By Docstoc
					                                         Assessing Manufacturing Risks
                                          Production Readiness Reviews                               Page 1

Charge of the Light Brigade

        “Half a league, half a league,
           Half a league onward,
          All in the valley of Death
           Rode the six hundred.
        “Forward, the Light Brigade!
       Charge for the guns!” He said.
          Into the valley of Death
           Rode the Six Hundred.



 The Charge of the Light Brigade is best remembered
 as a symbol of warfare at its most courageous and
 most tragic. But military blunders are not new or
 uncommon and neither are the mistakes we make
 as acquisition managers. Not assessing
 manufacturing risk, both early and often, is a
 common mistake that can lead to poor performance,
 and cost and schedule overruns.                              Source: National Army Museum, London
                                        Assessing Manufacturing Risks
                                         Production Readiness Reviews                              Page 1

Charge of the Light Brigade (Optional intro screen)

       “Half a league, half a league,
          Half a league onward,
         All in the valley of Death
          Rode the six hundred.
       “Forward, the Light Brigade!
      Charge for the guns!” He said.
         Into the valley of Death
          Rode the Six Hundred.


                         The Charge of the Light Brigade is best        Source: National Army Museum, London
                         remembered as a symbol of warfare at its
                         most courageous and most tragic. But
                         military blunders are not new or uncommon
                         and neither are the mistakes we make as
                         acquisition managers. Not assessing
                         manufacturing risk, both early and often, is a
                         common mistake that can lead to poor
                         performance, and cost and schedule overruns.
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews               Page 2

Lesson Learning Objective

 Describe the Production Readiness Review (PRR)
  process how, when and where it might be used.

 Enabling Learning Objectives:

 • Identify the role and goal of manufacturing
   and how risks can affect achieving that goal
 • Describe a PRR and the guidance associated with
   PRRs
 • Identify how and when a PRR will be conducted
   from a macro perspective
 • Describe a PRR from a micro perspective




                                                             JSF Production Line
                                                             Source: navy.mil
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews   Page 3

Role of Manufacturing

The Role of Manufacturing is to:
• influence the design
• prepare for production, and
• execute the manufacturing plan.

Manufacturing needs to take actions that will reduce
 overall program cost, schedule and performance.
 A poorly designed product may not be affordable
 and may not meet the needs of the user. The
 manufacturing engineer needs to ensure that the
 design is producible. The manufacturing team
 needs to ensure that the factory floor is well
 characterized and ready for production. This
 includes planning for production and then
 executing that plan.

The goal of manufacturing is to provide uniform,
 defect-free product that achieves consistent
 performance and at a lower cost.
                                       Assessing Manufacturing Risks
                                            Production Readiness Reviews                       Page 4

Influence the Design

Producibility Engineers work with design, manufacturing, test and other engineering functions to
  influence the design to ensure that the design can be efficiently manufactured
  (manufacturable). Producibility is the measure of the relative ease of making a product. That is,
  you can manufacture a part out of inexpensive material, using unskilled workers, simple tools, and
  manufacture it in a very short time. Some early design considerations include the use of
  economical materials, efficient manufacturing techniques, use standardized parts and components,
  ensure your processes are repeatable, and inspectable.

 Example: The Navy was evaluating it‘s
 F/A-18 program for improvement
 opportunities and came across the
 Oxygen Bottle Holder. Using the
 application of Design for Manufacturing
 and Assembly (DFMA) process they
 were able to reduce part count and
 improve the design‘s efficiency by 80%.
 Reducing part count not only improved
 the manufacturability of the item, but
 also improved the ability to operate the
 item and maintain it.
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews                                      Page 5

Prepare for Production

Production Planning is often conducted at a number
  of levels:

• Long-range planning is generally accomplished
  annually, tend to be plant-wide and are focused
  on the out-years.
• Medium-range planning usually covers a period of
  6-18 months with time increments that are
  monthly or quarterly and are more product
  focused.
• Short-range planning covers the period from one
  day to six months with weekly time increments.

Production Planning and Control should consider
 the following:
   • Production Plan
   • Master Production Schedule
   • Material Requirements Planning
   • Capacity Requirements Planning

                                                     http://www.army.mil/aps/07/images/photos/Web/collage.jpg
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews                                  Page 6

Execute the Manufacturing Plan

Levels of sophistication of Production Planning and
  Control vary widely. These include elaborate
  software systems such as Enterprise Resource
  Planning (ERP), and Manufacturing Resource
  Planning (MRP II), to simple flow charts and hand-
  written papers. Major planning concerns can
  include:
    – How much labor is needed
    – Variations in personnel experience
    – How much and what kind of equipment
    – How large a facility is needed
    – How much material
    – The amount of financing required

Planning is an important activity, but the execution
  of the manufacturing plan and management of all
  the activities required to make the plan succeed is      http://www.logcom.usmc.mil/maintctr/images/mrp2.gif

  critical to program success.
                                             Assessing Manufacturing Risks
                                                Production Readiness Reviews                                       Page 7

Uniform, Defect-Free Product

The goal of manufacturing is to provide uniform,
 defect-free products. Philip Crosby defined
 quality as ―conformance to requirements.‖ You
 achieve that by reducing variation around the
 target value. Crosby is also credited with coining
 the term “Zero Defects” which was a part of his
 14-step Quality Improvement Process.

Zero Defects:

Crosby introduced the Zero Defects approach as a
 quality control program at the Denver Division of
 the Martin Marietta Corporation on the Titan
 Missile program, which carried the first astronauts
 into space in the late 1960s. It was then
 incorporated into the Orlando Division, which built
 the mobile Pershing Missile System, the Sprint
 antiballistic missile, and a number of air to ground http://www.smdc.army.mil/smdcphoto_gallery/eagle/Jun07/17-LLS-
 missiles used during the Vietnam War.                Tom%202.JPG Tom Ray teaches the principles of Lean Six Sigma to
                                                            green belts during a class on Redstone Arsenal, Ala.
                                       Assessing Manufacturing Risks
                                          Production Readiness Reviews                                 Page 8

Consistent Performance
Products must perform consistently. Wheather you
  are at home or at work inconsistent
  performance could lead to poor reliability and
  high maintenance costs. We all know what
  happens when you buy a car that is always having
  problems. It is in the dealers shop all the time
  and you are not a happy customer.

Inconsistent performance could lead to disaster
  on the battlefield. In the early years of the Viet
  Nam War, M-16‘s were prone to jamming. This
  caused soldiers to carry back-up weapons. The
  enemy carried the AK-47 and that weapon was
  much more reliable and able to withstand the
  harsh jungle environment. The AK-47‘s design
  was simpler and the manufacturing tolerances
  more robust. These together made the AK-47 a
  superior weapon from a reliability point of view.
                                                       http://www.history.army.mil/images/matrix/101abd/photos/
                                                       SC632687t.jpg
www.army.mil/-images/2008/10/17/24009/
                                         Assessing Manufacturing Risks
                                            Production Readiness Reviews   Page 9

 Lower Cost (Affordable)

  Affordability often goes back in part to a producible
    design, the use of efficient manufacturing processes
    and the reduction in variation and low sustainment
    costs. The B-58 Hustler was an engineering marvel.
    It was the Air Force‘s first supersonic bomber. But,
    it was in the Air Force inventory for only 10 years.
    Why?

  The demise of the B-58 occurred for several reasons.
   Originally intended for high speeds and high
   altitudes, the introduction of Soviet Surface-to-Air
   Missiles (SAMs) forced the B-58 into a low-level
   penetration role that it was not designed for. In
   addition, The B-58 was extremely expensive to
   acquire.. It was a complex aircraft that required
   considerable maintenance, much of which required
   specialized equipment and ground personnel. The B-
   58 was three times as expensive to operate as the
   B-52. Also against it had a high accident rate:
   22.4% of the production aircraft were lost in
   accidents. In short, the B-58 was not affordable.
                                      Assessing Manufacturing Risks
                                         Production Readiness Reviews    Page 10

Knowledge Review

Which of the following is not one of the roles of manufacturing?



           A. Influence the design




            B. Ensure the proper conduct of the Critical Design Review



             C. Prepare for production



            D. Execute the Manufacturing Plan




      Answer: B. Influence the 5Ms
                                        Assessing Manufacturing Risks
                                              Production Readiness Reviews    Page 11

Knowledge Review

The goal of manufacturing is to provide __________ that achieves consistent
performance and at a lower cost.


           A. a producible design




            B. uniform, defect-free product



             C. a manufacturing plan



             D. hardware and software



      Answer: B: uniform, defect-free product
                                      Assessing Manufacturing Risks
                                         Production Readiness Reviews                           Page 12

Common Manufacturing Risks

The goal of manufacturing is to provide uniform, defect-free product that achieves consistent
 performance and at a lower cost and deliver that product to the warfighter. However,
 achieving that goal is difficult because of the many manufacturing risks. One way to look at
 manufacturing risk is to use the fishbone diagram below that depicts the Design process and
 the 5Ms. Each of these is a potential source of risk. Select each 5M to learn more.



                       Measurement    Materials   Machines




                             Design                                 Product




                                  Methods    Manpower
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews                       Page 13

Common Manufacturing Risk: Design

Unstable Requirements / Engineering Changes:

Any instability is bad for the factory floor. When the
 customer‘s requirements change or the product
 design changes it impacts the factory floor.
When a design is set, the factory floor gets set to
 produce that design and when the design is
 changed significant time and resources (5Ms) may
 be needed to reconfigure the manufacturing
 processes for the new requirements. Impacts are
 even greater if the requirements or design changes
 frequently.



Example: The Osprey started off as an aluminum
bird, but the requirement to make it stealthy
changed many of the external features to
composites. This drastically changed the 5Ms and
significantly added cost and schedule growth to the
original program.
                                                                  Source: www.history.navy.mil
                                         Assessing Manufacturing Risks
                                            Production Readiness Reviews                   Page 14

Common Manufacturing Risks: Design (continued)

Producibility:

Is the measure of the relative ease of making a product.

It takes into account not only the simplicity (or lack thereof) of
  the design in terms of component features, but also the
  intent to reduce complexity by judicious selection of the
  manufacturing methods, tools, and processes used to
  create/modify those components. However, producibility can
  be a risk because in many organizations, it is assumed to be
  simply the ability to actually make a product that functions.

  Example: The graphic on this page is an illustration of
  how an item can be created (designed) on paper, but                 Can you build it?
  cannot be manufactured. In this case the design is just
  an illusion, but then thinking that just because you
  designed it you can build it is also an illusion. The main
  point here is that ―just because you can designed it
  does not mean you can build that design efficiently.                Source: PQM 301 course
                                      Assessing Manufacturing Risks
                                         Production Readiness Reviews                          Page 15

Common Manufacturing Risks: Design (continued)

Configuration Management:

One aspect of managing the design is consistently
 managing and controlling any changes to the
 functional and physical characteristics of an
 item. Uncoordinated and unrecorded changes
 that are made in a rush to ―fix the item‖ or to
 ―get the product out the door‖ to meet schedule
 can impact manufacturing processes used to
 create that item. While the design may fix the
 performance or other technical problem,
 manufacturing may now have to scramble to
 meet affordability constraints.

                                                               Two Nellis AFB F-16s with different
                                                               paint schemes (Air Force Photo)
Example: During the early 1980‘s the F-16 program was having a pod built that was for Low
Altitude Navigation and Targeting Infrared for Night (LANTIRN) system. During test in Maine
the engineers were swapping out boards from different pods in an attempt to get good test
results. After getting the results they wanted, they made the decision to move forward with
LRIP not knowing the real pedigree of the parts or processes.
                                           Assessing Manufacturing Risks
                                             Production Readiness Reviews                                Page 16

Common Manufacturing Risks: Measurement

Metrology and Calibration:

Often overlooked are the systems needed to measure
 things from the raw material to the testing of the
 final system. These measurement systems provide
 for precision and accuracy in the manufacturing
 process.

Examples of measurement include:
• Inspection,
• Gauges,
• Tolerances, and
• Statistical Process Control (SPC).




Example: Eglin’s Precision Measurement Equipment Laboratory (PMEL) provides for the calibration and
 repair of test, measurement and diagnostic equipment and ensures measurement traceability through the Air
 Force Primary Standards Laboratory to the National Institute of Standards and Technology (NIST). The Eglin
 PMEL support 22,000 pieces of test equipment for over 400 customers using 75 technicians.
                                            Assessing Manufacturing Risks
                                               Production Readiness Reviews                               Page 17

Common Manufacturing Risks: Measurement (continued)

Special Test Equipment:

Special test equipment is needed, to test or otherwise
 measure and evaluate the functional and physical
 characteristics of a specific item, component,
 assembly or subsystem. Special test equipment
 may require its own design, development,
 manufacturing, and support efforts, and can be a
 significant challenge. New test equipment may
 require new skills on the shop floor that could also
 entail training and certification requirements. These
 new requirements can be expensive and may have
 long-lead times associated with the development
 and procurement of these test items.


Example: Lockheed Martin Corp., Fort Worth was awarded a $190,000,000 cost-plus-incentive-fee
 modification to a previously awarded advance acquisition contract for special tooling and special test
 equipment associated with the Lot two Low Rate Initial Production (LRIP II) of the F-35 Joint Strike
 Fighter. This contract is an example of how special tooling and test equipment can impact a program
 just from a cost perspective.
                                     Assessing Manufacturing Risks
                                       Production Readiness Reviews                         Page 18

Common Manufacturing Risks: Materials

Changes to the supply chain:

Material includes raw materials to parts, assemblies
 and subassemblies for the system and for the
 production equipment itself. Much of the production
 is actually performed by subcontractors, vendors in
 the ―supply chain.‖ A change in just one supplier
 could have a dramatic impact on production costs
 and schedule. A different supplier may not have the
 same quality levels as the original supplier.
Sometimes companies change suppliers because the
 original equipment manufacturer has decided not to
 produce an item any longer and that item becomes         Abrams M1A2 Tank with TUSK improvements
 an obsolescence issue.                                   Requires Prime/Subcontractor integration,
                                                          Source: U.S. Army Public Affairs release


Example: In A 2007 briefing at the DMSMS Conference, Mr. Kevin Fahey, PEO, Ground Combat
Systems, pointed out that the M1A2 had 397 obsolescence issues that impacted domestic and
foreign sales during that year. DMSMS and obsolescence is an issue that impacts many programs,
large and small, complex and non-complex.
                                          Assessing Manufacturing Risks
                                             Production Readiness Reviews                             Page 19

 Common Manufacturing Risks: Materials (cont)

  Insufficient Materials Characterization:

  Many technology development efforts focus on
     understanding the materials performance
     characteristics as scientist and engineers seek to
     improve the Technology Readiness Level (TRL) of that
     material. Manufacturing engineers, if they are going
     to utilize this new material in production need to
     understand the materials properties, manufacturing
     procedures and quality assurance or inspection
     requirements. Failure to sufficiently characterize
     materials could lead to poor yields, low reliability and
     a host of other problems.
Example: Metal Matrix Composites have been around for over 40 years,
but did not come into widespread use until the mid-80’s. A Dec 2003
Army Research Lab study on Nondestructive Evaluation of a Metal
Matrix Composite has shown that many manufacturing issues came
to light as various producers pushed the boundaries of processes       ARL Material Science Lab Nanoscale
and performance without fully understanding or characterizing          Properties Project
the new material. The need to characterize the manufacturing           Source:
characteristics of a new material is an inherent program office        http://www.almc.army.mil/alog/issues/Nov
tasks.                                                                 Dec06/Story_Images/MS331_p1.jpg
                                         Assessing Manufacturing Risks
                                           Production Readiness Reviews                           Page 20

Common Manufacturing Risks: Machines
                                     F-14 in Depot Maintenance
Special Tooling:                     Source: http://www.navair.navy.mil/frcsw/images/news/archive/apprentice.jpg

Special tooling is structure (jigs and fixtures) that is used
 to position, hold in place, or otherwise secure an item,
 component or subassembly so that a manufacturing
 process can be properly performed on a part. The B-2
 bomber had millions of dollars worth of ―non-recurring‖
 costs for production of that was attributed to special
 tooling needed to hold structural assemblies in place for
 low observable materials processing. Much of the costs
 involved in past ―hard‖ special tooling are due to the
 need for precise dimensions; the need for proper design,
 development, manufacturing, upkeep and support of the
 tooling itself. So while the cost of special tooling can be
 high, the value is also high


Example: The F-14 Tomcat program had been in production for several years when quality control
personnel started seeing more defects that were attributed to wear on special tooling (meonite
dies). Grumman design, quality and manufacturing engineers determined that the F-14 needed a
$10M tooling refurbishment program to get the processes back in spec.
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews                          Page 21

Common Manufacturing Risks: Machines (continued)

Variation:

Machines varies in types This is often dependent on the
 volume of production. Machines also vary in terms of their
 ability to deliver quality.

Product or process that vary from target values can cause
  production and post production problems. For example,
  fitting a bolt that was built on the high side into a hole that
  was drilled on the low side may make the fit difficult to
  accomplish and longer build times. This same variation
  could lead to poor alignment on moving parts which could
  then lead to excess vibration and a lower reliability rate in
  the field.
 Example: In the early 1980‘s Ford built a new plant in Batavia, Ohio to build transmissions for
                                                                         Source: PQM 301 course
 the Escort. Sales were so brisk Ford could not keep up with production and asked Mazda to
 build transmissions. Mazda built the transmissions to print and shipped them to Ford where
 they were added to the assembly line just as if they were built by Ford. Reliability data began
 coming in showing that the Mazda transmissions were more reliable so Ford investigated. And
 they found that Mazda transmissions were more reliable because Mazda not only controlled the
 variation, as did Ford, but they also continued to reduce variation on key characteristics.
 Reducing variation on the key characteristics was critical to Mazda building more reliable
 transmissions.
                                           Assessing Manufacturing Risks
                                             Production Readiness Reviews                           Page 22

  Common Manufacturing Risks: Methods
Insufficient Process Proofing:

Methods represents the way that raw materials are formed, shaped,
 and held together.

Contractors need to demonstrate that new processes have been
 successfully applied in an situation that closely resembles the actual
 final manufacturing environment. The less realistic the initial
 manufacturing, the less confidence there is that the end item will
 function as desired. Insufficient process proofing is a very common
 risk when organizations that are involved in developing the                 SSN 777 Construction
 prototype items (labs) are used to develop and mature the                   Source:
 manufacturing methods to make that item.                                    www.navysite.de/ssn/ssn777.htm




   Example: The Seawolf sub‘s design replaced HY80 steel with HY100 for additional
   strength. The program began production testing on the HY100 steel at two shipyards on
   some full-scale parts. But the Newport News shipyard did not identify key characteristics
   nor did they fully identify interactions between material factors that later caused the welds
   to crack. This led to the development of improper welding practice that resulted in cracks
   in the welds. This indicates that manufacturing had not gotten control of those processes.
                                          Assessing Manufacturing Risks
                                            Production Readiness Reviews                              Page 23

Common Manufacturing Risks: Manpower
                                                                Source: www.bts.gov/transportation_statistics
  Unstable Production Quantities and Rates:

  Programs often undergo changes to the demand
    for product. Sometimes the requirement is to
    surge, but more often it is to reduce the original
    planned production quantities. When the
    customer‘s demand—quantities or delivery times
    is significantly increased or decreased, there are
    serious impacts to production operations and
    cost. Contractors must be able to size their
    facilities and workforce based on reasonable rate
    and quantity estimates if they are going to be
    able to maintain the readiness to produce.



Example:
The F/A-22 was re-baselined 14 times between 1992 and April of 2004, the DDG 51 11 times
between 1993 and Aug 2002, and the SM-2 Block V, 11 times between 1993 and Aug 1999.
While re-baselining to meet budget constraints is often expected, it does have a negative impact on
manufacturing planning and costs as the factory floor now has to accommodate new production
rates.
                                            Assessing Manufacturing Risks
                                               Production Readiness Reviews                            Page 24

Knowledge Review

Achieving the goal of manufacturing (provide uniform, defect-free product s difficult because of the
many manufacturing risks. Which of the following is not one of the 5Ms?


              A. Manpower




              B. Measurement



               C. Management



               D. Machines




       Answer: C. Management
                                     Assessing Manufacturing Risks
                                       Production Readiness Reviews   Page 25

Knowledge Review

Changes in the supply chain is associated with which of the 5Ms?



           A. Manpower




            B. Material



             C. Methods



            D. Machines



      Answer: B: Material
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews               Page 26

Lesson Learning Objective

 Describe the Production Readiness Review (PRR)
  process how, when and where it might be used.

 Enabling Learning Objectives:

 • Identify the role and goal of manufacturing and
   how risks can affect achieving that goal
 • Describe a PRR and the guidance associated
   with PRRs
 • Identify how and when a PRR will be conducted
   from a macro perspective
 • Describe a PRR from a micro perspective




                                                             JSF Production Line
                                                             Source: navy.mil
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews   Page 27

Production Readiness Reviews (PRR) – A Description

Production Readiness Reviews (PRRs) are manufacturing
  reviews that are used to identify risks, issues and
  opportunities early. The reviews are used to assess the
  capability of the manufacturing process to deliver on time
  and within cost, as well as to influence the design process
  and prepare for production. The primary focus is on
  identifying risks associated with:

   • Manufacturing Plan as part of the Program Acquisition
     Strategy.
   • Producible and sustainable design.
   • Control of manufacturing and industrial production
     processes (e.g. variation reduction, etc.) to be used or
     developed and used.
   • Identification of adequate resources necessary to
     execute production requirements, including full rate
     production and associated capacity issues.
   • Layout and characterization of the factory floor.
                                         Assessing Manufacturing Risks
                                            Production Readiness Reviews                         Page 28

PRR Guidance

The Defense Acquisition Guide, Chapter 4.3.3.4.7
 provides the following guidance:

The PRR examines a program to determine if:

• the design is ready for production
• the production planning is adequate
• production or production preparations incur
  unacceptable risks

The review evaluates the full, production-configured
 system to determine if:

• it correctly and completely implements all system
  requirements
• traceability of final system requirements to the final
  production system is maintained                          Photo source: marcorsyscom.usmc.mil
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews                               Page 29

PRR Guidance (continued)

Defense Acquisition Guide 4.3.3.4.7

A successful review is predicated on:

• the final production configuration meets all system
  requirements
• Having a production capability for proceeding into
  Low-Rate Initial Production and Full-Rate Production




                                                           Members of the DCMA Redondo Beach Joint
                                                           Process Action Team (JPAT)
                                                           Source: DCMA
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews               Page 30

PRR Guidance (con’t)


The program manager should convene a PRR of the
 prime contractor and major subcontractors.

The PRR(s) should be conducted in an iterative
 fashion, concurrently with other technical reviews
 (SVR, PDR, CDR, etc.).

Periodic production readiness assessments should be
  conducted to identify and mitigate risks as the design
  progresses. The "final" PRR should occur at the
  completion of the Engineering and Manufacturing
  Development (EMD) phase and should support the            Photo source: DCMA.mil
  Milestone C decision.
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews            Page 31

PRR Guidance (continued)

The program manager should tailor the PRR to the
 technical scope and risks associated with the
 system, and provide the details in the Systems
 Engineering Plan.

A follow-on tailored, PRR may be appropriate in the
  Production and Deployment phase for the prime
  contractor and major subcontractors if:

• Changes from the EMD phase and during the
  production stage of the design, in either materials
  or manufacturing processes, occur,
• Production start-up or re-start occurs after a
  significant shutdown period,
• Production start-up with a new contractor, or
• Relocation of a manufacturing site. the
                                                         Photo source: DCMA.mil
                                        Assessing Manufacturing Risks
                                           Production Readiness Reviews   Page 32

PRR Guidance (continued)

Part of early manufacturing planning in the program
  office is to ensure that there is a requirement for
  PRRs in their acquisition planning and contract
  documents. If there is no requirement for a PRR
  then assessing the risk for going into production is
  going to be very difficult.

The contractual requirement for PRRs at the prime
 contractor level needs to include critical
 subcontractors and vendors. And the flowdown to
 the subcontractors and vendors needs to include
 government participation as a part of the prime
 contractor team.
                                    Assessing Manufacturing Risks
                                       Production Readiness Reviews                    Page 33

Knowledge Review

According to the DAG 4.3.3.4.7, the PRR examines a program to determine if the
_____ is ready for production and if the prime contractor and major subcontractors
have accomplished adequate production planning without incurring unacceptable risks.
Which of the following best fits the blank?

           A. 5Ms




           B. design



            C. Manufacturing Plan



            D. Contractor




     Answer: B. design
                                      Assessing Manufacturing Risks
                                           Production Readiness Reviews                  Page 34

PRR’s Identify Risk

 The PRR(s) should be conducted in an iterative fashion, concurrently with __________.




            A. In-Process Reviews (IPRs)




            B. the Technology Readiness Assessment (TRA)



             C. Functional Configuration Audit (FCA)



             D. other technical reviews (SVR, PDR, CDR, etc.).


       Answer: C: other technical reviews (SVR, PDR, CDR, etc.).
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews   Page 35

You’ve Got Some Explaining to do
Perhaps many of you remember the ―I Love Lucy‖
  classic where Lucy and her friend Ethyl get a job in a
  chocolate factory wrapping chocolates. Things were
  going well when the factory was in low rate
  production, but the laughs start when the production
  line kicks into high gear.

One of the main differences between the ―I Love Lucy‖
 show and real life on the factory floor is that you can
 laugh about Lucy‘s predicament, but it is not so
 funny when it happens in real life.

 Perhaps if Lucy and Ethyl conducted a PRR prior to
 joining the company they would have been better
 prepared for Full Rate Production.
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews               Page 36

Learning Objective

Describe the Production Readiness Review process
 how, when and where it might be used.

Enabling Learning Objectives:

• Identify the role and goal of manufacturing and
  how risks can affect achieving that goal
• Describe a PRR and the guidance associated with
  PRRs
• Identify how and when a PRR will be
  conducted from a macro perspective
• Describe a PRR from a micro perspective




                                                             JSF Production Line
                                                             Source: navy.mil
                                           Assessing Manufacturing Risks
                                              Production Readiness Reviews                                Page 37

The Macro View of a PRR

A PRR is conducted to determine:

• if a program is ready for production,
• to identify manufacturing related risks, and
• to verify that the prime contractor its suppliers
have the manufacturing capability and capacity to
perform the work as defined in the contract.

The macro view is a top level view of ―how‖ to
conduct a PRR and includes the following activities
or steps:
1. Planning
2. Execution
3. The Outbriefing
4. Follow-up

Note: Key to being able to conduct the review is having the
   PRR called out as a contract requirement.
                                                       http://www.schriever.af.mil/shared/media/ggallery/webgrap
                                                       hic/AFG-061213-020.jpg
                                       Assessing Manufacturing Risks
                                          Production Readiness Reviews                     Page 38

1. Planning: Organization

Planning for a PRR includes defining the organizational
  structure, to include manpower requirements, and a charter
  establishing the scope, depth, criteria, procedures, and
  schedule for the review. The team should include:

•   A senior individual to chair the PRR team. The team leader
    should determine the team membership, organize and
    manage the team efforts, and supervise preparation of the
    findings.
•   Individuals having industrial, production and QA training
    and experience that qualify them to probe program
    accomplishments and plans in sufficient depth to make
    objective judgments of production readiness and risks.
•   Technical specialists should be used to augment team           Source: www.au.af.mil
    membership when specific expertise (avionics, advanced
    materials, software, etc.) is needed.
•   The team should develop a Charter that clearly outlines
    responsibilities and activities to be accomplished, both by
    government personnel and contractor personnel.
                                       Assessing Manufacturing Risks
                                          Production Readiness Reviews                       Page 39

1. Planning: Team Leader

The Government PRR Chairperson is responsible for:

• Developing the PRR Plan and overall preparation
  and operation of the team to ensure that all areas
  are reviewed in accordance with established
  procedures
• Coordinating with contractor counterparts to
  ensure that the government personnel have
  access to contractor data
• Preparation of the final report and for presenting
  the findings and recommendations to the senior
  management for the government program office
  and contractor program office
• Providing an approved Production Readiness                       Source: www.ih.navy.mil
  Review Plan along with assessment questions to
  the contractor prior to the visit

Note: The PRR is not a surprise inspection.
                                       Assessing Manufacturing Risks
                                          Production Readiness Reviews                    Page 40

1. Planning: Government Team Members

The Government team members are responsible for:

• Objectively assessing the associated level of risk in
  their assigned areas in accordance with the PRR Plan
• Utilize their judgment, experience and knowledge to
  determine the appropriate topics to address and
  questions to ask
• Keep adequate records so that they can document
  and defend their conclusions and recommendations
• Keeping all members of their team informed of their
  actions and findings. This is especially true for
  keeping their team chairperson and their contractor
  counterpart informed.

                                                                   Source: www.army.mil
DCMA representatives should be included as a part of
 the PRR assessment team.
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews                    Page 41

1. Planning: Contractor Team members

The contractor team members are responsible for:

• Providing a focal point for the visit to ensure
  security and other administrative considerations are
  taken care of prior to the review
• Ensuring that the pre-assessment or self-
  assessment questions are answered and that the
  contractor team is prepared to provide the
  government reviewers with adequate documentation
• Support the assessment process during the on-site
  review
• Make available key personnel to answer questions




                                                                 Source: www.pacaf.mil
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews   Page 42

1. Planning: Pre-PRR Activities
Some of the pre-meeting activities could include
 the following:

• Setting up the Agenda
• Correspondence with DCMA, Program
  Management Office (PMO) and contractor about
  the coming review
• Setting up the team composition, roles and
  responsibilities to include allocating review
  areas
• PRR training is necessary
• Setting up team work packages
• Reviewing product baseline and program
  schedules
• Reviewing historical files for insight into past
  problems
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews   Page 43

1. Planning: The PRR Plan

The PRR Plan (government or contractor) should include
 the following elements:

• Purpose
• Charter and scope
• Approach
   – Evaluation factors and criteria
   – Methodology and procedures
   – Plan for evaluating subcontractors
• Roles and Responsibilities
   – Participants and resources
• Reporting
• Disposition of findings (risks) and recommendations

Remember that PRRs of subcontractors and suppliers are
 the responsibility of the prime contractor. The
 government has no privity of contract with
 subcontractors and suppliers.
                                        Assessing Manufacturing Risks
                                         Production Readiness Reviews                     Page 44

1. Planning: The PRR Plan Inputs

Inputs to the PRR Plan could include:

•   Program Requirements (the contract, SOW, etc.)
•   Product Baseline
•   Schedules
•   Contractor Staffing
•   Manufacturing and QA Plans

Tools to help you in the planning could include:
• The PQM Community of Practice
   – Includes PRR Checklists (Navy, Air Force and OSD
      Checklists)
   – PRR Guides
• Willoughby Transition Templates


                                                                  Source: www.au.af.mil
                                      Assessing Manufacturing Risks
                                         Production Readiness Reviews                          Page 45

1. Planning: PRR Macro View

The qualification of the manufacturing process at both
 prime contractors and major subcontractors is essential.
 Properly planned, staffed, and executed PRRs are valid
 tools for assessing the depth of production engineering
 and planning activities. Subcontractors/suppliers are
 included in PRRs.

There are five macro areas that a PRR should evaluate:
• Design Maturity: System requirements have been
 validated and verified. The number and magnitude of
 engineering change is decreasing.
• Producible: The design has been maximized for ease                    Source: www.army.mil
 of manufacture.
• Manufacturing Processes: Are available, have been
 proofed and are capable and in control (statistically
 predictable).
• Facilities: Are adequate and available, as are
 personnel.
• Supply Chain: Suppliers and vendors are available,
 capable and reliable.
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews                      Page 46

2. Execution: On-Site Activities
                                                                       DAY 1
On-site activities should begin with a government
   only meeting to make sure that the program
                                                       0800 – 0815        Welcome
   office and on-site government personnel are on
   the same page. A typical agenda could include:
                                                       0815 - -830        Gov‘t Overview

1. Initial briefings:
                                                       0830 – 0900        Contractor
   - Government only to set stage with DCMA                               Overview
   - To the contractor on the purpose of visit
                                                       0900 – 0915        Breakout
   - By contractor on capabilities and status
                                                                          Overview
2. Conduct the evaluation
3. Document the findings                               0915 – 0930        Break
   - Generate Corrective Action Request or Request
       for Information                                 0930 – 1030        Facility Tour
   - Summaries of individuals review areas
   - Overall Report of Findings                        1030 – 1200        Splinter Meetings
4. Outbriefing
   1. Findings to include areas of concern and risk    1200 – 1245        Lunch
   2. Action Items
                                                       1245 - 1700        Splinter Meetings
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews                          Page 47

2. Execution: Initial Briefings

Initial briefing (Government Only):
• A smart thing to do is to conduct a government
    only meeting before you meet with the
    contractor. This meeting should include the PRR
    team and DCMA personnel as a minimum. This
    is your opportunity to baseline the entire team
    on what is going to occur and get a background
    briefing from DCMA personnel that understand
    the current problems at the plant.

Government/Contractor Kickoff Meeting:
• This meeting is intended to provide the
   government and contractor an opportunity to
   introduce themselves and state the purpose of
   the visit.
                                                           Source: Integrator.hanscom.af.mil
The contractor should provide a brief overview of the
   company with a focus on the purpose of the visit,
   your program. Keep this meeting short as you
   will find no issues in the conference room. The
   problems are on the factory floor.
                                        Assessing Manufacturing Risks
                                           Production Readiness Reviews                        Page 48

2. Execution: Breakout Sessions

After the initial briefings the team will breakout
    according to the PRR Plan into individual
    assessment areas.        During these assessments,
    team members will interview key contractor
    personnel and review all appropriate data and                      Team Leader
    documentation related to their area of
    responsibility. Breakout sessions often occur
    after the teams visit the factory floor to gather
    data or to witness specific events.                  Production      Materials
                                                           Team                           QA Team
                                                                          Team
Information gathered during this interview process
    shall form the basis for the teams risk
    assessment.

The government team should meet at the end of
   each day to go over the findings, report on any
   progress or problem areas and begin
   documenting request for action.

                                                                 Source: Integrator.hanscom.af.mil
                                         Assessing Manufacturing Risks
                                           Production Readiness Reviews                         Page 49

2. Execution: The Evaluation

The evaluation itself consists of the following items:

• Evaluation/Risk Areas, typically this includes nine functional
  assessment areas.
• Risk Definitions (Low, medium and high) are agreed to using
  structured Risk Management practices.
• Conduct of Evaluation. The evaluation itself is an
  examination that allows the team to dig deeper into
  manufacturing problem areas. When a problem is found the
  team can ask for additional information. The intent is to have
  the contractor document their progress with real evidence of
  success.

Remember the intent of the evaluation is to provide insight into
 risks and provide an opportunity to improve. It is not a
 report card. When people see the PRR as a report card then
 there is a tendency to diminish the significance of the
 findings and that does not help the program office or the
 warfighter.                                                         Source: www.dscr.dla.mil
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

2. Execution: The Evaluation (popup)

nine functional assessment areas.

Is comprised of nine (9) assessment areas:

•   Engineering and Product Design
•   Materials and Purchased Parts
•   Industrial Resources
•   Quality Assurance
•   Program Management
•   Integrated Logistics Support
•   Software Management
•   Production Engineering and Planning
•   Exit Criteria

Show the expanded sub areas
                                         Assessing Manufacturing Risks
                                           Production Readiness Reviews                        Page 50

2. Execution: Documentation of Findings

Documentation of findings can include the
   following:

1. Request for Information or Request for Action,
   are request for additional information to
   document an area of interest.
2. Summaries of individuals review areas is
   accomplished by the team. This is done at the
   end of each day and is used to keep the entire
   team appraised of the status to date.
3. The Report of Findings is the final report,
   usually in the form of a briefing that is given to
   the contractor before the team leaves the
   facility with a formal report coming from the
   contracting officer.
                                                                  Source: I made this one from an
                                                                  Air Force Photo and graphics
                                      Assessing Manufacturing Risks
                                         Production Readiness Reviews            Page 51

3. Contractor Outbrief:

There is no set formula for how to conduct the
   contractor outbriefing. In most cases you
   would want to highlight the good areas as well
   as the areas of concern. The following items           PRR Results
   may be covered at the outbriefing:                     •   Risks
                                                          •   Risk Assessments
   1. Areas of concern (may be an issue rather            •   Action Items
      than a risk)                                        •   Follow-up
   2. Risk assessments (high, medium and low)
   3. Action Items/Follow-up
                                        Assessing Manufacturing Risks
                                           Production Readiness Reviews                       Page 52

3. Outbrief: The Report to the Program Manager

The outbrief to the contractor and the report/briefing to the PM are different reports, but should
   come to the same conclusions. The final PRR is supposed to answer whether the system product
   baseline is established in a way to support hardware fabrication, that manufacturing processes
   are in place to support production and quality, that risks are known and the program schedule is
   executable. The report to the PM needs to provide the PM with the information that they need
   in order to make sound choices on future production decisions.

The official, written report should document the review in detail, to include:
• Team composition
• Areas reviewed
• Findings and recommendations.

The final report should be delivered within 30-60 days after the completion of the review and
   should not contain any major findings or recommendations that were not originally addressed or
   were addressed in follow-up meetings between the government and contractor.
                                         Assessing Manufacturing Risks
                                           Production Readiness Reviews                      Page 53

4. Follow-up and Closure:

As Yogi Berra would say ―It ain‘t over ‗til its over.‖ The
   PRR is only a look at the production program at a
   point in time. There are almost always follow-up
   activities to address the areas of concern. And new
   risk pop-up all the time. Follow-up activities could
   include some of the following:

1. Follow-up Activities on all open items
2. Folllow-up with contractor personnel and with on-
   site government personnel

Remember, the risks you identified are still there, and
 they will probably remain there until you ensure that
 they are closed out.
                                                               Source: www.navair.navy.mil
                                     Assessing Manufacturing Risks
                                        Production Readiness Reviews                       Page 54

Knowledge Review

From a macro perspective the PRR consists of four steps. Which of the following is
not one of those steps?


            A. Planning




            B. Executing



             C. Follow-up



            D. Tying to the Preliminary Desing Review (PDR)




      Answer: D. Putting the PRR in contract language is not one of the PRR steps from a
      macro perspective
                                        Assessing Manufacturing Risks
                                         Production Readiness Reviews                Page 55

Knowledge Review

From a macro view the purpose of a PRR is to determine is a program is ready for
production, to identify manufacturing related risks, and to verify that the prime
contractor its suppliers have the manufacturing ________ and ______ to perform the
work as defined in the contract.

           A. capability, capacity


            B. Executing



              C. Follow-up



            D. Put PRR in contract language



      Answer: A. capability, capacity
                                                   Assessing Manufacturing Risks
                                                      Production Readiness Reviews                                        Page 56

When to Conduct a PRR

Typically planning for a PRR begins during Engineering and Manufacturing Development and are
 conducted to support a Milestone C decision to show that the system is ready for Low Rate Initial
 Production (LRIP). PRRs are also used to support Full Rate Production (FRP) decisions.



                                                  Acquisition Framework Chart
                                                 (links Technical Reviews and Audits)

                               A                            B                               C
                  Material              Technology                   Engineering and              Production and         Ops and
                  Solution              Development             Manufacturing Development          Deployment            Support
                  Analysis         Prototyping     System
                 Material                          Design                     Post – CDR
                 Development                                                                    LRIP        FRP
                 Decision
                                                                              Assessment        IOT&E       Decision


                                                       TRA                             TRA
  Technical Reviews
  and Audits

               ITR       ASR                SRR SFR PDR                 CDR TRR      SVR OTRR PCA                  ISR
                                                                                     (FCA
                                                                                     PRR)
                                      Assessing Manufacturing Risks
                                         Production Readiness Reviews                Page 57

When to Conduct a PRR (continued)

A PRR is conducted to ensure a contractor is ready
  to transition from development to production.
  PRRs are generally scheduled to support Low-Rate
  and Full-Rate production decisions. The following
  are just a few other reasons for conducting a PRR
  (note that they revolve around risks):

• Block upgrade with design changes
• Change to a new subcontractor on a subsystem or
  component
• A new material is substituted for an existing
  material
• The company is going to do work on a second
  shift or at a new location
• Production re-start after shutdown                          Source: www.army.mil

These PRRs could be and should be tailored to
 accommodate different categories of systems or a
 specific situation or program risks.
                                       Assessing Manufacturing Risks
                                          Production Readiness Reviews                          Page 58

Types of PRRs: Large and Small

PRRs can be conducted in a number of ways:

• The first is the more traditional in-depth review and takes a large, multi-functional team. They
  perform an onsite review at the contractor‘s facility. This approach is manpower intensive, it
  may require support from outside the program office, and is disruptive to the contractor‘s
  operations. This approach could take from one to two weeks on a large program. However, the
  focused effort gains the attention of senior management and adds high visibility to the review.

• An alternative is to conduct smaller ―in-process‖ PRRs in which the elements are evaluated in a
  more informal basis by SPO POCs and their contractor counterparts. This approach is less
  manpower intensive and disruptive, but a concerted effort must be made to maintain progress,
  focus, and interest, such as by continuing to brief the status at program reviews.
                                                      Assessing Manufacturing Risks
                                                        Production Readiness Reviews                                     Page 59

Incremental PRRs

From a timeline perspective, PRRs can be conducted as a stand-alone event or can be conducted
  incrementally. For example, as a program moves through Engineering and Manufacturing
  Development, manufacturing and QA personnel can evaluate the impact of design and
  manufacturing decisions on the capability to produce the system, subsystem or component in a
  production representative environment or pilot line. As other improvements occur, then the
  program office can conducts another review. Incremental PRRs may be accomplished on significant
  portions of an overall system in an incremental manner as subsystems and processes mature.



                       A                          B                               C
          Material              Technology                 Engineering and              Production and         Ops and
          Solution              Development           Manufacturing Development          Deployment            Support
          Analysis         Prototyping   System
         Material                        Design                     Post – CDR
         Development                                                                  LRIP          FRP
         Decision
                                                                    Assessment        IOT&E         Decision

                                                             PRR     PRR    PRR       PRR     PRR
                                   Incremental PRRs
                                         Assessing Manufacturing Risks
                                          Production Readiness Reviews                     Page 60

Deciding When and Where to Conduct PRRs
PRRs can be conducted on the entire system or on elements of the system/subsystem/components.
 Deciding when and where to conduct a PRR is a matter of considering risks. One way to identify
 program risk is by looking at the program Work Breakdown Structure (WBS) to help determine
 where the risk are and then conducting PRRs on those areas. For example, if your new fighter
 program is using an existing engine and communication system, but elements of the airframe and
 radar are new, then you will want to evaluate the airframe and radar. You may elect to just
 monitor the current engine and communication production program rather than conduct a PRR on
 mature subsystems.




                                                    Aircraft



                              Airframe    Propulsion           Comm.   Nav.



                 Controls   Gear Box       Thrust          Altimeter   Radar   Computer
                                           Vector
                                     Assessing Manufacturing Risks
                                        Production Readiness Reviews                        Page 61

Deciding When and Where to Conduct PRRs: Selection Criteria

There are significant manufacturing risks to be addressed via a PRR at prime contractor locations
 even when actual manufacturing work is subcontracted out. Such risks include integration and
 assembly of components manufactured by different suppliers, interfaces and interoperability,
 scheduling of deliveries from different suppliers to meet overall production rates, quality
 management/oversight, configuration control/change management practices, and so forth.
 Deciding what subsystems and components require a PRR is a challenge. Creating a PRR Selection
 Matrix as seen below is one way to assess risk by looking at a number of factors.




   Component      Supplier
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews

PRR Selection Criteria (popups)

Mission Critical: The program or item is a key component in an essential subsystem, without which
 the weapon system would not be able to complete its assigned mission.

Reliability and Maintainability Critical: The program or item is a key component that has not
 matured to a specified R&M value during its development phase.

New Technology: The program or item contains subsystems or components that are advancing the
 state-of-art and have not yet been fully proven on systems in production.

Hardware Complexity: The program or item contains subsystems or components that have
 volume, weight, or other parameters which provide detailed design and development constraints
 with tight tolerances which are technical challenges with few alternative choices of action.
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews

PRR Selection Criteria (popups)

Cost Critical: The cost of the program or item being developed has high dollar value so that
 perturbations could cause breeches of cost thresholds.

Schedule Critical: The program or item being developed is of such importance to the weapon
 system and program that late deliveries could cause severe breaches of schedule thresholds.

Support Critical: The program or item under development, once introduced into the fleet will have
 a significant impact on the weapon system logistic support requirement.

Past Performance: The contractors past experience in development or production on the program
 or item (or similar program or item) and track record for delivering within established cost,
 schedule and performance requirements, could have an adverse impact on the success of the
 program.

TBD: This is criteria you develop based on your programs needs and objectives.
                                         Assessing Manufacturing Risks
                                           Production Readiness Reviews                          Page 62

PRRs: When There is Change

Manufacturing managers need to adapt to frequently changing program requirements (e.g., changes
 in production quantities, delivery schedules, production rates, program budgets, and mission
 requirements.

For example, until late in 2003, the JSF program‘s SDD phase was scheduled to run until around
  2008, at which time full rate production was scheduled to begin, with a projected initial operational
  capability around 2010. To address growing weight problems encountered in the development
  phase, DOD extended the SDD phase one year, and correspondingly delayed the F-35‘s scheduled
  first flight from late 2005 to the summer of 2006 (first flight occurred on December 15, 2006, and
  the beginning of low-rate initial production shifted from 2006 to 2007. Procurement profiles in the
  Future Years Defense Plan (FYDP) are as follows:


                         FY08       FY09       FY10       FY11       FY12       FY13

              USAF       6          8          12         24         42         48

              USN        6          8          18         19         40         42


                                Source: CRS Report for Congress dated 19 July, 2007
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews                     Page 63

Production Readiness Review (PRR) Entry Criteria

The PRR requires that PRR technical documentation
 is made available to the PRR participants prior to
 the review. These documents could include the
 following:

• Results of the previous PRRs conducted at the
  major suppliers' facilities
• Transition to Production or Manufacturing Plan
• Production configuration baseline approved by the
  customer
• Change control process/documentation
• Manufacturing/producibility and quality
  requirements addressed during the
  design/development phase
• Current risk assessment
                                                           Source: www.stsc.hill.af.mil
                                       Assessing Manufacturing Risks
                                          Production Readiness Reviews   Page 64

Typical PRR Exit Criteria (DAG 4.3.3.9.3)

Whether your doing a typical PRR or a modified
 PRR, success criteria should include affirmative
 answers to the following exit questions:

(1) Has the system product baseline been
  established and documented to enable hardware
  fabrication and software coding to proceed with
  proper configuration management?
(2) Are adequate processes and metrics in place
  for the program to succeed?
(3) Are the risks known and manageable?
(4) Is the program schedule executable
  (technical/cost risks)?
(5) Is the program properly staffed?
(6) Is the detailed design producible within the
    production budget?
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews                              Page 65

PRR Risk Definitions

The answers to the previous questions can be evaluated as low, medium or high risk based on
 the following risk criteria:

Low Risk: Manufacturing operation planning has been initiated as is progressing in compliance
  with internal work schedules. Areas of concern involve minor issues or management systems
  refinements needed to enhance visibility and control. Lack of management attention to these
  concerns could cause deterioration of the risk assessment of the review area to one of medium
  risk. Processes commonly used in the commercial sector. Processes are capable and highly
  predictable. (Blue indicates a pop-up)

Medium Risk: Manufacturing operations planning is not progressing in compliance with internal
 work schedule or not yet initiated causing uncertainty. Areas of concern indicate situation or
 condition that could cause deterioration of the review area to one of serious risk if allowed to
 exist without corrective action.

High Risk: Adverse conditions exist that would be highly disruptive to conducting a successful
  production program. Areas of concern identify those risk factors requiring substantial resources
  and management attention to correct and assure a successful production program. Requires
  development of previously unused processes and/or the use of new materials. Materials and
  processes are not characterized. Process outputs are unstable and incapable.
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews                      Page 66

Knowledge Review

A PRR is conducted to ensure that a contractor is ready to transition from ____ into
the ___________.


           A. component development, Technology Development Phase




            B. system development, EMDD Phase



             C. Engineering and Manufacturing Development, Operations and Support Phase



            D. development, Production and Deployment Phase



      Answer:   D. development, Production and Deployment Phase
                                       Assessing Manufacturing Risks
                                        Production Readiness Reviews   Page 67

Knowledge Review

PRRs are normally conducted to support a ___________.



          A. Milestone A decision




           B. Milestone B decision



            C. Milestone C decision



           D. Milestone D descision



     Answer: C. Milestone C decision
                                           Assessing Manufacturing Risks
                                             Production Readiness Reviews              Page 68

Vignette
                                                               Baron Heinrich von Stein,
Do you know where some of the early concepts for
 controls and standard procedures came from?

The model for Henry Ford‘s administrative system
 came largely from rules set up by Prussian
 bureaucracy during the 19th century. These
 administrative innovations included:

•    the centralized material and logistics planning,
•   control by rules,
•   standard operating procedures, and
•   the merit principle.

Baron Heinrich von Stein‘s procedures allowed the
 Prussians to outperform other countries. The OSD
 and other PRR checklists that have been
 developed provide standardized processes and
 controls for accomplishing PRRs.
                                     Assessing Manufacturing Risks
                                        Production Readiness Reviews               Page 69

Learning Objective

Describe the Production Readiness Review process
 how, when and where it might be used.

Enabling Learning Objectives:

• Identify the role and goal of manufacturing and
  how risks can affect achieving that goal
• Describe a PRR and the guidance associated with
  PRRs
• Identify how and when a PRR will be conducted
  from a macro perspective
• Describe a PRR from a micro perspective




                                                             JSF Production Line
                                                             Source: navy.mil
                                     Assessing Manufacturing Risks
                                       Production Readiness Reviews   Page 70

PRR Micro View

The micro view takes a deep dive within the
 evaluation process. The OSD PRR evaluation is an
 assessment of nine areas. These nine assessment
 areas include:

•   Engineering and Product Design
•   Materials and Purchased Parts
•   Industrial Resources
•   Quality Assurance
•   Program Management
•   Integrated Logistics Support
•   Software Management
•   Production Engineering and Planning
•   Exit Criteria

Select an assessment area above to see some sample
 questions. For access to the OSD PRR Checklist go
 to the following url

https://acc.dau.mil/CommunityBrowser.aspx?id=1573
  91&lang=en-US
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

Sample Questions (1st popup) Engineering and Product Design focus on the extent to
                                           which the design is producible and affordable. It determine
By Area:                                   the effectiveness of contractor approaches toward
                                           integrating manufacturing issues into the design and
•   Engineering and Product Design         development process. Below are some typical questions:
•   Materials and Purchased Parts
                                           • Are the contractor‘s engineering drawings and documents
•   Industrial Resources
                                           complete?
•   Quality Assurance                      • Have the physical, functional, and performance
•   Program Management                     characteristics for all parts and assemblies been described?
•   Integrated Logistics Support           • Does the manufacturing organization meet with design
•   Software Management                    engineers to discuss the latest developments in
•   Production Engineering and Planning    manufacturing technology?
•   Exit Criteria                          • Does the manufacturing screening program use
                                           Environmental Stress Screening (ESS) to eliminate latent
                                           defects and workmanship problems?
                                           • Were design modifications considered where required
                                           material properties would indicate the need for high cost,
                                           limited quantity exotic materials
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

Sample Questions (2nd Popup) Materials and Purchased Parts focuses on the
                                            effectiveness of contractor production planning activities
By Area:                                    relative to materials such as: make-or-buy decisions;
                                            master and subsidiary schedules; critical path analysis;
•   Engineering and Product Design          parts shortages and backlogs; visibility into
•   Materials and Purchased Parts           subcontractor production or delivery problems; material
                                            accountability; material routing and material handling.
•   Industrial Resources
                                            Below are some typical questions:
•   Quality Assurance
•   Program Management                      • What is the involvement of all appropriate functional
•   Integrated Logistics Support            areas such as engineering, manufacturing, and quality
•   Software Management                     assurance in the Make/Buy decision?
•   Production Engineering and Planning     • How has plant capacity affected the make / buy
•   Exit Criteria                           decision?
                                            • What are the procedures for the review of designs,
                                            drawings, and specifications of subcontracted items such
                                            as hardware, computer software, technical data, and
                                            identified high-risk subsystems?
                                            • How is it assured that the subcontractor quality system
                                            is in compliance with the prime contract quality
                                            assurance program requirements?
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

Sample Questions (3rd popup)

By Area:                                    Industrial Resources focuses on ensuring that there is a
                                            healthy, cost effective technology and industrial base that
                                            is around to support and be responsive to the needs of
•   Engineering and Product Design          the DoD. Below are some typical questions:
•   Materials and Purchased Parts
•   Industrial Resources                    • What facilities constraints are identified in program
•   Quality Assurance                       documentation and does the preferred system concept
•   Program Management                      address these constraints?
•   Integrated Logistics Support            • How have the facilities and equipment required for this
                                            program been factored into the program schedule?
•   Software Management
                                            • How has the plant layout been coordinated with the
•   Production Engineering and Planning     various internal disciplines such as methods and
•   Exit Criteria                           processes, quality assurance, production control, and
                                            manpower?
                                            • What is the contingency plan for adaptation to the plant
                                            layout for revised production rates?
                                            • Has tooling which is considered high risk been identified
                                            and, what plans have been made to reduce the risk?
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

Sample Questions (4th poopup)

By Area:                                    Quality Assurance focuses on policy, process,
                                            procedures and organizational structure needed to
                                            implement a fully functional quality assurance program
•   Engineering and Product Design          have been implemented and are achieving the desired
•   Materials and Purchased Parts           results. Below are some typical questions:
•   Industrial Resources
•   Quality Assurance                       • What is the process for performing internal audits and
•   Program Management                      initiating corrective actions?
•   Integrated Logistics Support            • What is the quality organization's role in the
                                            preparation, classification, and distribution of waivers,
•   Software Management
                                            deviations, and engineering changes?
•   Production Engineering and Planning     • Are quality records analyzed and used as a basis for
•   Exit Criteria                           management action to correct product and system
                                            deficiencies?
                                            • How does the quality program provide for lines of
                                            communication among all functional organizations?
                                            • How is the quality assurance planning for inspection and
                                            testing compatible with manufacturing methods and
                                            processes?
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

Sample Questions (5th popup)

By Area:                                    Program Management focuses on overall program
                                            management functions and operations as they relate to
•   Engineering and Product Design          production of the system/subsystem or component.
•   Materials and Purchased Parts           Below are some typical questions:
•   Industrial Resources
                                            • Do procedures exist that define the degree of control
•   Quality Assurance                       the program manager has over budgeting, financial
•   Program Management                      commitments, and allocation of resources within the
•   Integrated Logistics Support            company?
•   Software Management                     • Are organizational responsibilities effectively identified
•   Production Engineering and Planning     in order to minimize any duplication of effort?
                                            • Have key personnel assignments been planned to
•   Exit Criteria
                                            assure availability and experience for the transition from
                                            SDD to the production phase?
                                            • Does the cost methodology documentation completely
                                            define the total production costs for each component,
                                            assembly, or appropriate activity for the system?
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

Sample Questions (6th popup)

By Area:                                    Integrated Logistics Support focuses on the ability of
                                            the ILS functions to impact design and production
                                            decisions in a way to reduce Total Life Cycle cost and
•   Engineering and Product Design          positively impact the ―ilities.‖ Below are some typical
•   Materials and Purchased Parts           questions:
•   Industrial Resources
•   Quality Assurance                       • Does the program acquisition strategy include full life-
•   Program Management                      cycle support planning and address actions to assure
•   Integrated Logistics Support            sustainment and continuous improvement/affordability?
                                            • Who composes the Integrated Logistics Support
•   Software Management
                                            Management Team (ILSMT), and how does the team
•   Production Engineering and Planning     working as a team focused on affordable readiness?
•   Exit Criteria                           • Has a Failure Reporting, Analysis, and Corrective Action
                                            System (FRACAS) been initiated?
                                            • Have Performance Based Logistics (PBL) concepts been
                                            incorporated as the preferred supply support strategy?
                                            • Have trade studies been conducted to identify
                                            supportability risks, if new or emerging technologies are
                                            being considered?
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews

Sample Questions (7th popup)

By Area:                                     Software Management focuses on the ability of
                                             software engineers and QA specialists to positively
                                             impact the software development process and integrate
•   Engineering and Product Design           the software program with the production program.
•   Materials and Purchased Parts            Below are some typical questions:
•   Industrial Resources
•   Quality Assurance                        •Is all system computer software listed as configuration
•   Program Management                       items, and how are other items identified and
                                             controlled?
•   Integrated Logistics Support
                                             • Are the requirements and design documents updated
•   Software Management
                                             as a result of Class I and Class II ECP changes?
•   Production Engineering and Planning      • Have hardware / software problem areas been
•   Exit Criteria                            identified as a result of system analyses?
                                             • Is there documentation to show the degree of
                                             computer software testing that is to be performed?
                                   Assessing Manufacturing Risks
                                    Production Readiness Reviews

Sample Questions (8th popup)

By Area:                             Production Engineering and Planning focuses on the
                                     producibility of the design in terms of early identification
                                     and trade-offs concerning special materials, special
• Engineering and Product Design     processes, unique handling/inspection requirements;
• Materials and Purchased Parts      evaluation of special tooling, test equipment, fixtures,
• Industrial Resources               gages, and jig requirements; precision tolerance
• Quality Assurance                  requirements or unique manufacturing requirements;
• Program Management                 use of CAD/CAM and Computer-Aided Engineering
• Integrated Logistics Support       (CAE).
• Software Management
                                     • What proven manufacturing processes are being used
• Production Engineering and         where possible with trade studies performed to justify
  Planning                           the use of new technology?
• Exit Criteria                      • What producibility analyses requirements have been
                                     established?
                                     • What are the training programs for new and current
                                     employees?
                                     • What is the process for breaking down the production
                                     process into discrete work packages?
                                     • What are the detailed schedules for fabrication,
                                     assembly, and installation, and how do they relate to the
                                     master production schedule?
                                      Assessing Manufacturing Risks
                                       Production Readiness Reviews

Sample Questions (9th popup)

By Area:                                Exit Criteria: The PRR is considered complete when all
                                        draft Request for Action (RFAs) are signed off, and an
                                        acceptable level of risk is ascertained. The program
•   Engineering and Product Design      manager will approve entering LRIP or FRP based upon
•   Materials and Purchased Parts       acceptable PRR results and manageable program risk.
•   Industrial Resources                Below are some typical questions:
•   Quality Assurance
•   Program Management                  • Has the system product baseline been established and
•   Integrated Logistics Support        documented?
                                        • Are adequate processes and metrics in place for the
•   Software Management
                                        program to succeed?
•   Production Support and Planning     • Is the program schedule executable (technical/cost
•   Exit Criteria                       risks)?
                                        • Is the detailed design producible within the production
.                                       budget?
                                        • Were all PRR RFAs properly completed (closed)?
                                        • Can the system produced satisfy the CPD?
                                        Assessing Manufacturing Risks
                                           Production Readiness Reviews                    Page 71

OSD PRR Checklist

Additionally the checklist is sorted into eleven (11)
 categories that are involved in the review and by
 selecting a button you can sort the questions by
 that function, or you can hide functions and see
 only a specific category area:

•   Programmatic
•   Earned Value Management                     Programmatic     PQM     EVM      Interoperability
•   Interoperability
                                                Technology   Software  Risk    Logistics
•   Technology
•   Software
                                                Training    T&E    HSI
•   Risk
•   Logistics
•   Training
•   Test and Evaluation
•   Human Systems Integration
•   PQM – Production, Quality and Manufacturing
                                         Assessing Manufacturing Risks
                                          Production Readiness Reviews                   Page 72

PRR Execution: Conduct of the Evaluation
Typically a PRR team comes on-site with the team members given work assignments or evaluation
 areas. The team members could include functional experts in manufacturing, quality assurance
 or other functional area, and could include product specialists such as materials engineers,
 avionics engineers, or other product line specialists. The team members need to be able to
 address the nine assessment areas using expertise from the eleven technical disciplines.


                             Name                         Area

                        George (lead)      Engineering and Product Design
                                           Materials and Purchased Parts

                        Tom (lead)         Industrial Resources
                                           Quality Assurance


                        Shirley (lead)     Program Management
                                           Integrated Logistics Support


                        Jorge (lead)       Software Management
                                           Production Engineering and Planning
                                           Exit Criteria
                                      Assessing Manufacturing Risks
                                       Production Readiness Reviews   Page 73

Knowledge Review

Which of the following is not one of the nine PRR assessment areas?



           A. Engineering and Product Design




            B. Industrial Resources



             C. Software Management



             D. Logistics




      Answer: D. Logistics
                                      Assessing Manufacturing Risks
                                        Production Readiness Reviews   Page 74

Knowledge Review

Which of the following is not one of the PRR review categories?



           A. Earned Value Management




            B. Logistics



             C. Human Systems Integration



             D. Contracting




      Answer: D. Contracting
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews                   Page 75

Summary Page 1

Enabling Learning Objectives:

• Identify the role and goal of
  manufacturing and how risks can affect           Measurement   Materials   Machines
  achieving the goals

The Role of Manufacturing is to:
• influence the design
• prepare for production, and                           Design                          Product
• Execute the manufacturing plan.

The goal of manufacturing is to provide
 uniform, defect-free product that achieves
 consistent performance and at a lower cost.
 Achieving that goal is difficult because of the            Methods     Manpower
 many manufacturing risks. One way to look
 at manufacturing risk is to use the fishbone
 diagram below that depicts the Design process
 and the 5Ms. Each of these is a potential
 source of risk.
                                        Assessing Manufacturing Risks
                                          Production Readiness Reviews                          Page 76
Summary Page 2

Enabling Learning Objectives:

• Describe a PRR and the guidance associated with PRRs

PRRS are manufacturing reviews that are used to identify risks, issues and opportunities. PRRs assess
   the capability of the manufacturing process to deliver on time and within cost.

The PRR examines a program to determine if the design is ready for production, the production
 planning is adequate, and if production may incur unacceptable risks

The review evaluates the full, production-configured system to determine if:

• it correctly and completely implements all system requirements
• traceability of final system requirements to the final production system is maintained
• have a production capability for proceeding into Low-Rate Initial Production or Full-Rate Production
                                       Assessing Manufacturing Risks
                                         Production Readiness Reviews                         Page 77

Summary Page 3

Enabling Learning Objectives:

• Identify when and how a PRR will be conducted from a macro perspective

Typically planning for a PRR begins during Engineering and Manufacturing Development and continues
   through Production and Deployment. The PRR is conducted about the same time as the System
   Verification Review (Functional Configuration Audit). PRRs often occur just prior to and support a
   Milestone C decision and can be used to show that the system is ready for Low Rate Initial
   Production (LRIP) and Full Rate Production.
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews                      Page 78

Summary Page 3 Continued

Enabling Learning Objectives:

• Identify when and how a PRR will be conducted from a macro perspective

The macro view is a top level view and at that point the PRR includes the following steps:

1.   Planning
2.   Execution (conduct of the PRR)
3.   Outbrief
4.   Follow-up and Closure

The PRR examines nine functional areas.

•   Engineering and Product Design
•   Materials and Purchased Parts
•   Industrial Resources
•   Quality Assurance
•   Program Management
•   Integrated Logistics Support
•   Software Management
•   Production Engineering and Planning
•   Exit Criteria
                                          Assessing Manufacturing Risks
                                           Production Readiness Reviews                      Page 79

Summary Page 4

Enabling Learning Objectives:

• Describe a PRR from a micro perspective

The micro view takes a deep dive within the evaluation process. It is an in-depth evaluation of nine
 areas. Some of the questions are higher level questions and others are at the working level. These
 nine assessment areas include:

•   Engineering and Product Design
•   Materials and Purchased Parts
•   Industrial Resources
•   Quality Assurance
•   Program Management
•   Integrated Logistics Support
•   Software Management
•   Production Engineering and Planning
•   Exit Criteria

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:6
posted:8/24/2011
language:English
pages:92