Docstoc

six-sigma-in-measurement-systems-evaluating-the-hidden-factory

Document Sample
six-sigma-in-measurement-systems-evaluating-the-hidden-factory Powered By Docstoc
					Six Sigma in Measurement Systems:
   Evaluating the Hidden Factory
                                            OK
   Inputs   Operation             Inspect        First Time
                                                  Correct
                                       NOT
             Rework                    OK
            Hidden Factory
               Scrap

                                   Time, cost, people



              Bill Rodebaugh
             Director, Six Sigma
                       GRACE

                             slide 1
                          Objectives
   The Hidden Factory Concept
       What is a Hidden Factory?
       What is a Measurement System’s Role in the Hidden
        Factory?
   Review Key Measurement System metrics including
    %GR&R and P/T ratio
   Case Study at W. R. GRACE
       Measurement Study Set-up and Minitab Analysis
       Linkage to Process
       Benefits of an Improved Measurement System
   How to Improve Measurement Systems in an
    Organization

                                slide 2
The Hidden Factory -- Process/Production
                                                     OK
     Inputs          Operation             Inspect          First Time
                                                             Correct
                                                 NOT
                       Rework                    OK
                     Hidden Factory
                        Scrap

                                            Time, cost, people

•What Comprises the Hidden Factory in a Process/Production Area?
    •Reprocessed and Scrap materials -- First time out of spec, not reworkable
    •Over-processed materials -- Run higher than target with higher
    than needed utilities or reagents
    •Over-analyzed materials -- High Capability, but multiple in-process
    samples are run, improper SPC leading to over-control


                                      slide 3
The Hidden Factory -- Measurement Systems
                                                        OK
       Sample          Lab Work              Inspect          Production
       Inputs
                                                    NOT
                        Re-test                     OK
                      Hidden Factory
                         Waste
                                                  Time, cost, people

 •What Comprises the Hidden Factory in a Laboratory Setting?
     •Incapable Measurement Systems -- purchased, but are unusable
     due to high repeatability variation and poor discrimination
     •Repetitive Analysis -- Test that runs with repeats to improve known
     variation or to unsuccessfully deal with overwhelming sampling issues
     •Laboratory “Noise” Issues -- Lab Tech to Lab Tech Variation, Shift to
     Shift Variation, Machine to Machine Variation, Lab to Lab Variation


                                        slide 4
           The Hidden Factory Linkage
   Production Environments generally rely upon in-
    process sampling for adjustment
   As Processes attain Six Sigma performance they begin
    to rely less on sampling and more upon leveraging the
    few influential X variables
   The few influential X variables are determined largely
    through multi-vari studies and Design of
    Experimentation (DOE)
   Good multi-vari and DOE results are based upon
    acceptable measurement analysis


                            slide 5
                          Objectives
   The Hidden Factory Concept
       What is a Hidden Factory?
       What is a Measurement System’s Role in the Hidden
        Factory?
   Review Key Measurement System metrics including
    %GR&R and P/T ratio
   Case Study at W. R. GRACE
       Measurement Study Set-up and Minitab Analysis
       Linkage to Process
       Benefits of an Improved Measurement System
   How to Improve Measurement Systems in an
    Organization

                                slide 6
       Possible Sources of Process Variation
                                       Observed Process Variation



                    Actual Process Variation                               Measurement Variation



   Long-term              Short-term            Variation           Variation due            Variation due
Process Variation      Process Variation       w/i sample              to gage               to operators



                                           Repeatability     Calibration         Stability        Linearity




      2 Observed Pr ocess   2 Actua l Pr ocess   2 Measuremen t System
          2 Measuremen t System   2 Re peatabilit y   2 Re producibility
        We will look at “repeatability” and “reproducibility” as primary
                      contributors to measurement error


                                                   slide 7
      How Does Measurement Error Appear?

Actual process variation -               15
                                                   LSL                                   USL
No measurement error




                             Frequency
                                         10




                                         5




                                         0

                                              30   40    50   60      70       80   90   100   110
                                                                   Process

                                         15

Observed process                                   LSL                                   USL
variation -                  Frequency
                                         10

With measurement error
                                          5




                                          0

                                              30    40   50   60      70       80   90   100   110
                                                                   Observ ed


                                  slide 8
                Measurement System Terminology
   Discrimination - Smallest detectable increment between two measured values
   Accuracy related terms
      True value - Theoretically correct value
      Bias - Difference between the average value of all measurements of a sample and the
        true value for that sample
   Precision related terms
      Repeatability - Variability inherent in the measurement system under constant
        conditions
      Reproducibility - Variability among measurements made under different conditions
        (e.g. different operators, measuring devices, etc.)
   Stability - distribution of measurements that remains constant and predictable over time for
    both the mean and standard deviation
   Linearity - A measure of any change in accuracy or precision over the range of instrument
    capability




                                               slide 9
             Measurement Capability Index - P/T
       Precision to Tolerance Ratio

                         515 * MS
                          .
                   P/T                                      Usually expressed
                                                                as percent
                         Tolerance
       Addresses what percent of the tolerance is taken up by
        measurement error
       Includes both repeatability and reproducibility
            Operator x Unit x Trial experiment
       Best case: 10% Acceptable: 30%

Note: 5.15 standard deviations accounts for 99% of Measurement System (MS) variation.
     The use of 5.15 is an industry standard.


                                                  slide 10
    Measurement Capability Index - % GR&R

                         MS
%R & R                                      x 100
            Observed   Pr ocess Variation
                                                         Usually expressed
                                                            as percent



   Addresses what percent of the Observed Process Variation is
    taken up by measurement error
   %R&R is the best estimate of the effect of measurement
    systems on the validity of process improvement studies (DOE)
   Includes both repeatability and reproducibility
   As a target, look for %R&R < 30%




                                              slide 11
                          Objectives
   The Hidden Factory Concept
       What is a Hidden Factory?
       What is a Measurement System’s Role in the Hidden
        Factory?
   Review Key Measurement System metrics including
    %GR&R and P/T ratio
   Case Study at W. R. GRACE
       Measurement Study Set-up and Minitab Analysis
       Linkage to Process
       Benefits of an Improved Measurement System
   How to Improve Measurement Systems in an
    Organization

                               slide 12
                            Case Study Background
   Internal Raw Material, A1, is necessary for Final Product production
        Expensive Raw Material to produce – produced at 4 locations Worldwide
        Cost savings can be derived directly from improved product quality, CpKs
        Internal specifications indirectly linked to financial targets for production costs are used to
         calculate CpKs
        If CTQ1 of A1 is too low, then more A1 material is added to achieve overall quality – higher
         quality means less quantity is needed – this is the project objective
   High Impact Six Sigma project was chartered to improve an important quality variable,
    CTQ1
   The measurement of CTQ1 was originally not questioned, but the team decided to study
    the effectiveness of this measurement
        The %GR&R, P/T ratio, and Bias were studied
        Each of the Worldwide locations were involved in the study
   Initial project improvements have somewhat equalized performance across sites. Small
    level improvements are masked by the measurement effectiveness of CTQ1




                                                  slide 13
            CTQ1 MSA Study Design (Crossed)


         Site 1 Lab                 Site 2 Lab        Site 3 Lab     Site 4 Lab




  Site 1 Sample 1 Site 1 Sample 2     Site 2 Sample 1…..


Op 1 Op 2 Op 3                             6 analyses/site/sample
                                           2 samples taken from each site
T1 T2                                      2*4 Samples should be representative
                                           Each site analyzes other site’s sample.
                                           Each plant does 48 analyses
                                           6*8*4=196 analyses
                                           slide 14
                                                                                                                     Gage name:           Z-14 MSA

Surface Area   CTQ1 MSA Study Results (Minitab Output)                                                               Date of study:
                                                                                                                     Reported by:
                                                                                                                     Tolerance:
                                                                                                                                          JULY 2002
                                                                                                                                          All Labs
                                                                                                                                          110
                                                                                                                     Misc:

                               Components of Variation                                                                            Response By Sample
               120                                                                                             890
                                                                                    %Contribution
               100
                                                                                    %Study Var
               80                                                                                              840
Percent




                                                                                    %Tolerance
               60
               40                                                                                              790

               20
                                                                                                               740
                0
                         Gage R&R     Repeat        Reprod   Part-to-Part                                Sample        1      2       3   4     5     6       7       8

                                    R Chart by Operator                                                                           Response By Operator
               100   CB1 CB2 CB3 LC1 LC2 LC3   V1    V2   V3 W1   W2   W3                                      890
Sample Range




                                                                                                               840
               50                                                           UCL=52.45

                                                                                                               790
                                                                            R=16.05
                0                                                           LCL=0                              740
                     0                                                                                        Oper   CB1 CB2 CB3 LC1 LC2 LC3 V1 V2 V3 W1 W2 W3

                                Xbar Chart by Operator                                                                      Operator*Sample Interaction
               900   CB1 CB2 CB3 LC1 LC2 LC3   V1    V2   V3 W1   W2   W3                                      900                                                            Operator
                                                                                                                                                                                  CB1
Sample Mean




                                                                                                                                                                                  CB2
               850                                                          UCL=851.5               Average    850                                                                CB3
                                                                            Mean=821.3                                                                                            LC1
               800                                                                                                                                                                LC2
                                                                            LCL=791.1                          800
                                                                                                                                                                                  LC3
                                                                                                                                                                                  V1
               750
                                                                                                                                                                                  V2
                     0                                                                                   Sample        1      2       3    4     5        6       7       8
                                                                                                                                                                                  V3
                                                                                                                                                                                  W1
                                                                                                                                                                                  W2


                                                                                     slide 15
 CTQ1 MSA Study Results (Minitab Session)
Source            DF       SS        MS          F            P
Sample                 7    14221    2031.62         5.0079   0.00010
Operator            11      53474    4861.27     11.9829      0.00000
Operator*Sample     77      31238     405.68         1.4907   0.03177
Repeatability       96      26125     272.14
Total             191      125058
                                     %Contribution
Source                     VarComp   (of VarComp)
Total Gage R&R             617.39     90.11             Sample, Operator,
                                                        & Interaction are
  Repeatability            272.14     39.72
                                                        Significant
  Reproducibility          345.25     50.39
    Operator               278.47     40.65
    Operator*Sample         66.77         9.75
Part-To-Part                67.75         9.89



                                     slide 16
                   CTQ1 MSA Study Results
                                                                        Mean
                           P/T                      Equal Variances Differences
Site       %GRR                      R-bar
                          Ratio                      within Groups
                                                                    (Tukey Comp.)
             94.3
 All                        116      16.05             No (0.004)    Only 1,2 No Diff.
         (78.6 – 100)*
             38.9
Site 1                      29        7.22             Yes (0.739)   All Pairs No Diff.
         (30.0 – 47.6)
             91.0
Site 2                      96       17.92             Yes (0.735)    Only 1,2 Diff.
         (70.7 – 100)
             80.0
Site 3                      79       20.37             Yes (0.158)   All Pairs No Diff.
         (60.8 – 94.8)
             98.0
Site 4                      120      18.67             Yes (0.346)   Only 2,3 No Diff.
         (64.8 – 100)
                         *Conf Int not calculated with Minitab, Based upon R&R Std Dev
                                         slide 17
           Dotplots of C16 by (Minitab Output)
CTQ1 MSA Study Results C17
                         (group means are indicated by lines)
             Dotplot of All Samples over All Sites
       890




       840
C16




       790




       740




                                                                 WO SA
                                                        VF SA
                 CB SA




      C17
                                  LC SA




              Site 1           Site 2                Site 3     Site 4


                                          slide 18
   CTQ1 MSA Study Results (Minitab Session)
Analysis of Variance for Site
Source     DF        SS          MS                F          P
Site        3     37514    12505         26.86          0.000
Error     188     87518         466
Total     191    125032
                                      Individual 95% CIs For Mean
                                      Based on Pooled StDev
Level       N      Mean    StDev      -+---------+---------+---------+-----
Site 1     48    824.57    15.38                                  (---*---)
Site 2     48    819.42    22.11                             (---*---)
Site 3     48    800.98    20.75       (---*---)
Site 4     48    840.13    26.58                                              (---*---)
                                      -+---------+---------+---------+-----
Pooled StDev =    21.58               795              810         825         840


                   Site and Operator are closely related

                                        slide 19
Per
                40                                                                                           790


               CTQ1 MSA Study Results (Minitab Output)
                20
                                                  740
                0
                         Gage R&R   Repeat          Reprod       Part-to-Part                          Sample        1    2
                                 R of All Samples for All Sites
                           X-barChart by Operator
                              R
               100                                                                                           890
                                                                                               Discrimination
                     CB1 CB2 CB3 LC1 LC2 LC3   V1    V2   V3 W1         W2   W3
Sample Range




                                                                                                 Index is “0”,
                                                                                                      840
                                                                                  UCL=52.45
                50
                                                                                                however can
                                                                                                      790
                                                                                  R=16.05       probably see
                0                                                                 LCL=0
                                                                                               differences of 5
                                                                                                      740
                     0                                                                                      Oper   CB1 CB2 C

                                Xbar Chart by Operator                                                                   O
               900   CB1 CB2 CB3 LC1 LC2 LC3   V1    V2   V3 W1         W2   W3                              900
Sample Mean




               850                                                                UCL=851.5      Most of the




                                                                                                  Average
                                                                                                     850
                                                                                  Mean=821.3     samples are
               800
                                                                                  LCL=791.1    seen as “noise”
                                                                                                     800

               750
                     0                                                                                 Sample        1



                                                             slide 20
  Pe
                      50


CTQ10 MSA Study Results (Minitab Output)
                                 Gage R&R         Repeat          Reprod   Part-to-Part

                                     R Chart by WO for
                            X-bar R of All SamplesOP Site 4
                      70                     W1                    W2              W3
                      60                                                                  UCL=60.99
       Sample Range

                      50
                      40
                      30
                      20                                                                  R=18.67
                      10
                       0                                                                  LCL=0
                             0

                                            Xbar Chart by WO OP
                      900                    W1                    W2              W3

                                                                                          UCL=875.2
  Sample Mean




                      850
                                                                                          Mean=840.1


                      800                                                                 LCL=805.0



                             •Mean differences are seen in X-bar area
                             0



                            •Most of the samples are seen as “noise”
                                                       slide 21
Sampl
                      CTQ1 MSA Study Results – Process Linkage
                                           R=17.92


                                 Site 2 Example
                      0                    LCL=0       760
                        0                            LC OP LC1

                                                Xbar Chart by LC OP                                                                                          LC OP*Sa
                     860                        LC1                             LC2              LC3                                              850
                     850                                                                                       UCL=853.1
                                                                                                                                                  840
                                                                                                                                                       MSA Study
Sample Mean




                     840
                                                                                                                                                  830




                                                                                                                                        Average
                     830
                     820                                                                                       Mean=819.4                         820 Results with
                     810                                                                                                                          810
                                                                                                                                                  800 Mean = 819.4
                     800
                     790
                                                                                                               LCL=785.7                          790
                     780
                            0                        I and MR Chart for TSA (t)                                                              Sample      1   2   3


                     1000
                                                           1
                                                                       11                                                                           2002 Historical
  Individual Value




                                1       1             1
                                1                                  1

                                                                                                                                                       Process
                      900                                                                          2                 5
                                                                                                                     5            UCL=899.2
                                                                                                  2     6
                                                                                                       6 6
                                                               2            6
                                                                                                  2
                                                                                                  2    62 62
                                                      4                                                  2 2
                                                                                                           2                      Mean=832.5
                      800
                                    6       6
                                                 22
                                                 2 4
                                                 6
                                                                                                                 2
                                                                                                                                  LCL=765.8
                                                                                                                                                     Results with
                                                                   5

                      700
                                                 1
                                                                                      1
                                                                                          1
                                                                                                                                                     Mean = 832.5
      Subgroup              0                        100                          200                  300                  400



                                                      Selected Samples are Representative
                      150
                                                         1 1
                                                       1
                                1       1             11         1
        g Range




                                                                   11
                      100                             1            1
                                                                        1
                                                                                                                                  UCL=81.95
                                                                                              slide 22
             Perc
                                    CTQ1 MSA Study Results – Process Linkage
                               50                                                                                                                             810

                                          I and MR Chart for TSA (t)
                                0                    Site 2 Example                                                                                           760
                                        Gage R&R                Repeat                 Reprod          Part-to-Part                                     Sample             1       2        3        4         5   6   7   8
                            1000
                                                                               1
                                                          R Chart by LC1OP                                                                                                                          By LC OP
Individual Value




                                            1           1             1   1
                              100           1           LC1                            1LC2                     LC3
                             900                                                                                                                             5860
                                                                                                                                                             5                 UCL=899.2
                                                                                                                                                                                       MSA Study Results
                                                                                                                                       2    6
                                                                                                   6                                  2    6 6
                                                                                                                                           62 6
             Sample Range




                                                                                   2                                                  2
                                                                                                                                      2        2
                                                                          4                                                                  2 2
                                                                                                                                               2                               Mean=832.5
                                                                                                                                                                                       with Range = 17.92,
                                                                     22                                                     UCL=58.54
                             800
                               50                                    2 4                                                                           2
                                                6           6        6                                                                                        810
                                                                                       5                                                                                       LCL=765.8
                                                                                                            1
                             700
                                                                     1                                                1
                                                                                                                            R=17.92                                                     Calc for Subgroup
                                0                                                                                           LCL=0                             760
    Subgroup                            0                                100                               200                             300                       400
                                    0                                                                                                                        LC OP     LC1                               LC2               LC3

                             150
                              860
                                                        Xbar Chart by LC OP
                                                        LC1
                                                                       1
                                                                           1
                                                                                           1
                                                                                           LC2                  LC3                                           850
                                                                                                                                                                                                     2002 Historical LC O
                                                                                                                                                                                    LC OP*Sample Interaction
                                            1           1                 11                                          1
                                                                                                                                                                                                         Process
       Moving Range




                              850                                                      11                                   UCL=853.1                                                                                   L
                             100                                                       1                                                                      840
                                                                          1
      Sample Mean




                              840                                                              1                                                                                                                        L
                                                                                                                                                              830




                                                                                                                                                   Average
                                                                                                                                                                               UCL=81.95
                              830
                              820
                              50
                                                                                                                            Mean=819.4                        820                                      Results with     L


                                                                                                                                                                                                      Range = 25.08
                              810                                                                                                                             810
                              800                                                                                                                                              R=25.08
                                                                 2                                                                                            800
                              790                               2                                  2                      2
                                                                                                                                                                                                     Calc for pt to8 pt
                               0                    2           2                                                                                                              LCL=0
                              780
                                                    2           22                                 2                      2 LCL=785.7                         790
                                    0                                                                                                                   Sample                 1       2        3    4  5   6  7



                                                            When comparing the MSA with process operation, a large
                                                             percentage of pt-to-pt variation is MS error (70%) --- a
                                                                  back check of proper test sample selection
                                                                                                                                      slide 23
    CTQ1 MSA Study Results – Process Linkage
               Site 2 Example

   Key issue for Process Improvement Efforts is “When will we see
    change?”
        Initial Improvements to A1 process were made
        Control Plan Improvements to A1 process were initiated
        Site 2 Baseline Values were higher than other sites
        Small step changes in mean and reduction in variation will achieve goal
   How can Site 2 see small, real change with a Measurement System with
    70+% GR&R?

     Use Power and Sample Size Calculator with and without impact
     of MS variation. Lack of clarity in process improvement work,
      results in missed opportunity for improvement and continued
                      use of non-optimal parameters

                                            slide 24
    CTQ1 MSA Study Results – Process Linkage
               Site 2 Example
2-Sample t Test                                      2-Sample t Test
Alpha = 0.05   Sigma = 22.23                         Alpha = 0.05   Sigma = 6.67
             Sample     Target   Actual                           Sample    Target   Actual
Difference     Size     Power     Power              Difference     Size    Power     Power
         2     2117     0.9000   0.9000                       2      192    0.9000   0.9011
         4        530   0.9000   0.9002                       4        49   0.9000   0.9036
         6        236   0.9000   0.9002                       6        22   0.9000   0.9015
         8        133   0.9000   0.9001                       8        13   0.9000   0.9074
        10        86    0.9000   0.9020                      10         9   0.9000   0.9188
        12        60    0.9000   0.9023                      12         7   0.9000   0.9361
        14        44    0.9000   0.9007                      14         5   0.9000   0.9156
        16        34    0.9000   0.9018                      16         4   0.9000   0.9091
        18        27    0.9000   0.9017                      18         4   0.9000   0.9555
        20        22    0.9000   0.9016                      20         3   0.9000   0.9095


               Simulated Reduction of Pt to Pt variation by 70% decreases
                          time to observe savings by over 9X.
                                          slide 25
    CTQ1 MSA Study Results – Process Linkage
                 Site 2 Example
          Benefits of An Improved MS
   Realized Savings for a Process Improvement Effort
        For A1, an increase of 1 number of CTQ1 is approximately $1 per ton
        Change of 10 numbers, 1000 Tons produced in 1 month (832  842)
        $1 * 10 * 1000 = $10,000
   More trust in all laboratory numbers for CTQ1
   Ability to make process changes earlier with R-bar at 6.67
        Previously, it would be pointless to make any process changes within the 22 point
         range. Would you really see the change?
   As the Six Sigma team pushes the CTQ1 value higher, DOEs and other
    tools will have greater benefit




                                            slide 26
                          Objectives
   The Hidden Factory Concept
       What is a Hidden Factory?
       What is a Measurement System’s Role in the Hidden
        Factory?
   Review Key Measurement System metrics including
    %GR&R and P/T ratio
   Case Study at W. R. GRACE
       Measurement Study Set-up and Minitab Analysis
       Linkage to Process
       Benefits of an Improved Measurement System
   How to Improve Measurement Systems in an
    Organization

                               slide 27
    Measurement Improvement in the Organization

   Initial efforts for MS improvement are driven on a BB/GB project basis
        Six Sigma Black Belts and Green Belts Perform MSAs during Project Work
        Lab Managers and Technicians are Part of Six Sigma Teams
        Measurement Systems are Improved as Six Sigma Projects are Completed
   Intermediate efforts have general Operations training for lab personnel,
    mostly laboratory management
        Lab efficiency and machine set-up projects are started
        The %GR&R concept has not reached the technician level
   Current efforts enhance technician level knowledge and dramatically
    increase the number of MS projects
        MS Task Force initiated (3 BBs lead effort)
        Develop Six Sigma Analytical GB training
        All MS projects are chartered and reviewed; All students have a project
        Division-wide database of all MS results is implemented


                                             slide 28
    Measurement Improvement in the Organization

    Develop common methodology for Analytical GB training

    Six Sigma Step   Action                             Typical Six Sigma Tools Used
    Define            Target measurement              Project Charter
                        system for study
                      Identify KPOVs
    Measure           Identify KPIVs                  “Soft” tools: Process Map, Cause & Effect
                      Evaluate KPOV                   Matrix, FMEA
                        performance                     “Stat” tools: Minitab Graphics, SPC,
                                                        Capability Analysis
    Analyze           Measurement System              Gage R&R, ANOVA, Variance Components,
                        Analysis                        Regression, Graphical Interpretation
    Improve           Reduce Reproducibility          “Soft” tools: Fishbone Diagram, Focused
                      Reduce Repeatability            FMEA
                      Reduce Operator or              “Stat” tools: D-Study, t-Tests and
                        Instrument Bias                 Regression, Design of Experiments
    Control           Final Report                    SPC, Reaction Plans, Control Plans, ISO
                      Control Plan for KPIVs          synergy, Mistake Proofing




                                             slide 29
                                  Final Thoughts

   The Hidden Factory is explored throughout all Six Sigma programs
   One area of the Hidden Factory in Production Environments is
    Measurement Systems
   Simply utilizing Operations Black Belts and Green Belts to improve
    Measurement Systems on a project by project basis is not the long term
    answer
   The GRACE Six Sigma organization is driving Measurement System
    Improvement through:
        Tailored training to Analytical Resources
        Similar Six Sigma review and project protocol
        Communication to the entire organization regarding Measurement System
         performance
        As in the case study, attaching business/cost implications to poorly performing
         measurement systems


                                             slide 30

				
DOCUMENT INFO