Docstoc

Quali Sense

Document Sample
Quali Sense Powered By Docstoc
					        Quali-Sense




    SENSOMETRICS 2008
The Panel Checking Workshop

CAMO Software, by Dongsheng Bu (US Office)
    Why Use the Quali‐Sense?

• Quality check of data
• Prepare data for further analysis
  – Can we trust the panel?
  – Calculate a reliable panel average


• Evaluate the performance of each panelist
  – Give feedback on performance
  – Need for training
  – Remove unreliable panelists
            Preparing Panel Data for Further 
                 Multivariate Analysis
                                              Replicate 1         Replicate 2   Replicate 3




                                          t
                                      lis
                                    ne
                                  Pa
                              Product


                                        Attributes
                                                                    Average
                                                           Sensory attributes
Remove unreliable panelists
                                                 Product

Calculate a reliable panel average
                  How the Quali‐Sense do 
                      the Analyses?
           Five tests that cover most sensory aspects



•   Assessor sensitivity

•   Assessor reproducibility

•   Assessor Agreement

•   Assessor Crossover

•   Eggshell Correlation
         Workshop Dataset Overview
• To study the effect of storage on the sensory quality of a salad dressing
• There are several “holes” in the design that the dressing had gone bad
• Some of the assessors were not present on all sessions.
  Dataset Rearrangement

    There are 38 designs that have data from all panelists




All 38 designs/products, plus 2 subsets re-grouped
by Evaluation date were analyzed separated,
i.e. All38, Oct94&Dec94, May95




  A design can be treated as a product
  The code in a name relates to storage condition
  e.g. P1_1121 means 17 Oct 94/Glass/Dark/23ºC storage
                                                     Overview on Panelists
                                Reproducibility, Sensitivity and Agreement

                                                        Counts of poor performance
                                            Line Plot                                                                                     Line Plot
8                                                                                             10




6




4                                                                                              5




2




0                                                                                              0
                                                                                  Variables                                                                                     Variables




                                                                                                         A:2


                                                                                                               A:3


                                                                                                                        A:4


                                                                                                                              A:5


                                                                                                                                    A:6


                                                                                                                                             A:7


                                                                                                                                                      A:8


                                                                                                                                                            A:9


                                                                                                                                                                  A:10


                                                                                                                                                                         A:11


                                                                                                                                                                                 A:12
          R:2


                R:3


                          R:4


                                R:5


                                      R:6


                                               R:7


                                                        R:8


                                                              R:9


                                                                    R:10


                                                                           R:11


                                                                                   R:12




All38 Oct94&Dec94 May95                                                                       All38 Oct94&Dec94 May95


    Reproducibility Test                                                                              Agreement Test
    Panelists 2, 5, and 10 have large counts                                                          Panelists 2 and 5 have large counts
    in poor performance                                                                               in poor performance

    Sensitivity Test is OK for all panelists (results not shown)
        Reproducibility
p-Value < 0.05 indicates poor performance and colored in red
Big and red dots in the Spotty plot indicate poor performance




                                  Monitors the ability of a
                                  single assessor to
                                  reproduce their result
                                  with respect to the rest
                                  of the panel.
                      Agreement
         Having a panelist with poor performance in the panel
         will affect the results from the whole panel.

Outlying assessor
                     Agreement

Monitors each assessors agreement with the rest of the
panel. Agreement errors could be due to several reasons:

                                        Magnitude

                                        Expert

                                        Non-discriminator

                                        Cross-over

                                        Non-perceiver
         Overview on Sensory Attrributes
         Reproducibility, Sensitivity and Agreement

                                         Counts of poor performance
                                                                                                                     Line Plot
8




6




4




2




0
                                                                                                                                                                                                                               Variables
           S:Acidic_Fla



                          S:Acidic_Odo


                                          S:Colour_Int



                                                         S:Colour_Hue


                                                                        S:Fresh_Flav



                                                                                       S:Fresh_Odou



                                                                                                      S:Mustard_Fl


                                                                                                                        S:Off_Flavou



                                                                                                                                       S:Off_Odour


                                                                                                                                                     S:Rancid_Fla



                                                                                                                                                                    S:Rancid_Odo


                                                                                                                                                                                   S:Sour_Flavo



                                                                                                                                                                                                  S:Sour_Odour



                                                                                                                                                                                                                 S:Sweetness



                                                                                                                                                                                                                               S:Whiteness
All38 Oct94&Dec94 May95



      Sensitivity Test shows many panelists have problem with Sweetness

      Reproducibility Test and Agreement Test are OK (results not shown)
                       Sensitivity
• Measures the ability of a single assessor to identify
  product differences.
• A low p-value significant
  difference between
  products, and is thus good.


                                                Poor performance




                                               Good performance
                       Rank Correlation
     ”Eggshell” plot indicates the ranking of each
     product by the panelists.


Assessor 5 has
ranked the products
differently from the
other panelists
                           Agreement and 
                           Crossover Errors




                                       Subsets (Oct94&Dec94, May95) results

Table shows level of agreement, and
also cross-over error, stemming from
wrong use of scale.

Panelists rank product differently
In Sweetness
     Summary from Quali‐Sense


• Generally speaking, data quality is pretty good

• Panelists 2 and 5 have problem in reproducibility and

  agreement

• The panel need to improve the ability of Sweetness

  test, both in sensitivity and rank correlation
              PCA (no P2 and P5) ‐ Welcome Back 
                      to Multivariate World‐ The Unscrambler
        PC2                             Scores                                                 PC2                                X-loadings               Colour Intensit
  4                                                                               0.5
Early Evaluation Date
            3322                                     4421                                                                                         Colour, Hue
                            3621                            43216321
  2                                  4621
                       3222 5322 6322
                           4322                                                                                 Mustard Flavour
                              3521                                                                            Acidic Odour
                      42226122 5621 4121
                        4122 3121 6621 6121 6221
                                 3221     4221       6421                                                     Acidic Flavour
                     5222 6222
                       5122 31224521 3421                     5321
                               5521 5221
                                6521
                                   5121
                                                  5421
                                                                                                             Fresh Flavour
                      23222221                  3321                                                         Fresh Odour
  0                        2621
                       2122 2421                                                    0
                    2222 2521 3212 2321
                   1621 2121                                                                                                                      Rancid Flavour
                   13121311                            4312                                                                                        Rancid Odour
                   1322
                  1321
                                           2312                                                                                                     Off-Flavour
                                                                                                                                                     Off-Odour
                  1122                         3312 2311 3311
                                                      4211
                 1221
                 1421                  6212
                                        2212
                 1212
                    1211                         4212
 -2              1222
                 1121
                 1521                              6312
                                                2211
                                          5212       5312                                                           Sour Odour
                                                 3211
                                                   5211                                                                           Sour Flavour

                                                                                                                          Whiteness
 -4                                                                               -0.5
                                                                            PC1                                                                                          PC1

         -5                    0                       5                    10                  -0.4               -0.2               0         0.2    0.4
 AveNoScaleMC, X-expl: 69%,16%                                                    AveNoScaleMC,X-expl: 69%,16%
                                                                                                                                          Attribute map?
  4
        PC2                             Scores                                    X-loadings                                              And 3D loading?
      Light                                                                                                           X
                        Dark                             Dark                                          -Y
                             Dark                                                        0.9
  2                                    Dark                     DarkDark
                       Dark Dark Dark
                            Dark
                                Dark                                                                                                        Colour, Hue
                      DarkDark DarkDark Dark Dark Dark
                        Dark Dark Dark Dark                                              0.6                                 Sour Flavour
                     Dark DarkDark Dark
                       Dark Dark          Dark                     Dark
                                 Dark Dark
                                 DarkDark
                                                      Dark
                      DarkDark                     Dark
  0                         Dark
                       Dark Dark
                    Dark Dark Daylight    Dark                                                                     Sour Odour
                   Dark Dark
                       Daylight                            Daylight                      0.3
                   Daylight
                   Dark
                  Dark                                                                                                                               Colour Intensit
                                             Daylight                                                          Fresh Odour
                                                                                                                     Flavour
                  Dark                            DaylightDaylight
                                                         DaylightDaylight                                                                                   Rancid Odour
                 Dark
                 Dark                    Daylight
                                          Daylight                                                              Acidic Flavour
                                                                                                                 Acidic Odour                                        Flavour
                 Daylight
                    Daylight                        Daylight                               0                       Mustard Flavour                          Off-Odour
                                                                                                                                                            Off-Flavour
 -2              Dark
                 Dark
                 Dark                                  Daylight
                                                   Daylight
                                            Daylight     Daylight                                                                                               0.6
                                                    Daylight
                                                       Daylight                                                                                                  0.3
                                                                                                                     Whiteness                                   0
                                                                                         -0.3                                                                    -0.3
 -4                                                                                      -0.4               -0.2              0             0.2             0.4 -0.6
                                                                            PC1

         -5                    0                       5                    10
 AveNoScaleMC, X-expl: 69%,16%                                                    AveNoScaleMC,X-expl: 69%,16%,11%
                  Conclusion

  We suggest such workflow in Product Development


The Unscrambler      Plan

                    Sensory      Monitor panel performance
                                 Quality check
 Quali-Sense
                    analysis     Prepare data for analysis




The Unscrambler
                     Data 
                    analysis
     Acknowledgement

• CAMO Oslo Norway Office
  – Marion Cuny


• CAMO Bangalore India Office
  – Suresh Kumar


• Sensometrics 2008 Committee
Play Demo
www.camo.com/products/quali-sense.html

Download 30-Day Trial Software
www.camo.com