Paired Comparison Preference Analysis (Paired Difference)
This report presents Paired Comparison Preference Analysis of two different branded
crackers. The paired comparison preference analysis is also called the two-sample test
technique where the panelists could either find difference or no difference between the two
samples given. Conducting the evaluation, the researchers found out that in the product they
presented, the panelists didn’t determine the difference between the two food samples. They
presented the food product in pairs with similar characteristics but different brands by using
the paired comparison test method. To understand the data and interpret them, the researchers
used the chi-square method.
Paired Comparison Sample Test also known as Two Sample Test is a method of
evaluating products which involves the presentation of paired products. It is presented in a
manner of two samples at the same time. This overall difference test method is used when the
test objective is to determine whether a sensory difference exists or does not exist between
two samples. It is also known as the simple difference test. The test is appropriate in
situations where samples have extreme intensities, give rapid sensory fatigue, have long
lingering flavors, or cannot be consumed in large quantities, or a combination thereof. The
test is also appropriate for situations where the stimulus sites are limited to two (for example,
two hands, each side of the face, two ears). The test provides a measure of the bias where
judges perceive two same products to be different. The test has the advantage of being a
simple and intuitive task.
It simply determines whether two products are perceived to be the same or different
overall. his test method describes a procedure for comparing two products. This test method
is sometimes referred to as the simple-difference test. The procedure of the test described in
this test method consists of presenting a single pair of samples to each assessor. The
presentation of multiple pairs would require different statistical treatment and it is outside of
the scope of this test method. This test method is not attribute-specific, unlike the directional
difference test. It is not intended to determine the magnitude of the difference; however,
statistical methods may be used to estimate the size of the difference. Furthermore, this test
method may be chosen over the triangle or duo-trio tests where sensory fatigue or carry-over
is a concern, or where a simpler task is needed.
Main Objective: To determine which of the two food products had more acceptable
1. To determine if the difference between the samples is significant
2. To present samples for evaluation using the simple difference paired comparison
3. To understand how to analyze data from paired comparison tests
IV. Review of Related Literature
Two sample test is a method where in a pair of products are presented at any one time for
evaluation. Like the single approach, sample presentations could also be given successively
although this time, they come in pairs. The response of the panelists could be either they find
a “difference” or “no difference” between the pair successively presented. (Sensory Quality
Assessment, Gatchalian et al).
Methodology and Uses
The main criterion for the use of this test is for two samples to be presented either at the same
time or one at a time successively with very short time intervals. A decision is to be made
soon after completion of the second sample evaluation. Information desired consists of
knowing whether a difference exists between samples or if there is a difference, what is the
degree, direction or nature of this difference. Based on desired information, a corresponding
well-designed score sheet can be developed, although approach to sample presentation
remains the same.
Significance and Use
This overall difference test method is used when the test objective is to determine whether a
sensory difference exists or does not exist between two samples. It is also known as the
simple difference test. The test is appropriate in situations where samples have extreme
intensities, give rapid sensory fatigue, have long lingering flavors, or cannot be consumed in
large quantities, or a combination thereof. The test is also appropriate for situations where the
stimulus sites are limited to two (for example, two hands, each side of the face, two ears).
The test provides a measure of the bias where judges perceive two same products to be
different. The test has the advantage of being a simple and intuitive task.
This test method describes a procedure for comparing two products. A same-different test
determines whether two products are perceived to be the same or different overall. The
procedure of the test described in this test method consists of presenting a single pair of
samples to each assessor. The presentation of multiple pairs would require different statistical
treatment and it is outside of the scope of this test method. This test method is not attribute-
specific, unlike the directional difference test. This method may be chosen over the triangle
or duo-trio tests where sensory fatigue or carry-over are a concern, or where a simpler task is
needed. This standard may involve hazardous materials, operations, and equipment. It does
not purport to address all of the safety concerns, if any, associated with its use. It is the
responsibility of the user of this standard to establish appropriate safety and health practices
and determine the applicability of regulatory limitations prior to use.
Besides measurement of discrimination between a pair of products, the two sample test can
also be utilized in panel selection through determination of panelist’ threshold levels for a
selected basic taste. There are others who find this sensory test method useful for panelist
A special use of the paired comparison is in the determination of differences in specific
sensory characteristics. An identified product attribute may be utilized to describe each of the
sample pair. This will then enable the judge to identify first the specific attribute and second,
the intensity of such characteristics which makes one sample different from the other.
Significance and Use
The directional difference test determines with a given confidence level whether or not there
is a perceivable difference in the intensity of a specified attribute between two samples, for
example, when a change is made in an ingredient, a process, packaging, handling, or storage.
The directional difference test is inappropriate when evaluating products with sensory
characteristics that are not easily specified, not commonly understood, or not known in
advance. Other difference test methods such as the same-different test should be used. A
result of no significant difference in a specific attribute does not ensure that there are no
differences between the two samples in other attributes or characteristics, nor does it indicate
that the attribute is the same for both samples. It may merely indicate that the degree of
difference is too low to be detected with the sensitivity (α, β, and Pmax) chosen for the test.
The method itself does not change whether the purpose of the test is to determine that two
samples are perceivably different versus that the samples are not perceivably different. Only
the selected values of Pmax, α, and β change. If the objective of the test is to determine if the
two samples are perceivably different, then the value selected for α is typically smaller than
the value selected for β. If the objective is to determine if no perceivable difference exists,
then the value selected for β is typically smaller than the value selected for α and the value of
Pmax needs to be stated explicitly. (http://www.astm.org/Standards/E2164.htm)
Advantages and Disadvantages of Use
One of the major advantages of the two sample test is the possibility of making direct
comparisons that allow the determination of relative difference between paired samples. This
could be in terms of presence or absence, degree, or direction of difference.
Some objections to the use of the two sample test include the observation that this approach
cannot be used to determine the true difference or real discrimination between samples. It is
suggested that this approach should be used only for measuring quality judgments a sin
consumer acceptance or preference tests. ( Amerine, et al. 1965; Compusense, 1997)
Another disadvantage is the need for homogeneity in appearance between paired samples. An
obvious difference will make the whole approach a useless exercise. Hence, the requirement
of homogeneity between samples becomes one of the major limitations.
Food Sample – Biscuit (Sky flakes and Magic Flakes)
Drinking Glass – 10 pcs.
Paper Plates – 10 pcs.
Scoresheet – 10 pcs.
1. The two procured different brand food samples were prepared first.
2. The samples were coded separately and presented for sensory evaluation to 10
selected panelist following the scheme of sample presentation stated below:
Set 1 A B
Set 2 B B
Set 3 A A
Set 4 B A
3. The coded samples were presented to the panel together with the score sheet.
4. The panel was asked whether there is a difference in the given food samples
or there is no difference between the two.
5. The score sheets were collected at the end of the evaluation and the results
6. The results were interpreted using Chi-Square test and determined if
preference is significant at 1% and 5% level.
VI. Results and Interpretation
Table 1: Tabulated Result of Correct Judgment in Paired Difference Test
Panelist A=B B=A
Cariaga ☺ ☺
Mendiola ☺ ☺
Morales ☺ ☺
Total 5 + 7 = 12
Legend: ☺= Correct Judgment
Ho: Sample A is not the same as Sample B
Ha: Sample A is the same as Sample B
Specific Level of Significance: α – 0.01 and 0.05 or 1% and 5%
Base on the tabulated results:
At 1% probability level = 12 < 15 (min number of correct judgment)
At 5% probability level = 12 < 16 (min number of correct judgment)
Using Chi – Square method:
(Observed Value – Expected Value) ² / Expected Value = Chi – Square value
+ = 0.4 + 2.5 = 2.9
= 2.9 < 3.84(Value of Chi-Square required for 5% probability levels)
= 2.9 < 6.64(Value of Chi-Square required for 1% probability levels)
Comparison Shows that: Since,
Computed Chi-Square = 2.9 < 3.84 = Value of Chi-Square required for 5% prob. Level
Computed Chi-Square = 2.9 < 6.64 = Value of Chi-Square required for 1% prob. Level
Then Ha is accepted. The difference between the samples is insignificant.
The evaluation was conducted to obtain information regarding consumer’s finding
on two different brands of baked biscuits. It was deemed of great importance to know
whether the two brands of samples subjected to evaluation were different or not.
The Chi – Square resulted in the rejection of the null hypothesis (Ho) which
implies that the property and other factors of the first sample are comparable to the other.
Therefore, the difference noted in the two samples is considered insignificant
since base on the tabulated results and Chi Square computed, sample A is comparably the
same as Sample B. The presented samples proved the accepted statistical hypothesis (Ha)
using the simple difference paired comparison method.
Brannan, Grace Divino, Miflora Minoza Gatchalian. “Two Sample Test”. Sensory
Quality Measurement – Statistical Analysis of Human Responses.
“Sensory Evaluation”. http://www.astm.org/Standards/E2139.htm
Amerine, et al. Sensory Evaluation.1965
SENSORY EVALUATION IN FOODS
Laboratory Report no. 4
Paired Comparison Preference Analysis
Agasen, Joseph Carlo
Baleda, Ma. Roselle
Daymon, Glyvie Mae
Dimatulac, Janina Marie
Flora, Anna Margarita
Santos, Tricia Nicole
Professor Ana Maria L. Espiritu
January 10, 2009