Likert Scaling - wwwdrweedmancom

Document Sample
Likert Scaling - wwwdrweedmancom Powered By Docstoc
					                                         Chapter 14
                                        Likert Scaling:
                                      A how to do it guide
         Likert scaling is a process which relies heavily on computer processing of results and as a
consequence is my favorite method of attitude scaling. However, it was not so long ago that this
process of scaling was considered too time consuming and other methods, like Thurstone
methods, were preferred.
         I will present Likert methods by tracing through an example. You should realize, of
course, that what I will present is my interpretation of Likert methods.
         The first step in this scaling method, after you have selected the attitude to be assessed, is
to assemble a group of judges. These judges have as their task the development of potential
statements which would tap the attitude domain. In short, you want these judges to compose a
list of statements which can be responded to on a five point scale. This five point scale will
range from "strongly agree" to "strongly disagree", with the middle of the scale identified by the
response alternative "undecided" or "neither agree nor disagree". I have chosen for our example
to develop a test of self-attitude. The list of the statements that I developed to be potential
members of our final questionnaire can be found in the first draft of the field test questionnaire
on the following page. You should note that I did not utilize a panel of judges to get these
statements, I just made them up myself. You would, of course, use judges. Your judges should
have some knowledge of the topic area. In assembling your judges, usually 3 to 5 are sufficient,
you instruct them to develop items which would be positive and items that would be negative.
By positive items I mean any items on which a strongly agree response would indicate a favor-
able disposition toward the attitude and by negative I mean items on which a strongly disagree
response would indicate a negative disposition. For an example of these types of statements, see
statements 1 and 9 on our field test questionnaire. You should instruct your judges to avoid
items which appear to be ambiguous (ask for two or more opinions in one statement). I
included one of these ambiguous statements in our questionnaire to demonstrate how the scaling
methods is able to systematically eliminate the statement (see statement 6). It is important that
your potential statements represent a large range of possible opinions and are relevant to the
attitude domain. I have attempted to include some irrelevant items, see statements 5 and 7.
You should develop about 40 items and aim for a final test of about 20 items (in our example we
will use less items in our original pool).
         Once you have finished this initial phase, you should have your field test questionnaire
typed and administered to as many people as practical. I administered our field test
questionnaire to 25 people, mostly friends.
         The next step is to score each of the items and calculate a total score by summing the
various items. What you will end up with, in our example, is 13 scores. You always give 5 for
"strongly agree"; 4 for "agree"; 3 for "neither agree nor disagree"; 2 for "disagree"; and 1 for
"strongly disagree". You assign the scoring in this manner regardless of whether you think the
item is a positive item or a negative item.
         We are now ready for our first set of calculations. Your task is to calculate the
correlation between each item and the total for the test. In other words, what is the correlation
between responses to item 1 and the total of all items? Item 2 and the total? etc. I have
performed these calculations and they are summarized in the correlation table on the following
page and are listed under Run 1.
         The correlations that identify items as being negative will need to have the polarity of the
scale coding reversed. Therefore, for questions 3, 6, 9 and 10 we will need to reverse the
scoring, assigning a 5 to the response "strongly agree" and 1 to "strongly disagree." Can you
see that reversing the scoring polarity of these items will change the total score for the test?
You will need to recompute the total score also.
         The next step is to calculate the 12 correlations again. These correlations are presented
in the second column of correlation coefficients. We want to continue reversing the polarity of
the items with negative correlations until we reach the point where all the correlations are
positive. We will reverse the scoring polarity of questionnaire items 2, 3, 4, 7, 9, 10, and 11
since they gave negative correlations on our second run. Notice that we had reversed the
polarity of item 4 in the previous step and at this step we are going to reverse the polarity back to
its original scoring. Column 3 in our correlation table gives the correlations for our third run.
(Again, note that we had to recompute the total for this third run).
         Notice that in our third set of correlations we need, again, to reverse the polarity of some
items back to their original polarity. We will continue this process until we reach some stable
pattern, reversing the polarity and recalculating the correlations. At the end of the fourth set of
correlations it appears that we have some questionnaire items (number 1, 10, and 11) which seem
to want to continue to alternate their polarities.
         Likert Version of The Self-Concept Questionnaire
Below are 12 statements regarding attitudes which I would like you
to rate on a five point scale. I want you to circle one of the symbols
AS through SD to represent your opinion on each of the statements.
The scale is defined as follows:
           SA = Strongly agree with the statement
           A = Agree
           ? = Neither agree or disagree
           D = Disagree
           SD = Strongly disagree with the statement

1) I like myself.
     SA        A         ?         D          SD
2) I feel I will make a significant contribution to mankind.
     SA        A         ?         D          SD
3) I have a good relationship with my mother.
     SA        A         ?         D          SD
4) I have a good relationship with my father.
     SA        A         ?         D          SD
5) I enjoy math/science courses.
     SA        A         ?         D          SD
6) I like my name, but I am concerned by my weight.
     SA        A         ?         D          SD
7) I am good at drawing things.
SA             A         ?         D          SD
8) I wish I were someone differ.
     SA        A         ?         D          SD
9) I have a low opinion of myself.
SA             A         ?         D          SD
10) I have had a good life?
     SA        A         ?         D          SD
11) My friends have a good opinion of me.
     SA        A         ?         D          SD
12) I do not have many friends.
     SA        A         ?         D          SD
            Correlations Between the 12 Test Items
                 an the Total for the 12 Items

Items                              Correlation Runs
               1        2        3      4     5     6                 7         8
 1          -.13      .12      .18    .18   .56   .54               .57    .55
 2           .08     -.36      .39    .39   .45   .58               .64    .77
 3          -.10      .08     -.02    .00   ---   ---               ---    ---
 4          -.08     -.04     -.16      .21      .15      .13      ---     ---
 5           .08      .05      .07      .10      .15      .17      ---     ---
 6           .25     -.32     -.34     -.29     -.14     -.26     -.35     ---
 7           .14     -.11     -.03      .00      ---      ---      ---     ---
 8           .17      .05      .28      .29      .38      .42      .53     .60
 9          -.18      .24     -.17      .15      .44      .46      .70     .68
10          -.16     -.17     -.19     -.18      .08      ---      ---     ---
11           .29     -.12      .16     -.17      .06      ---      ---     ---
12           .00      .21      .13      .16      .47      .52      .63     .65

At this point, we need to begin the process of eliminating questionnaire items
from consideration in the final form of the instrument.
        We will begin the elimination process by discarding questionnaire items
with the lowest correlations (items 3 and 7). The new correlations, with items 3
and 7 eliminated, are presented in column 5. As you can see we now have two
additional items which we can disregard (items 10 and 11). The new
correlations, now with items 3, 7, 10, and 11 eliminated, are presented in column
6 (notice that we have continued to reverse the polarity of item 1, in a futile
attempt to achieve a positive correlation).
        At this step we will eliminate items 4 and 5. Have you noticed that the
correlations for the items that have not been eliminated continue to become
larger? The correlation, with items 3, 4, 5, 7, 10, and 11 eliminated, are
presented in column 7. Now we will eliminate our final item (fickle item 1) and
run the last set of correlations.
        Through this process, we have arrived at a five item test of self-attitude
which is composed of items 2, 6, 8, 9, and 12 from our original pool of items.
Critique of the Likert Scaling Process

        There are a number of criteria on which we can evaluate the outcomes of a
scaling process. The first of these guidelines is zero point. In other words, with
our Likert version of a self-attitude scale do we have a number such that scores
below this number would reflect negative self-attitude and score above this
number would reflect positive self-attitude? The neutral point for each item is a
score of 3, "neither agree or disagree." Since we have a five item test, if a
respondent achieved a total score of 15 then logically he/she should have a neutral
self-attitude. Therefore, with respect to the zero point guideline it would appear
that the Likert scaling process satisfies the requirement. If I were to rate the
degree to which Likert scaling accomplishes this task on a five point scale where
1 would indicate no compliance and 5 would indicate very good compliance, I
would give Likert scaling a 4 on zero point compliance.
         Our next criteria on which we evaluate a scale is equality of units. In
other words, does our test provide scores which have interval-like characteristics,
therefore enabling us to use more sophisticated statistics with the test results?
Surely, a larger total score on our test would indicate a more positive self-attitude
than a smaller total score. Therefore, it would appear that we have, at minimum,
an ordinal score. If every item on our self-attitude scale correlated perfectly with
the total score for the test and if there were zero correlations among the scale
items we would have excellent equality of units since each questionnaire item
would be contributing uniquely to the total score. However, no test I know of
has fulfilled such a stringent requirement. The intercorrelations among our five
items are summarized below. As you can see our inter-item correlations are not
extremely high except for the correlation between items 9 and 12. This pattern of
intercorrelations would suggest good equality of units, I will give our scale a
rating of 3 on our five point rating system.
         Unidimensionality of items refers to characteristics of the test items. In
short, is there a process to insure that the test items assess only one attitudinal

                      Intercorrelations Among
                        Questionnaire Items
                  1      2      8      9      12
    1            1.00   .35    .34    .05     .08
    2            .35       1.00       .48        .34         .41
    8            .34        .48      1.00        .16         .16
    9            .05        .34       .16       1.00         .61
   12            .08        .41       .16        .61        1.00

dimension? The assurance is satisfied first by using judges to develop the test
items and instructing them to review the items to insure that they assess only one
content. Secondly, unidimensionality is supported by the correlation process. If
an item were not uni-dimensional then its correlation pattern should indicate an
unstable pattern, like we found for item 6 on our trial instrument. I would give
Likert scaling a rating of 4 on our five point scale of compliance in terms of
usunidimensionalityof items.
        Unidimensionality of scale is concerned with whether or not the total
score for the test is an index of one attitudinal component or several attitude
In order to determine the degree to which Likert scaling violates this
uunidimensionalitycriteria, a factor analysis procedure would have to be
employed. However, employing judges to develop the various item contents
would seem to support acceptable total score unidimensionality for Likert scaling.
So I will give Likert scaling a score of 2 on our compliance rating system for total
score unidimensionality.
        An index of the reliability of our Likert scale can be determined from the
item inter-correlations presented above. The mean correlation among the 5 test
items was .44 and the estimated reliability of the test was .91.
                         k x ra
   reliability =
                       1 + (k-1) x ra

   where k = number of items

         ra = mean item inter-correlation

                       5 x .44       2.5
  reliability =                 =      = .91
                1 + (5-1) x .44   2.76

        What is important here is to realize that the Likert scaling procedure
provides you with a method for determining the reliability of the test that is
developed. Therefore, on our five point rating scale of compliance I will give
Likert scaling a 5 on the reliability criteria. Remember that our rating of 5 means
that the scale development process provides a very good way of estimating
reliability and does not necessarily imply that our particular test has good
         The last criteria on which we evaluate a scaling process is validity, more
specifically content validity. As you may recall, content validity, in a sense, is an
evaluation of the amount of rigor which has been used in the test item
development. Those procedures which involve more rigor are viewed as more
content valid than those procedures which utilize less rigor. In our case I would
give Likert scaling a 5 on our rating scale of compliance.
        A final rating needs to be made of the ease of application of Likert scaling.
In other words, compared to other scaling methods, how much effort is required
to develop a Likert type attitude test? I would consider the degree of effort to be
minimal, if you have the availability of a computer and if you are versed in
computer programming. I would rate Likert scaling as a 4 in ease of application.
        The summary of the ratings of Likert scaling is presented below.
        Compliance Ratings of Likert Scaling*
       Criteria                            Rating
      Zero Point                           4
      Equality of Units             3
      Unidimensionality of items           4
      Unidimensionality of scale           2
      Reliability                   5
      Content Validity                     5
      Ease of application           4
 * 1 = no compliance, through 5 = very good compliance