VIEWS: 237 PAGES: 73

									Quantitative Research:
Questionnaire Design

      Alan Sawyer
     Marketing 6816

            Overview of Steps
   Information sought
   Method of administering
   Question content
   Response format
   Question wording
   Question sequence

    Methods of Administering
   Telephone
   Personal interview
       Mall intercept
       Recruited to central location
       Door-to-door
   Web-based
   Mail

                  Information Sought
   Guided by research problem
   Determine first if secondary information is
       (i.e, is survey needed?)

   Determine if respondents can answer the key
   Avoid straying off into “interesting” areas of
The Researcher - Respondent
  Communication Process

Encode question   Decode question


Decode answer        Encode answer

    Quality of Measurement

   Reliability:
       Are the measures consistent?
       Are they free of random error?

   Validity:
       Are we really measuring what we think we are

         *For a measure to be valid, it must be reliable.

Reliability and Validity
    Neither reliable nor valid

Reliability and Validity
 Highly reliable, but not valid

Reliability and Validity
  Highly reliable and valid

                      Question Content
   Is the question necessary?
       How will it be used to address the research purpose?

   Has the question been adequately covered by other

   Do respondents have the necessary information?
      Can they give info?

   Will respondents provide the information?
       Avoid embarrassing or threatening questions
       Address sensitive issues at end of the survey

            How Respondents Deal with Overly
            Personal or Embarrassing Questions
   Item non-response
       not answer that question

   Refusal
       Decide to not complete questionnaire

   Distortion (give incorrect answer)
       Example:                                Men Told     Men Told
          recent survey about number           Answers     Answers Not
                                               Anonymous    Anonymous
             of sex partners:
          Men: over-estimated when they       Women Told   Women Told
            thought their answers were not      Answers     Answers Not
           anonymous;                          Anonymous    Anonymous

          Women: did not overestimate)                              11
              Question Content
   Will the respondent be able to remember the
    information sought?

            Types of Forgetting
   Omission
       unable to recall something that took place

   Telescoping
       remembering an event as occurring more recently
        than when it actually occurred

   Creation
       remembering an event that did not actually occur
         Sample Question
 Below are listed several products.

 Please indicate with an “X” each product that
 you or anyone in your household has
 purchased in the last three months?

 Below are listed several products. Please
 indicate with an “X” which products you or
 anyone in your household has ever

 For each product ever bought, please indicate
 with an “X” which products have been
 purchased in the last three months.
          % Stating that Product Had been
          Purchased in Last Three Months

                “A” question “B” questions
white glue           46%        32%
aspirin              68%        57%
replacement tires    32%        24%
CD album             41%        32%

                 Form of Response
       Open-ended
         What is your favorite brand of candy bar ?
         How much would the price of a Snickers bar have to increase before
          you would switch to another brand?
         What brand would you switch to?

       Dichotomous
         Do you eat candy bars?   Yes_____         No_____

       Multichotomous
          Which of the following statements best describes your attitude
           toward Snickers…
         a.   Satisfies appetite
         b.   Provides me with energy
         c.   Satisfies chocolate/candy cravings
         d.   Is fun to eat                                               17
                   Answers to Closed-end
Answers should
   be mutually exclusive
       No overlapping categories
       e.g., “never used; used once; used twice; used 3 or more”
       e.g., “Age: 0-10; older than 10 but less than 20;
         older than 20 but less than 30; 30 – 50; older than 50”

   be collectively exhaustive
       account for all possible answers

   use words of respondents
       Those she or he would or does use
                        Guidelines for
                      Question Wording
   Use simple words and language
       Have you ever experienced cognitive dissonance after buying a
        new car?

   Avoid ambiguity
       Do you go food shopping regularly?
       How many people work here? ______

   Avoid leading questions
       Many consumers are concerned about the security of internet
        transactions. What is your chief concern about using the internet
        to make purchases?
           More Question Guidelines
   Avoid implicit assumptions
       How can Starbucks improve its excellent customer service?

   Avoid generalizations and estimates
       How many times have you gone to the movies in the past year?

   Avoid double-barreled questions
       How would you rate the friendliness and knowledge of Circuit City

           Sample Question
   Do you own a Sony television?

            Sample Question
   Would you like to join the marines,
      if there a draft installed for all men?

            Sample Question
   Do you believe that this new salt substitute
    would be as good as regular salt?

              Sample Question
   Which would you say is contributing most to the
    war effort --- labor union leaders, industry, or

             Sample Question
   (Survey of auto service garages)
What is your average inventory of auto

             Guidelines for Sequencing
   Use simple, interesting opening questions to
         build momentum
   Use the funnel approach, asking broad questions
   If multiple filter questions are necessary, try to ask
    them before asking more detailed questions.
       e.g., “have you patronized a fast food restaurant in the past two
        Guidelines for Sequencing
   Group questions by topic or form

   When switching topics, use transitional phrases
    to make it easier for respondents to switch their
    for trains of thought.

   Place demographics, as well as difficult or
    sensitive questions, at the end                27
            Context Effects
   Whether ask “How do you feel today?” at

   Whether ask about something positive or

        Effect of Sequence Effects:
            Priming with Mood
    Ask about Weather on a Bad      Ask about Weather on a Good
     Day Before Asking Second        Day Before Asking Second
      Question about Opinion           Question About Opinion

   Do Not First Ask about Weather   Do Not First Ask about Weather
           on a Bad Day                    on a Good Day

Weather Affects Mood and Priming of Mood
 Affects Answer to Key Opinion Question
        (similar results for other primes)
            Example of            Order Effects
   Survey about brand confusion between two companies
       Muffler King and Speedy Muffler King
       Legal case about confusion due to entry of
         Speedy Muffler King to market already occupied by
         Muffler King

   Respondents more apt to show signs of confusion when
    SMK questions preceded questions about MK than vice
       When first asked about MK, they were alerted that Speedy
         Muffler King was not a “fast Muffler King” but a different firm
       No alert (and thus confusion) when first asked about Speedy …
               Effect of Prior Mention of
                Sensitive Issue Before
                  Rating of the Brand
   Two standard poll questions.
     1. whether people thought the country was headed in the right
        direction or was "seriously off on the wrong track.“
     2. whether those interviewed approved or disapproved of the job
        George W. Bush was doing as president.

   To measure the "Bush effect," the interviewers varied the order of the
        Half of the respondents were first asked what they thought about Bush's
         performance. Then they were asked about the direction of the country.
        For the other half, the order was reversed.

   Asking first about Bush boosted the proportion that saw the country
    headed “in the right direction” by 8 percentage points -- from 34
    percent to 42 percent.                                             31
   Bush's magic didn't hold in the other direction, however.
        His job approval rating fell by about 6 percentage points (to 60 percent)
         when people were first asked where they thought the country was

   Larry Hugick, vice president of Princeton Survey Research
    Associates: "People's opinions about the president and the direction
    of the country are intertwined,"
        "If the Bush approval question is first, people tend to respond in the
         context of their views of Bush and therefore express more positive views
         about where the country is headed.
        If the direction of country question is first, dissatisfaction with the country
         tends to depress Bush’s ratings.“
             (negative prime  more negative ratings)
   But Bush isn't the equal of the master presidential prestidigitator, Bill
    Clinton -- at least not yet.
        When The Post conducted a similar test of Clinton's mood-swaying
         prowess in 1998, asking the job approval question first was enough to
         boost the right direction number by 10 percentage points.

   But unlike what happened with the Bush test, asking the right direction
    first didn't move the Clinton job approval rating one bit
       suggesting that public views of Clinton in his sixth year in the White

         House were more firmly fixed than current opinions of Bush.

Experiment conducted by The Washington Post, the Henry J. Kaiser Foundation and
Harvard University as part of a national poll. N = 2,886 randomly selected Americans.
NY Times, November 17, 2002; Page B05.
             Context Effects:
          Anchoring & Adjusting

   There are approximately 180 wars going on today.
    None of these wars are taking place on the continent
    of Africa. How many countries do you think there are
    in Africa?

   There are approximately 20 wars going on today.
    None of these wars are taking place on the continent
    of Africa. How many countries do you think there are
    in Africa?

                  Context Effects:
               Anchoring & Adjusting

                  180 Wars        20 Wars
 Mean                    50.3           20.5
 Variance               290.1           71.1
 Sample Size                 11             11

Q:   How many countries do you think
      there are in Africa?
                  Scale Effects:
                 Perceived Frequency

               Scale 1         Scale 2
         Up to ½ hour     Up to 2 ½ hour
         ½ to 1 hour      2 ½ to 3 hours
         1 to 1 ½ hours   3 to 3 ½ hours
         1 ½ to 2 hours   3 ½ to 4 hours
         2 to 2 ½ hours   4 to 4 ½ hours
         > 2 ½ hours      > 4 ½ hours

Q:   How much do you watch television per day?
              Scale Effects:
             Perceived Frequency

                              Scale 1 Scale 2
Percent Reporting > 2 ½ hrs     16%      38%

               Scale Effects:
     “How much do you drink a day?
    (asked in an alcoholic treatment center)

   0 ounces of alcohol      0 ounces of alcohol
   1-4 ounces               1-4 ounces
   5-8 ounces               5-8 ounces
   9-12 ounces              9-12 ounces
   12-16 ounces             12-16 ounces
   more than 16 ounces      17-20 ounces
                             21-24 ounces
                             24-28 ounces
                             29-32 ounces
                             33 or more ounces
    “How interested are you in
    buying this new product?”
                                  Very Much Interested
Immediately after description                 2.8%

After asking the advantages                  16.7%

After asking the disadvantages               0.0%

After asking the advantages and
   then the disadvantages                    5.7%

After asking the disadvantages and
   then the advantages                       8.3%
Brand Equity Measurement
   Awareness
   Associations
       Knowledge
       Liking
   Perceived Quality
   Loyalty
       Preference
       Intention
       Past Behavior   40
                     Brand Awareness
   Unaided Recall
       Which brands of paper towels are you familiar with?
       Which brands would you consider purchasing? _______
         (referred to as the “Consideration Set”)

   Aided Recall
       Which extra-absorbent paper towel brands can you name?

   Recognition
       With which of the following brands of paper towel are you familiar?
        __Bounty, __Viva, __Scott, etc. …                               41
              Brand Associations
   Knowledge (Beliefs or Image)
       Please rate each of the following brands of paper
        towel on these dimensions:
            Absorbency
            Economy
            Strength
            Quality
            Attractive designs
       Response scale: 1= low; 10= high
            Or 1-4, 1-5, 1-7 scale


                       Aj =  (wi) (bij)

Aj    = Attitude toward Brand j (a product or service)
wi    = the goodness/importance of the attribute
bij   = belief about brand j on attribute i
      = summation over n attributes

Alternative formulation is the Ideal Point Model:
    Aj =  (wi) (bij - Ij), where I = ideal point or ideal level of Attribute I
Can alter equation further to deal with attributes where more is not better
  (I.e., sweetness, weight)

   Representing Associations

AGGREGATED PERCEPTIONS                               

                     May Need an
               Extension of Model
                Aj =  (wik) (bijk)
Where k stands for the
             situation of use
 example: restaurant
      For good food
      For fast service
      For entertaining
      When with kids
      When celebrating
      When treating ourselfves

              Measuring Importance
   Remember that there are several ways to
    measure attribute / benefit importance
       See Appendix at end of these notes
       Scales, ranks, fixed-sum allocation, trade-offs, problem

   Choose one that works best for you, given:
       Resources
       Time
       Respondents

            Brand Associations
   Liking (Brand Attitude)
    Please rate Bounty paper towels in terms of their overall
        poor_ _ _ _ _ _ _ _ _ _excellent

    Bounty paper towels:
        good_ _ _ _ _ _ _bad
         like_ _ _ _ _ _ _dislike
    favorable_ _ _ _ _ _ _unfavorable

                 Brand Personality
   Set of human characteristics associated with or
    attributed to the brand

   Consistent and stable characteristics or tendencies

   Brand personality as representation of functional
    features or benefits

   Transfer of meaning
                Brand Personality
   Sincerity
       down-to-earth, honest, wholesome, cheerful

   Excitement
       daring, spirited, imaginative, up-to-date
                                                    What is the
   Competence
       reliable, intelligent, successful      personality of
   Sophistication                             YOUR brand?
       upper class, charming

   Ruggedness                                      (See Ch. 8 from Keller
       outdoorsy, tough                            book for more on this) 49
          Roles of Brand Personality
   Consumers affirm desired characteristics of
    themselves (actual or ideal)
       to others
       to themselves

   Brands as relationship partners

   Personality traits cue brand associations

 Products as Self Expression:
  Importance of Category & Brand

          Necessity      Luxury
Used in   category: low category: high
private   brand: low     brand: low

Used in   category: low category: high
public    brand: high    brand: high

                           Brand Loyalty
   Brand Preference
       Please rank order the following brands in terms of your preference:
        Bounty, Viva, Scott, etc.

   Purchase Intention
       What is the likelihood that you will purchase Bounty paper towels in
        the next week?
           0% 10% 20% 30%…80% 90% 100%

       How likely are you to buy Bounty paper towels in the next week?
                          very unlikely _ _ _ _ _ _ _ _ _very likely

           Another Aspect of Brand Loyalty:
           Value Perception for GatorAde
               Where is the price break?
         How Much Cheaper Would PowerAde Have to Be?

          > 70%          4.3%
            60%          4.3%
 Percent    50%                        13.0%
PowerAde    40%                 8.7%
 Discount   30%                        13.0%
            20%                                17.4%
            10%                                                30.4%
       same price               8.7%

                    0%   5%     10%    15%      20%    25%   30%   35%
                                Break Point Percentage
        Sample Requirements for Your
           Quantitative Research
   at least 10 interviews per group member
       (minimum of 40 total)

   From brand inventory, assess what types of
    incremental or updated information (e.g.,
    awareness, perceptions) is needed most
       Have to prioritize
       Cover as much as possible
       But fewer high quality questions are better than many
        low quality ones

              More about Your Research
   Probable questionnaire types
       Personal administration of written questionnaire
       Personal interviews (esp. for qualitative q’s)
       Internet or phone

   Might try to consider differences between:
       users,
       rejecters / defectors, and
       non-users

   I will be your consultant                              55
    Some Concluding Comments about
       Your Quantitative Research
   Use mostly scaled measures for quantitative surveys
       instead open-ends (hard and time cost to code)
   All measures should be relative to the competition
   Measures should capture the components of brand
    equity and the hierarchy of effects
       (Integrated Marketing Communications)

   Pretest, pretest, pretest!
        Why Pretest Questionnaires?
   Test for:
       Length of time
       Does it measure what you want?
            Ask both the client and the respondents
       Anything missing that should be asked?
       Clarity
            Unambiguous questions
            Unambiguous answers
               Why Pretest?             (cont’d)

   Test to learn:
       Variability of answers
            (Use to calculate needed sample size)

       Ease of answers

       Whether interviewers perform as trained?

                   How to Pretest
   Informal
       colleagues
       secretaries and other staff
       family
       convenience samples

   Pilot test
       individual interviews
       focus groups
               How to Pretest (cont’d)
   Debriefing pretest
       interview respondents after they have completed

   Protocol pretest
       ask respondents to think out loud as they fill out

   Revise and pretest again (and again if necessary);
       pretest until added pretests learn nothing new

  How To Measure
Importance Weights

           How To Measure
     Consumers’ Importance Weights
•   Integral to marketing and implementation of the
    marketing concept

•   Seven methods

•   Techniques help illustrate how consumers
    perceive products and make decisions

        How To Measure Consumers’ Utilities
           (Importance Weights) (Cont’d)
I. Semantic Differential Rating Scale

                          Extremely                 Extremely
                          Unimportant               Important

High Gas Mileage          1    2        3   4   5   6   7
Fast Acceleration Speed   1    2        3   4   5   6   7
Sport Styling             1    2        3   4   5   6   7
Roominess                 1    2        3   4   5   6   7
Good Handling             1    2        3   4   5   6   7

    How To Measure Consumers’ Utilities
       (Importance Weights) (Cont’d.)
II. Ranking Task

High Gas Mileage          ____
Fast Acceleration Speed   ____
Sport Styling             ____
Roominess                 ____
Good Handling             ____
etc.                      ____
etc.                      ____
…                         ____

     How To Measure Consumers’ Utilities
        (Importance Weights) (Cont’d.)
III. Fixed Sum Allocation Scales
         Allocate 100 points among the following attributes. The            number
   of points allocated to each attribute should        indicate its’ relative

                           #of allocated points
High Gas Mileage                     ____
Fast Acceleration Speed                       ____
Sport Styling                                 ____
Roominess                                     ____
Good Handling                                 ____
etc.                                          ____
etc.                                          ____
       How To Measure Consumers’ Utilities
          (Importance Weights) (Cont’d.)
IV. Trade-Off Scales

Fast Acceleration                     Low Gas Mileage
        ()        ()   ()   ()   ()   ()

Sporty Styling                        Roominess
        ()       ()    ()   ()   ()   ()

Low Gas Mileage                       Roominess
       ()       ()     ()   ()   ()   ()


How To Measure Consumers’ Utilities
   (Importance Weights) (Cont’d.)

V. Conjoint Analysis
(see notes from last spring)

How To Measure Consumers’ Utilities
   (Importance Weights) (Cont’d.)

VI.   Perceptual Maps of Peceptions
   (Infer determinance weights from spacing of
    brands of different dimensions or factors)

                 An Alternative View about
            How to Measure Importance Weights:

VII. Problem Detection Analysis
1. Importance weights often miss things that seem at face
   value to be important

2. Instead of focusing on consumers’ perceptions of
   importance, this focuses on problems
                          BBD&O Advertising Agency

 Problem Detection Analysis                       (Cont’d)

First Step:
a. Use focus groups and/or depth interviews

b. Ask about what irritates, frustrates, etc. when using
  the product

c. Accumulate a list of (50 150) problems

              Problem Detection Analysis:
          Second Step: Survey at least 200
            prospects about each problem and ask:

a. How often does it occur?   (Frequency)

b. How much does it irritate you when it
     occurs?            (Bothersomeness)

c. In your opinion, which of these items are already
   being used as advertising or product claims

            Problem Detection Analysis
                Example: Dog Food
Most Important Benefits
                          Biggest Problems
Balanced diet
                          Smells bad
Contains vitamins         No different sizes for
Tests good to dog                different dogs
Easy to prepare           Dog’s teeth dirty
                          Doesn’t chew like a

           Problem Detection Analysis:
               Example: Banking
Most Important Benefits
Modern                    Biggest Problems

Innovative                Service too slow

Friendly                  Too complicated
Low interest rates        Lines are too long
                          Can’t get a loan


To top