About ORC Macro by sanmelody


Are Incentives Really Needed?

National AAPOR Conference
Anaheim, CA

May 18th, 2007

James Dayton, MBA; Daniel Gundersen, M.A; Cristine Delnevo, PhD,
Zi Zhang, MD, MPH; Lori Westphal, PhD, MPH; Michelle L. Cook,
MPH; Susan J Cummings, BSN, CPHQ; Diane Aye, MPH, PhD;
Randal ZuWallack, MS; Naomi Freedner, MPH; Ruth Bernstein, BA
Cell Phone Surveys
   The potential to measure the cell phone population,
    particularly cell-phone only, is still relatively unknown.
   There are advantages to interviewing respondents on cell
     o Reaching a new generation—many adults, particularly
        younger adults, are opting to use cell phones as their
        primary communication medium
     o As a personal device, cell phones provide a direct
        communication link to respondents at virtually any time
        of day.
   Legislative and ethical
      o Laws prohibiting the use of predictive dialing means that all numbers must
          be manually dialed
             • Labor intensive and increased likelihood of dialing errors
      o Incoming calls to cells often eat-up minutes and can result in real financial
          costs to participants
      o Sense that cell phones are ‘private’

   Usage, privacy and safety
     o Unavailable for a survey (driving?, working?, public places?)
     o Respondents are not necessarily located where their area code and
        exchange suggest they are
     o Inbound calls to cells are often use plan minutes or have a definable cost to
     o Increasing risk of backlash as volume of unsolicited calls increases

   Sampling methods are relatively crude in comparison to contemporary RDD
     o List assisted methods are not applicable since there is no directory of cell
         phone numbers
     o Resort to traditional equal probability samples of cell phone numbers
            • With restrictions prohibiting automated dialing of cell phone numbers,
              sampling efficiencies are all the more important.
Study Specifics
   Participating States:
      o Massachusetts, Connecticut, New Jersey, Florida,
          Montana, Texas
      o Questionnaire:
            • Shortened BRFSS core questionnaire that had been
               piloted in other states
            • Survey introduction included IRB-approved informed
               consent, security/privacy, and a safety provision
            • Average interview length - ~10 minutes
   All records must be manually dialed
   Incentive / compensation offered in Massachusetts only
      o $15 in compensation to all contacts.
      o No offer of compensation in other states
   Sample Design:
      o Purchase randomly generated cell numbers based on
          assigned cell exchanges and 1,000 blocks from Genesys
   Methodology:
      o Fielding dates: Late October 2006 – February 2007
      o Target calling times: between 9:00 am and 9:00 pm on
         weekends and between 7:00 and 9:00 pm on
            • Generally, only requested call-backs were be made
              between 9:00 am and 7:00 pm weeknights.
   Minimum of 6 attempts on non-terminal records
      o Two weeknight, four weekend attempts over at least a
         sixteen day calling period
   Leave message on Voicemail on 1st and 5th attempts
   Must be an adult to complete survey
   No household randomization – assumed cell phone was not
   Completed interviews with both cell only households and
    cell/landline households
   Single refusal policy
Completed Interviews
   Original completed interview target per state – up to 500

     o Actual completed interviews:
        • Connecticut:    352
        • Florida:        320
        • Massachusetts: 302
        • Montana:        389
        • New Jersey:     300
        • Texas           298
   Identification of control
      o We elected to compare MA to NJ:
           • Both states demonstrate similar response
             characteristics in traditional BRFSS surveys, and
           • MA and NJ had a similar ratio of refusals before or
             during the introduction relative to the number of
             respondents who continue past the intro
                ▫ 2.75 in MA vs. a 2.65 ratio in NJ.
   Conclusion with limited evidence – incentive did not impact
      o The completion rates are 41.7% and 40.8% for MA and NJ,
   60 out of the 302 respondents completed the survey, but did
    not leave a name or address for receipt of the incentive.
Sample Usage
Interviewing Costs

* Assumes 20 Minute average interview length for BRFSS
Cell Phone Only?
State-Level Sample Targeting
Safety Concerns?
Will the Boss be Impressed?
•   Are incentives needed?
       o The limited scope of the research suggested they are not… yet
   What if incentives are not offered?
       o Ethical concerns
       o IRB concerns
       o Will the impact of incentives change as the number of unsolicited
          calls to cell phones increase?
   If incentives are offered:
       o Practical means of transferring incentive that does not require the
          respondent to reveal a great deal of personal information
            • PayPal???
   Respondents not terribly concerned about safety – do we need to be
    more concerned for them?
   Laws prohibiting use of auto-dialer adds substantially to costs
   Even with incentives, cell surveys might be the most cost-effective way
    to increase response rates among young adults, especially males.
Contact Information
James Dayton

To top