Hard to Reach Groups in Household Travel Surveys - TRB Travel

Document Sample
Hard to Reach Groups in Household Travel Surveys - TRB Travel Powered By Docstoc
					 Hard to Reach Groups
 Household Travel Surveys
Nancy McGuckin
Heather Contrino
Sub-Committee on Household Surveys
TRB 2012
Why are we at a critical juncture?
• Move away from RDD as a sample frame, move
  toward ABS

• ABS allows for targeted sampling BUT is more
  dependent on opt-in

• Changing sample frame shifts the focus from non-
  coverage back to non-response bias

• Incorporation of new technologies heightens
  concerns about bias
RDD issues as a sample frame:
 ▫ Inefficient sample: Working blocks diluted by the break-up
   of Bell

 ▫ Geographic uncertainty: Portability de-couples phone
   numbers and geography

 ▫ Non-ignorable coverage issues: Cell phone only
 Non-coverage in RDD samples is too
 high and non-random (non-ignorable)
 Percent Cell-Phone Only:

Source: CDC 2010 at:
 Address-based samples (ABS) are
 returning in popularity in US:
The United States Postal Service (USPS) Delivery
 Sequence File (DSF) is available:

  ▫ In combination with geocoded addresses very accurate

  ▫ Cell-only response proportion is encouraging

Still some challenges, e.g. Issues of coverage, rural
  routes, and multiple drop boxes must be understood
Benefits of address-based samples to
target hard-to-reach groups:
• Multi-mode methods of data collection can increase
  response rate, coverage

• ABS provides a natural and efficient framework for
  design and implementation of multi-mode surveys

• Areas with known non-respondents can be targeted
  for over-sampling:
    renters, low income, minority and/or immigrant
      populations, universities, high-income, etc.
      A multi-modal approach with address-based
      sampling generally increases response rates
      compared to RDD
                                                      RDD   ABS Multi-modal
     Percent Completed Interviews



                                         Total   CA    FL       MA               MN   SC           TX
Source: “A Comparison Of Address-based Sampling (ABS) Versus Random-digit Dialing (RDD) For General Population
Surveys”, Link, et al. Public Opinion Quarterly, Vol. 72, No. 1 2008, pp. 6–27
   ABS challenges at the national level
   include:                     A city style address had
                                                                                                                 127 times the response
      • A lot of variation in                                                                                     compared to all other
        telephone match rates by                                                                    40

        area1                                                                                                         City-Style   PO Box

          (e.g. 66 percent in North Carolina to 52
          percent in California)

                                                                       Pecent Completed Interview
      • Address style correlates with
        response rates (see graph)2                                                                 20

      • Compared to RDD samples,
        ABS significantly over
        represents non-Hispanic
        whites and people with
        higher levels of education2                                                                 0
                                                                                                         Total   CA   IL     NJ      NC     TX   WA
1 Jill M. Montaquila, Westat, at:
2 “A Comparison Of Address-based Sampling (ABS) Versus Random-digit Dialing (RDD) For General Population Surveys”,
Link, et al. Public Opinion Quarterly, Vol. 72, No. 1 2008, pp. 6–27
Continuum of response/non-response
(Fixed Response Model)

    Respondents:            Try Harder                Persistent Non-
    • Yay!                   (& NRFU):                  Responders:
                            • Too busy                •   Privacy extremists
                            • Privacy                 •   Paranoid of
                              concerns                    government
                            • Saliency                •   Off the grid

                            Special Challenge:
                      Literacy/Language Issues and
        understanding cultural differences as immigration increases
Each population has an inherent
probability of response:
Stochastic Model

                            Young         African-
                     Low    People        American
Hispanic           Income

                        Renters         Multiple
        Immigrants                        -Job

                        High         Families with
       Larger                          Children
  A Thousand Markets of One, e.g.
  Different strokes for different folks:
      Survey element:       Young (18-24)           Chinese                                Hispanic
                                                                    Worried about Worried about time
                           Mentioned money/
                                                   Mentioned      time commitment, commitment but
          Incentives       remuneration first
                                                    money         mentioned money    wanted to give
                                                                      afterward    opinion/"be heard'
                                                   community--    Prefer 'personal'
                                                                                      Like formal letter in
      Contact Preference     Letter or text     important to have nature of phone
                                                introduction from        call
                                                  trusted source

          Materials               Lack of connection between diary information and transportation

             GPS               Like/prefer         Did not like       Like/prefer         Did not like

     Trust of Government
                              Trusted gov't      Did not trust—     Did not trust        Trusted gov‘t
                                sponsor          afraid of scam     gov't sponsor          sponsor

Source: NYMTC NRFU one-on-one interviews conducted by PTV/NuStats for NYMTC
Where to start?
 For each study area:

    Geocode sample addresses

    Define hard-to-reach populations

    Estimate the importance (bias)—e.g. in NYMTC
     Russian immigrants were a concern

    Use census or secondary data sources (Axiom) to
     identify areas with hard-to-reach populations

    Build over-sampling/out-reach and alternate
     approaches into survey design
What more can we do?
We can do a lot!
•   Nonresponse research
•   Targeted outreach
•   Changes in design and methods
•   Statistical methods

                               Get Out of the Box and
                                 in the Real World
Nonresponse Research
• Traditional Research for estimation and
 ▫ Subsample nonrespondents (not just who, but
   how much different)
• New research to understand nonresponders
 ▫ It has been quite awhile since we really looked at
   what the likelihood of responding
• Message Testing
      Nonresponse does not mean not
                                       It is the initial
Targeted Outreach                    contact that hurts
                                         us the most
• Target Metro Areas
 ▫ Door to door, mailings, advertising
• Social Media
 ▫ 50% of US population has a facebook page –
   Internet World Stats
• Community Outreach
 ▫ Hard to reach populations (main study and
• Targeted Messaging
 ▫ In research by Groves, 14% increase in response
   on topic saliency alone
Updated Designs and Methods
• Variable Incentives
• Multiple data collection modes
 ▫ More than 4 in 5 adults have a cell phone (82% up
   from 65% in 2004 – Pew Research)
 ▫ 78.3% Internet penetration rate – Internet World
• Use of technology
 ▫ We use what they use
 ▫ More investment in passive data collection
• Longer term studies, panels, etc.
 ▫ Get them and keep them
• Use other data sources
• Prioritize information and reduce burden
Statistical Innovation
• A New Day – the methods need to dictate the
 ▫ Combining Frames
 ▫ Combining Methods
 ▫ Combining Data Sources

                                 Can’t the
                            figure something
In Summary
•   We know who are nonresponders are
•   We usually know why they aren’t responding
•   Frames have become better and more accessible
•   There are many strategies for improving the
    quality of our data
What more can we do?
• Look outside of Transport Research for ideas
     National Household Education Surveys Program
      Redesign (NHES)
     Behavioral Risk Factor Surveillance System (BRFSS)
• Invest in Research
• Encourage advancements in statistics and methods
• Get out of our box
A new approach/paradigm
 How about a volunteer population for mobility

 Sensing devices + prompted recall
   Create and maintain a relationship

   Offer something valuable to participants in return

   Sample like a large panel for represenation
Old View of Respondents:
New paradigm of respondents:
   Thank You!

Nancy McGuckin
Heather Contrino

Shared By:
yan tingting yan tingting