Narayanan_Giambo_Fry_Crafts

Document Sample
Narayanan_Giambo_Fry_Crafts Powered By Docstoc
					      Using the Web
            for
Surveys of Medical Providers
                   Vasudha Narayanan
                       Presented at
Third International Conference on Establishment Surveys
                      June 21, 2007




                                                          1
        Acknowledgements

Co-Authors
 Stephanie Fry, Jennifer Crafts
  & Pamela Giambo from Westat

Project Officers
 David Clark, William Taylor, Gladys Valentin
   & Mei Wang from Centers for Medicare &
   Medicaid Services
  This material was prepared by Westat Inc., under contract with the
  Centers for Medicare & Medicaid Services (CMS), an agency of the U.S.
  Department of Health and Human Services. The contents presented do
  not necessarily reflect CMS policy


                                                                   2
                 Overview
•   About the Surveys
•   Survey Methods
•   Questionnaire Design Issues
•   Sampling and Contacting Providers
•   Completes by Mode
•   Factors Affecting Web Response
•   Conclusions


                                        3
     About MCPSS & QIO Survey
• Two Surveys for Centers for Medicare &
  Medicaid Services (CMS)
    – QIO Provider Survey (2004)
    – MCPSS (2006 & 2007)
•   National Sample of Medicare Providers
•   Satisfaction Surveys
•   Approx. 15 minute survey
•   Multi-Mode Data Collection

                                            4
            Survey Methods
• Web with telephone follow-up; paper on
  request
• Telephone and paper surveys to:
  – Increase response rates
  – Reduce potential bias
• Importance of:
  – Screening
  – High quality mail outs
  – Professional Interviewers

                                           5
      Survey Methods (cont.)
• Field Period (12-16 weeks)
  – Screener
  – Notification of web survey
  – Reminder letters/e-mails
  – Hard copy questionnaire mailing/fax
  – Telephone non-response
  – Response rates (AAPOR-RR3)
    • QIO: 90%
    • MCPSS: 65%


                                          6
         Questionnaire Design
                 Usability Testing



• Purpose of Testing
  – Ease of navigation
  – User satisfaction and perceived burden
• Profile of Testers
• Response mode bias



                                             7
          Questionnaire Design
              Usability Testing Issues


• Allow for multiple respondents
  – Clarity of instructions to “designate a
    colleague”
• Clarity of instruction for submitting survey
• Ability to skip sections and questions




                                                 8
  Elements of the Final Design
• Allow multiple respondents for each
  survey
• Track section-level designees
• Track section-level status
• Crisp and easy to navigate web survey
• Seamless link between CATI & Web




                                          9
10
11
            Completes by Mode
           Web        Mail/Fax* Telephone Completes
2004      32.2%          21.9%            45.9%             17,628
2006      40.1%           1.7%            58.1%             16,121
2007      55.1%           0.9%            43.9%             17,747

       *non-response mail out of hard-copy survey in 2004

• Highest Rates of Web response – Hospitals
• Lowest Rates of Web response – Physicians


                                                                     12
Factors Affecting Web Response
• Percentage Web access
  – 93% physician practices (2007)
  – 97% other providers (2007)


Hypotheses:
• Ease of Web access at work station
• Ease of survey website navigation
• Ability to obtain help when needed

                                       13
              Conclusions
• Feasibility of a Web surveys no longer an
  issue
• Necessary factors for survey management
  and achieving high response rates
  – telephone non-response follow up essential
  – Frequent interface between data collection
    modes
  – Allow the respondent maximum flexibility
    while maintaining survey integrity

                                                 14
       Contacting the Authors
Vasudha Narayanan
  (vasudhanarayanan@westat.com)
Stephanie Fry (stephaniefry@westat.com)
Jennifer Crafts (jennifercrafts@westat.com)
Pamela Giambo (pamelagiambo@westat.com)




                                              15
Thank you




            16

				
DOCUMENT INFO