How to Design a Survey Instrument

Document Sample
How to Design a Survey Instrument Powered By Docstoc
					 How to Design a
Survey Instrument

     Auden D. Thomas, Ph.D.
   Stephanie L. Wehnau, M.S.

   Center for Survey Research
        Penn State Harrisburg
      The Process of Survey Research

             Define research objectives

   Choose mode                        Choose
    of collection                  sampling frame

Construct and pretest             Design and select
   questionnaire                       sample

               Design and implement
                  data collection

                    Code and edit data

           Make post-survey adjustments

                     Perform analysis
1. Topic Mapping
Topic Mapping
   Start with the research objectives

   List the topics to be investigated

   Translate topics into questions (measures)
Topic Mapping Example
   Questionnaire that evaluates alcohol use at Penn
    State Harrisburg (PSH) for Student Health Services:

       To what extent are students using alcohol at PSH?
       What beliefs exist about alcohol use at PSH?
       What do students know about the negative affects of
        alcohol use?
       What are the demographics of survey respondents?
2. Writing Questions
Guidelines for
Survey Design
   Mode (paper, phone, web, face-to-face)
   Keep your instrument short (respondent burden)
   Non-threatening questions to more sensitive
    topics (build rapport)
   Ask demographics last
   Ask the same question different ways (validity)
   Adopt or adapt Q’s (Census)
Types of Questions:
Open-ended Questions
   Respondents have the opportunity to provide
    an answer of their own.
   May be more difficult to analyze given the
    extensive amount of information that can be
   Examples:
       What do you think is the most important problem
        facing Pennsylvania today?
       What is your age?
Types of Questions:
Closed-ended Questions
   The respondent is asked to select an answer
    from among a list provided by the
   Widely used because they provide a greater
    uniformity of responses and are more easily
   Select one response or “Select all that apply”
Types of Questions:
Closed-ended Questions
   “Select one” Example:
       In general, how satisfied are you with the way
        things are going in Pennsylvania today?
           Very satisfied
           More or less satisfied
           Not satisfied at all
           Don’t know
           Declined to answer
Types of Questions:
Closed-ended Questions
   “Select all that apply” Example:
       What type of insurance do you have? Please
        select all that apply.
           Employer-based health insurance
           Medicare
           Government-provided insurance
           Military health care
           Purchased health insurance
           Other insurance
           Don’t know
           Declined to answer
Types of Questions:
Partially Closed-ended Questions
   The respondent can select from one of the
    response options or can supply his/her own
    response in an “other” category.
   Example:
       Which of the following is your favorite college
        men’s sport?
           Football
           Basketball
           Baseball
           Other: ___________
Writing Questions

     General Tips
Avoid ambiguous words
   Poor:
       When did you move to Pennsylvania?
           Right after I finished college.
           When I was 20.
           In 1998.

   Better:
       In what year did you move to Pennsylvania?
Define terms
   Don’t leave it up to the respondent to
    interpret a term – provide a definition so that
    everyone is answering the same question.
   Example:
       A drink of alcohol is considered 1 can or bottle of
        beer, 1 glass of wine, 1 can or bottle of wine
        cooler, 1 cocktail, or 1 shot of liquor. During your
        current pregnancy, how many days per week
        have you had at least one drink of any alcoholic
Avoid abbreviations
   Do not assume that your respondents will be
    familiar with abbreviations.
   Example:
       Are you a PennDOT Business Partner registered
        and qualified in ECMS?
       Many respondents did not know what ECMS was,
        so we added this definition:
           Engineering and Construction Management System
            (ECMS): It is the way of doing business electronically
            with PennDOT.
Ask one question at a time
(avoid double-barreled questions)

   Poor:
       Do you like apples and oranges?
       Do you want to be rich and famous?

   Better:
       Do you like apples?
       Do you like oranges?
       Do you want to be rich?
       Do you want to be famous?
Use concrete questions
   Be specific (with definitions, time frames,
    locations, etc.).
   Poor:
       How would you describe your health?
       How often do you exercise in a typical week?
   Better:
       In the past three months, how would you describe
        your health?
       How often did you exercise during the past week
        (start with today’s date and count back 7 days)?
Use complete sentences
   Poor:
       Place of residence?

   Better:
       What is the name of the city where you currently
Beware of questions that elicit
socially desirable responses
   Respondents may give false answers
    because they are embarrassed or feel bad
    about their answer.
   Examples:
       Do you view sexual material on the Internet?

       Have you ever smoked marijuana?
Avoid pre-disposing
   Remain neutral – don’t guide your respondent’s to
    an answer.
   Example:
       More people have seen the movie, Gone with the Wind,
        than any other motion picture. Have you seen this movie?

       Newspapers and television started talking about patient
        safety and similar problems in our healthcare system about
        five years ago. Do you think that, in the past five years,
        patient safety has gotten better, stayed the same, or gotten
Avoid biasing words
   Remain NEUTRAL!
   Feedback during an interview (in person or
    via phone) can also bias a respondent.
   Poor:
       Organs for transplant can be bought and sold on
        the black market in the U.S.
   Better:
       Organs for transplant can be bought and sold
        illegally in the U.S.
Avoid negative questions
   Negative questions can be confusing!
   Examples:
       The United Nations should NOT have more
        authority to intervene in military affairs.

       Our Diocese today has not developed innovative
        ministries for small and rural communities.
3. Writing Responses
Creating Response Sets
   Mutually exclusive and exhaustive categories
   Appropriate number of options
   Appropriate range of options
   Consider rotating response options (if
        4. Activity:
Question writing exercise!
 Auden D. Thomas, Ph.D.

Stephanie L. Wehnau, M.S.
    Assistant Director

Center for Survey Research
  Penn State Harrisburg
 777 West Harrisburg Pike
   Middletown, PA 17057