Survey exercise follow-up

Document Sample
Survey exercise follow-up Powered By Docstoc
					Survey exercise follow-up
Survey population
 Sample?
   Random
   Purposive
   Convenience
 Entire population
   Small, bounded population
     All the people who work here
 Non-response bias an issue in each case
Sample design
 Survey element – unit ABOUT WHICH information is
  collected; unit of analysis
    E.g., Person; household; event
 Survey universe – theoretical and hypothetical
  aggregation of elements
    E.g., Americans; users; visits
 Population – specified aggregation of survey
     Households in the US, defined as one person, or a
     collection of related or unrelated persons…as of
     March, 2002
    Visits to any web page in defined domain during
     March, 2002
Sample Design, cont.
 Survey population -- Aggregation of elements from
  which survey sample is actually selected
    Households in 50 states with listed telephone numbers
 Sampling unit – elements consider for selection at some
    listed residential phone numbers
    If multi-stage sample:
       State; telephone directory; phone numbers
 Sampling frame – list of sampling units
    Telephone directories for specified places
 Observation unit – element from which info collected
    person over 18 who either answers the phone or is
     fetched by a person under 18 answering the phone
Sample selection
 Consider possible biases of sample design
 Mixed methods may give uninterpretable samples
   E.g. mail to a group AND hand out in Sproul Plaza –
     so what population does your sample represent?
       And people on the mailing list who walk through Sproul
        have *2* chances of being sampled
 Consider practical issues
   Confidentiality often precludes getting a list, e.g.
     from the university
   If you can’t quantify units (people, events), you
     can’t sample proportionately
   Non-users or non-affiliated users (e.g., potential
     students; web users with no particular connection to
     the site) are hardest to track down
Introducing survey
 Purpose of introduction
   Justify your asking
   Induce their cooperation
   Allay possible fears
      “this is not a test…”
   Help them understand domain of survey, questions
 Describing purpose of survey
   Enough to gain cooperation, not enough to bias
      E.g. “a study of compensation in this field,” not “a study of
       whether women’s salaries lag behind men’s in this field”
   SHORT but not at the expense of clarity.
Getting cooperation
 Worthwhile topic, survey
   Not threatening
   Not a waste of time
   Interesting to them
 Good design
   If you are not credible, not worth their
 Short and easy to answer
 Short and clear
 People skim!
 The more you ask of them, the more likely
  they are to mess up
   “please rate from 1, most important, to 7, least
 Give instructions
   For skip patterns
     “Did you use the library today? If not, go to
       question 3. If yes,…
   For what to do if question does not apply
Choice of questions
 Think through: what will you do with
  the answers?
 If you don’t know, don’t ask
 If you do know, word in a way that
  helps you
 E. g. “Have you ever used a library?”
   What difference will it make if they
    answer “no”?
Choice of questions
 Respondents must be qualified to answer
   “What would be the best way to bring peace to
    the Middle East?”
 Respondents must think the topics of the
  questions are:
   Not invasion of privacy
   Not “dumb questions”; worth their time
   Not a test
     Of them
     Of some idea or proposal they may not like
      (unless you mean it)
        “if we had to cut the budget, should we lay people
         off or reduce salaries?”
Time dimension to questions
 General, usual, most often:
   What do you usually do?
 Specific
   What did you do today?
   What have done in the last month?
 NOT:
   What do you use this site for? (today? Ever?)
 Memory
   What’s important to you may not be to them
     “what did you do the last time you used this site?”
   Time and frequency are hard to remember
     “Approximately how many times in the last month…”
  Question wording
 Resolve any possible ambiguity!
    Definitions, illustrations, examples
       Have you ever used a library? What’s a library?
       “Did you use the library’s homepage?”
          Which page is that?
       What proportion of your income do you pay for housing?
          Individual or household income?
          Housing: rent or mortgage; taxes? Insurance?
 You cannot use “etc.” unless the reader can complete the
    NO: “borrow library books, ask ref questions, etc.”
    YES: “borrow library books, ask reference questions, or
     otherwise use the library’s resources or staff, in person or
Use parallel construction
 What did you use this site for today?
   Searching for information
   Placing an order
   Checking a previous order
   Searched for information
   I wanted to check on an earlier order
 What did you use this site for today? If
   What was the question to which “yes” was the
 Closed-ended are easier (which is usually,
  but not always, a good thing)
   Easier for users (recognition, not recall; less
     They are less likely to think of different choices
   Easier for you to summarize
 Be as exhaustive as possible with choices
   If you leave out an answer that a user might
    reasonably give, they may or may not think to
    add it
   They may become confused about the question
    or the survey
Answers, cont.
 Allow degrees of answers
   Yes/ No, always/never are rarely the case
   E.g., Usually; Sometimes; Rarely
 Allow multiple dimensions of answers
   How satisfied are you with your visit today?
     Very: found what I was looking for.
     Not very: did not find..
     What if I found what I needed but with
      difficulty? I may have been successful but am
      dissatisfied with the experience.
Answers, cont.
 Give clear instructions
   Check one? Check the one that best describes…?
    Check as many as apply?
   Define!
 Lay out clearly
   E.g. Ordinal scales all on one line
     How satisfied are you?
         Very satisfied   somewhat satisfied   not at all
         OR
          Very             somewhat      not at all
          Satisfied        satisfied     satisfied
      NOT: How satisfied are you? Very satisfied
       Somewhat satisfied Not at all satisfied
Ranges and intervals
 Want answers to be easy to for your reader
  to interpret
 Generally, use either:
   Intervals of equal size
   OR Intervals reflecting meaningful groupings
        Age: under 18   18-25    25-40 40-55…
   OR intervals get larger when you expect fewer
        How many years have you worked here?
         Less than 1   1-3 4-10    10-20 more than 20

 Avoid:
   Years worked here
        1 year                      10%
        2-5 years                   15%
        6-7 years                   20%
        8-15 years                  50%
 Have you checked out a book from
  the UC Berkeley Library system?
   Yes   no
 If you use Pathfinder, Gladis, or
  Melvyl to find books or periodicals,
  where do you use them?
   Terminal at main library
   Terminal at my departmental library
   I don’t use these services
 If you could allocate funds for more salary and had
  to pick from the following places, how would you
  prioritize where you would obtain these funds?
  (1 – first choice to find money through 7 – last place
     you would go)
   tuition
   Pension plan
   Taxes
   Decreased health coverage
   State bonds
   Fund raising
   Less vacation/sick leave
 How long has your library offered live
  online reference service?
 How long have you been a Reference
  Librarian? (providing reference services
  comprises at least 40% of your day)
     Up to 5 years
     Between 5 and 10 years
     Between 10 and 15 years
     Over 15 years
 Which part(s) of this web site
  interests you the most?
 Which part(s) of this web site do you
  find the most uninteresting?
 What parts of this web site do you
  like the most?
 Which parts of this web site do you
  dislike the most?
 1. Have you ever used the following online
   [list]
 2. If you checked any box in question 1,
  how important do you think online help
  would be in each of these activities? (only
  respond as applicable)
   [Item]
     Absolutely unimportant |--|--|--||--|--|--| absolutely important

Shared By: