Introduction to Sample Surveys

Document Sample
Introduction to Sample Surveys Powered By Docstoc
					            Introduction to
            Sample Surveys


政府統計處
Census and Statistics Department
                              Sample Surveys

 Sample survey has widespread applications,
  ranging from surveys conducted for business
  purposes to collection of data for use in public
  affairs and social studies.

 In order to ensure the reliability of survey
  results, it is essential that great care in
  addition to adequate professional knowledge
  be applied in the planning and conduct of
  surveys and in the analysis of data.
   政府統計處
   Census and Statistics Department
           Major Steps in Conducting
               a Sample Survey
   Overall planning
   Design and selection of sample
   Design of questionnaire
   Collection of raw data
   Compilation and analysis of statistics
    and dissemination of survey results



    政府統計處
    Census and Statistics Department
                             Overall Planning

 Clearly defined survey objectives
 Clearly defined target population
 Clearly defined statistical standards
  (definition of terms, counting rules,
  classification, statistical methodologies)
 Respondents should be able to provide
  the information required.
 Adequate resources such as manpower
  and time must be available.
  政府統計處
  Census and Statistics Department
 Design and Selection of Sample

 Sampling deals with the selection of a
  number of elements, i.e. a sample.
 A sample can be drawn using different
  methods. A probability sample, with
  each element having a known, non-zero
  chance of being included, should be
  used as far as practicable so that
  scientific inference can be drawn from
  the survey results.
  政府統計處
  Census and Statistics Department
 Design and Selection of Sample

 Non-probability sampling methods, such
  as haphazard sampling, should be
  avoided. Such methods are not scientific
  and bias usually exists in survey results.
 A complete and up-to-date sampling
  frame should be acquired.




  政府統計處
  Census and Statistics Department
 Design and Selection of Sample

 In case of unequal probability of
  selection, it is necessary to ensure that
  proper weighting methods are applied to
  survey results.
 Using established statistical
  methodology to compute the required
  sample size.
 Once the sampling units are selected,
  alterations are not allowed.
  政府統計處
  Census and Statistics Department
                       Common Probability
                        Sampling Methods
   Simple random sampling
   Systematic sampling
   Stratified sampling
   Clustering sampling
   Multi-stage sampling




    政府統計處
    Census and Statistics Department
Non-probability Sampling Methods

 Selection of elements is based on
  subjective judgment and experience, but
  not a random manner
 Common non-probability sampling
  methods
  Street interview/mall intercept
  Quota sampling
  Respondent-initiated telephone polling
  Internet survey
  政府統計處
  Census and Statistics Department
Non-Probability Sampling Methods

 Why unreliable?
  Scientific inference cannot be drawn from
   the sample data (results are confined to
     describing the group of respondents, but cannot be
     extracted to the entire population)
  Level of precision of the estimates cannot be
   scientifically assessed
  Biases will most likely exist in the survey
   results

  政府統計處
  Census and Statistics Department
Non-Probability Sampling Methods

 Why unreliable (cont’d)?
  No way to know how exactly the group of
   respondents is formed
        • In street interviews, the interviewers tend to
          select those “friendly faces” for interview.
        • In self-selecting polls, only those who have
          strong views on the survey may volunteer to
          participate.
  The results may be inaccurate and
   misleading, hence not of much use.

  政府統計處
  Census and Statistics Department
             General Principles in
           Designing Questionnaires
 Questions should be relevant to the
  survey objectives
 Questions should be arranged in a
  proper order
 Use screening question to enhance the
  flow of the questions.
 Use an appropriate language
 Clear instructions
  政府統計處
  Census and Statistics Department
             General Principles in
           Designing Questionnaires
 Questions wordings should be
  appropriate, specific and precise
 Avoid leading questions or questions
  being loaded in favour of a particular
  response
 Avoid difficult vocabulary
 Avoid composite and double negative
  questions
 Beware of memory error
  政府統計處
  Census and Statistics Department
             General Principles in
           Designing Questionnaires
 “Don’t know/No opinion” should be
  included as appropriate
 Long questionnaires are undesirable.
 Questionnaires should be tested on
  some prospective respondents before
  finalized.



  政府統計處
  Census and Statistics Department
                 Collection of Raw Data

 Methods of data collection for surveys
  include:
  Self-administered questionnaires by mail
  Personal interviews
  Telephone interviews
  Computer Assisted Telephone Interviewing
   (CATI)
 Very often, mixed modes of data
  collection can be used.
  政府統計處
  Census and Statistics Department
                 Collection of Raw Data

 An appropriate mode should be selected
  by carefully considering respondents'
  willingness to co-operate, the degree of
  complexity of the subject of enquiry and
  other relevant factors (e.g. practicability
  of using personal or telephone
  interviews).



  政府統計處
  Census and Statistics Department
     Comparison of Different Data
        Collection Methods
Characteristics               Personal     Telephone         Mail
                              interview     interview    questionnaire
Cost                            High        Medium           Low
Manpower                        High        Medium           Low
Time consuming                  High        Medium           Low
Non-response/                Relatively    Medium to         High
non-contact rate               lower          high
Suitability for scattered     Costly       Less costly     Medium
population
Interviewer bias                Yes           Yes             No
Response quality               High         Medium           Low
Length of questionnaire       May be       Preferably     Preferably
                              longer         short          short
Design of questionnaire     May be more     Simple         Simple
                            complicated
Asking embarrassing         Not suitable     Easier         Easier
questions
                 Collection of Raw Data

 Interviewers should be trained before
  they start working and closely
  supervised during fieldwork to ensure
  their quality of work.
 Identity and information supplied by
  individual respondents should be kept
  confidential. The survey results are to be
  presented in the form of aggregate
  statistics.
  政府統計處
  Census and Statistics Department
                 Collection of Raw Data

 Every effort should be made to achieve a
  high response rate (or reducing the
  number of non-responses).
 Methods to reduce non-responses:
  Keep the questionnaire brief and concise
  Consider rewards to respondents
  Better publicity measures (e.g.
    advertisement, advance letters)
  Assurance of confidentiality of individual
    data
  政府統計處
  Census and Statistics Department
                 Collection of Raw Data

 Methods to reduce non-responses         (Cont’d):
  More experienced interviewers, especially
   when handling refusals
  Visit households in evening time
  Visit at different time of different days
  Increase the number of re-visits/call backs
  Use self-administered questionnaires in case
   of non-contact


  政府統計處
  Census and Statistics Department
Compilation and Analysis of Statistics

 Data should be carefully and thoroughly
  checked before compilation .
 Appropriate statistical methodology
  should be adopted in compiling and
  analysing data.




  政府統計處
  Census and Statistics Department
            Reporting and Assessing
            the Reliability of Surveys
 To enable readers to make judgment on
  whether the findings are credible, a good
  survey report should include:
  Sponsorship of the survey
  Population covered
  Sampling method
  Mode of data collection
  Time period of data collection
  Wording of questions
  政府統計處
  Census and Statistics Department
            Reporting and Assessing
            the Reliability of Surveys
 A good survey report should include
  (cont’d):
  Sample size and response rate
  Point estimates and confidence intervals (if
   possible)
   Likely sources of non-sampling errors
 Information supplied by individual
  respondents should not be disclosed.

  政府統計處
  Census and Statistics Department
            Reporting and Assessing
            the Reliability of Surveys
 One may access the reliability of a
  survey by asking the following
  questions :
  Has probability sampling methods been
   used ?
  Is the sample size reasonably large ?
  Is the questionnaire design proper ? Any
   leading questions or wordings ?


  政府統計處
  Census and Statistics Department
            Reporting and Assessing
            the Reliability of Surveys
 One may access the reliability of a
  survey by asking the following questions
  (cont’d):
  What is the interviewing method ? Any
   improper influence by interviewers or third
   parties during the interview ?
  Is the response rate too low ?
  Are the sampling errors of acceptable
   magnitude ? How about non-sampling
   errors and biases ?
  政府統計處
  Census and Statistics Department