Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out

Checklist

VIEWS: 21 PAGES: 34

									Survey Design Checklist




Appendix C

Survey Design Checklist
This document should be used as a checklist to ensure that all aspects of the survey
design process have been addressed.

In most cases, a short answer will suffice to demonstrate that the particular point has,
indeed, been addressed. In other cases, more detailed information may be provided,
or attached, and this checklist may then serve as a useful shorthand form of survey
documentation. It may also be used as a guideline for writing more comprehensive
documentation for the survey.

Where appropriate, section references are given to the text on Travel Survey Methods
(Richardson, Ampt and Meyburg (1995), Survey Methods for Transport Planning,
Eucalyptus Press: Melbourne) to help you understand what is required to be
answered in this Checklist.




                                                                                      1
Survey Design Checklist



Section 1            Preliminary Planning
Administrative Details of the Survey

The name of the survey?



Who sponsored the survey?




Who designed the survey?




Who collected the survey data?




Who analysed the survey data?




Was there an advisory committee? YES    NO 

If so, who was on the Committee?




Dates and duration of the survey?



                                                2
Survey Design Checklist
Overall Study Objectives (s2.1)

What were the objectives of the project to which this survey contributed?




Why was a survey needed?




Specific Survey Objectives (s2.2)

What were the specific objectives of this survey?




                                                                            3
Survey Design Checklist
Review of Existing Information (s2.3)

What prior information was available?




What secondary information was available for sample expansion?




Formation of Hypotheses (s2.4)

What specific hypotheses, if any, were to be tested?




                                                                 4
Survey Design Checklist
Definition of Terms (s2.5)

What definitions are being used by the survey team for key items such as:

trip, household, mode, income etc. (as relevant to the specific survey)?




Determination of Survey Resources (s2.6)

What time was available for completion of the survey?




How much money was available for the survey?




What people were available to work on the survey?




                                                                            5
Survey Design Checklist



Section 2            Selection of Survey Method
Selection of Survey Time Frame (s3.1)

Was the survey cross-sectional or time-series (and why)?

Cross-sectional                    Time Series 

Why?




Selection of Survey Technique (s3.2)

What methods were considered for the survey technique?




What testing was performed on the different methods?




What method was finally selected (and why)?




                                                           6
Survey Design Checklist



Section 3            Sample Design
Definition of Target Population (s4.1)

What was the population for the survey?




How was this population defined and identified?




Sampling Units (s4.2)

What unit was used for sampling?




Sampling Frame (s4.3)

What sampling frame was used?




Where was the sampling frame obtained from?




How was the sampling frame obtained?




                                                  7
Survey Design Checklist

Why was the sampling frame first compiled?




How did the sampling frame perform in term of:

       accuracy



       completeness



       duplication



       adequacy



       up-to-dateness



Sampling Method (s4.4)

What sampling methods were considered?




What sampling method was finally chosen (and why)?




                                                     8
Survey Design Checklist

Was the selected sample representative of the population?




       If not, how will this be corrected later?




What was the specific sampling procedure (full details)?




                                                            9
Survey Design Checklist
Consideration of Sampling Bias (s4.5)

What sources of sampling bias were considered?




How serious were these biases considered to be?




What steps were taken to overcome these sources of bias?




                                                           10
Survey Design Checklist

Sample Size and Composition (s4.6)

What was the final sample size?



What stratifications were used in the sample design?




How was the sample size calculated?




       what were the key variables considered?




       what was the variability of these variables?




       what confidence limits were used?




       what levels of confidence were used?




                                                       11
Survey Design Checklist
Estimation of Parameter Variances (s4.7)

How are parameter variances to be estimated in the data analysis?




Conduct of Sampling (s4.8)

What procedure was used in selecting the sample?




Was random sampling used at all stages of sampling?




                                                                    12
Survey Design Checklist



Section 4             Survey Instrument Design
Question Content (s5.3)

What types of information are being sought in the survey?




Trip Recording Techniques (s5.3.5)

How are trips and activities being sought from respondents?




Physical Nature of Forms (s5.4)

What is the physical nature of the survey forms?




       what paper size and weight was used?

       what colours and printing methods were used?



                                                              13
Survey Design Checklist
Question Types (s5.5)

What classification questions were asked?




       where did the classification categories come from?




What attitude questions were asked?




       what testing was performed on the attitude scales?




                                                            14
Survey Design Checklist
Question Format (s5.6)

Which questions were asked as open questions (and why)?




Which questions were asked as closed questions (and why)?




      where did the closed question categories come from?




                                                            15
Survey Design Checklist
Question Wording (s5.7)

How has the question wording been tested for:

       simple vocabulary     YES 

       words appropriate to the audience    YES 

       length of questions   YES 

       ambiguous questions (get someone else to read them)   YES 

       double-barrelled questions   YES 

       vague words YES 

       loaded questions      YES 

       leading questions     YES 

       double negatives      YES 

       stressful questions   YES 

       grossly hypothetical questions       YES 

       the effect of response styles YES 

       periodicity questions YES 

Question Ordering (s5.8)

What reasons are there for the question ordering?




                                                                     16
Survey Design Checklist
Question Instructions (s5.9)

What instructions were provided for respondents/interviewers?




                                                                17
Survey Design Checklist



Section 5             Pilot Survey(s)
Description of Pilot Surveys (s6.2)

What pilot testing was performed?




If no pilot testing was done, why not?




Size of the Pilot Survey (s6.3)



                                         18
Survey Design Checklist
Lessons from the Pilot Survey (s6.2)

How adequate was the sampling frame?




What was the variability within the survey population?




What response rate was achieved?



How suitable was the survey method?




How well did the questionnaire perform?




How effective was the interviewer training?




Did the coding, data entry, editing and analysis procedures work satisfactorily?




Cost and Duration of Pilot Surveys (s6.2)

Cost:

Duration:



                                                                                   19
Survey Design Checklist


Section 6               Administration of the Survey
Survey Procedures

Self-Completion Questionnaires (s7.2)

- pre-contact procedures



- mail-out procedures




- response receipt procedures




- phone enquiry procedures




- postal reminder regime




- telephone follow-ups




- validation interviews



- non-response interviews




                                                       20
Survey Design Checklist



Personal Interviews (s7.3)

- pre-contact procedures




- call-back procedures




- maintenance of survey logs




- interviewer payment methods




- field supervisor tasks




- work distribution procedures




                                 21
Survey Design Checklist



Telephone Interviews (s7.4)

- sampling procedures




- dealing with non-response




- use of CATI systems




                              22
Survey Design Checklist




Intercept Surveys (s7.5)

- procedures for obtaining respondents




- distribution of surveys




- collection of surveys




                                         23
Survey Design Checklist




In-depth Interview Surveys (s7.6)

- pre-contact procedures



- call-back procedures




- maintenance of survey logs




- recording methods




- transcription methods




- interpretation of responses




                                    24
Survey Design Checklist



Section 7            Data Coding
Selection of Coding Method (s8.2.1)

What physical method was used for data coding?




Preparation of Code Format (s8.2.2)

What coding frame was used?

(provide full coding frame in Appendix)




What location-coding method was used?




Development of Data Entry Programs (s8.3)

What special data entry programs were developed?

(provide screen-shots of data entry screens in Appendix)




                                                           25
Survey Design Checklist
Coder and Data Entry Training (s8.2.4)

What training was provided for coders and data enterers?

(provide training manual in Appendix)




Coding Administration (s8.2.4)

How was the coding administered?




What quality control procedures were implemented?




How were changes made to coding frames?




                                                           26
Survey Design Checklist



Section 8             Data Editing
Initial Questionnaire Editing (s8.3.2)

What in-field checking was done by interviewer/supervisor?




What checking was done on receipt in survey office?




Verification of Data Entry (s8.3.1)

Was data entry verified for accuracy?        YES       NO 

Development of Editing Computer Programs (s8.3)

Were special data editing programs developed?    YES        NO 

If so, in what language were they written?



Consistency and Range Checks (s8.3.2)

What permissible range checks were applied?

(provide full list of checks in Appendix)



What logic checks were applied?

(provide full list of checks in Appendix)




                                                                    27
Survey Design Checklist
Missing Data (s9.2)

How was missing data coded?




Were estimates made of missing values?




                                         28
Survey Design Checklist



Section 9             Data Correction and Expansion
Editing Check Corrections (s8.3.2)

What procedures were used for office edits?




Secondary Data Comparisons (s9.1)

What secondary data was used for sample expansion?




What variables were used for expansion purposes?




Was expansion based on cross-tabulations or marginal totals?

       Cross-tabs                   Marginals 



What were the final expansion factors?

(provide full list of expansion factors in Appendix)



How are they to be applied when using the data?




                                                               29
Survey Design Checklist
Corrections for Internal Biases

What recognition was there of non-reported data?(s9.2)




Were non-reporting factors calculated?           YES    NO 



If so, how are they to be applied to the data?




What recognition was there of non-response?(s9.3)




Were non-response factors calculated?            YES    NO 



If so, how are they to be applied to the data?




                                                                30
Survey Design Checklist



Section 10            Data Analysis and Management
Exploratory Data Analysis (s10.1)

What EDA methods were used?




Model Building (s10.2)

Is the data to be used to build specific models?   YES    NO 

If so, what type?




Interpretation of Results (s11.1)

Are any limitations on the data clearly stated?    YES    NO 

How?




How is the sampling error expressed?




                                                                  31
Survey Design Checklist
Database Management (s8.2.3)

Is the structure of the datafiles clearly described?   YES    NO 

(provide full list of datafiles in Appendix)




Are the relationships between data files clear?        YES    NO 

(provide full set of relationships in Appendix)




Provision of Data Support Services (s11.2)

What support is available for users of the data?




Is it clear where such support can be obtained?        YES    NO 




                                                                      32
Survey Design Checklist



Section 11            Presentation of Results
Presentation of Results of Analysis (s11.1)

Are the major descriptive results presented:

- in a clear visual manner?   YES             NO 



- with accompanying written explanations? YES          NO 



- with appropriate interpretations?   YES          NO 



- and with clear statement of any qualifications?   YES       NO 




Publication of Results (s11.2)

Are the results of the survey or the survey methodology written up in concise form,
and available in the general literature?

       YES           NO 

If so, where?




                                                                               33
Survey Design Checklist


Section 12            Tidying-Up
Storage and Archival of Data (s11.3)

Where is the data stored?



Who is the contact person?

Name:

Position:

Organisation:

Address:



Are telephone, fax and e-mail numbers provided?
Telephone:
Fax:
e-mail:


Is this documentation stored electronically with the data?

        YES          NO 


Has the data been lodged with any other archival service?

        YES          NO 


Have all survey staff been fully paid?              YES       NO 


Have all outstanding bills been paid?               YES       NO 


What arrangements have been made for destroying original questionnaires?




                                                                           34

								
To top