survey by wuzhengqin


									                          SURVEY RESEARCH

 In general, a survey involves the collection of information from a large group of people or a
  population, commonly via:
        Opinion Surveys
        Political Polls
        TV Viewing Polls
        However…
 …our focus is on survey research, which is conducted to advance scientific knowledge or
  develop theory.
 Survey research is frequently utilized within the Social Sciences, including disciplines like
  Psychology, Marketing, and Organizational Behaviour.

 Positivism has become a dominant institutional form in social research.
 However, during the 1970's and 1980's prominent concerns were raised about the limits of
  quantitative data and methods often associated with positivism; including survey research
         Eg: Concerns explained.
 Assumption: Objective world which science can “mirror” with privileged knowledge.
 Key Focus: Search for contextual and organizational variables which cause organizational
 Key Theories in Paradigm: Contingency theory, systems theory, population ecology,
  transaction cost economics of organizing, dustbowl empiricism, etc.
 Goal of Paradigm: Uncover truth and facts as quantitatively specified relations among
 Criteria for Assessing Research:
         Prediction=Explanation, Rigour; internal/external validity, reliability.
 Unit of Analysis: The variable.
 Research Methods and Types of Analysis:
         Experiments, questionnaires, secondary data analysis, quantitatively coded
         Quantitative: Regression, Likert scaling, structural equation modelling.
         Qualitative: Grounded theory testing.


 In survey research, the researcher selects a sample of respondents from a particular population
  and administers a standardized questionnaire.
 A survey can be administered to large or small populations.
 Surveys are an effective tool to use when a population is too large to be observed directly.
 Survey research is used to collect data for descriptive studies, explanatory and exploratory
 In survey questionnaires, individuals or a representative of a group provide data that measures
  attitudes, opinions and orientations of a population.
 Surveys are not only used as a method of social inquiry, but also in public opinion polls, by
  political parties and by telemarketers.
Major Features of the Method
 Variables are often operationalized when researchers ask people questions as a way of getting
  data for analysis and interpretation (the questions are either asked by an interviewer, or written
  down and given to respondents for completion).
 2 Forms of surveys:
      Questionnaire: document containing questions and others types of items designed to
         solicit information appropriate for analysis.
      In-depth Interviewing: ask people questions in order to gather data.
 Survey research is especially appropriate for making descriptive studies of large populations.
 Data collected may be used for explanatory purposes.

The Beauty of Questionnaires
 It provides a method of collecting Data by either asking people questions or asking them to
  agree or disagree with statements representing different points of view.
 Used primarily in survey research, but also in experiments, field research and other modes of
 Questions can be open ended (respondents supply their own answer-similar to a short answer
  question on an exam), answers can be in the form of writing on paper or verbally reporting
  answers to an interviewer.
         Example: what is your opinion on abortion?
 It can also be closed ended (select from a list of answers provided) This type of
  questionnaire is more common than open-ended ones because they provide greater uniformity
  of responses, and are more easily processed.
         Example: From a range of 1-5, how would you rate this hotel? With 1 being poor,
           and 5 as excellent.

A Closer Look at Closed Ended Questions
 Two structural requirements for closed ended questions:
      1. Response categories provided should be exhaustive (should include all possible
         responses that might be expected-even if answers maybe obscure to you and me).
         Usually researchers try to ensure this by adding a category such as “other” and may
         follow “please specify.”
      2. The answer category must be mutually exclusive: respondent should not be compelled
         to select more than one to ensure this, we must carefully consider each combination
         of categories, and could the respondent reasonably choose more than one answer?
 If our survey is constructed carefully, we usually do not need to add an instruction to tell
  respondents to select the “best answer.”

Things to Note for when Asking Questions
 Make Items Clear: Questionnaire items should be precise so that the respondent knows
  exactly what the researcher is asking.
 Avoid Double-Barrelled Questions: Researchers asking respondents for a single answer to
  question that actually has multiple parts.
 Respondents must be competent to answer: When asking respondents to provide
  information, must keep in mind whether they can do so reliably.
 Respondents must be willing to answer: Often, we would like to learn things from people
  that they are unwilling to share with us.
 Questions should be relevant: Questionnaire should be relevant to most respondents.
 Shorts Items are the best: Respondents are often unwilling to study an item in order to
  understand it, respondent should be able to read something quickly, understand the main point,
  and select an answer without difficulty, answers should not be misinterpreted.
 Avoid Negative Items/Negative words: Appearance of a negation in a questionnaire items
  paves the way for easy misinterpretation.
 Avoid Biased Items and Terms: Meaning of someone’s response to a question depends in
  large part on its wording, some questions seem to encourage particular response more than do
  other questions.

Three Main Methods for Administering Survey Questionnaires
1. Self administered questionnaires: respondents are asked to complete the questionnaire
   themselves (example: Mail survey), administer questionnaire to a group of respondents
   gathered at the same place at the same time.
2. Surveys administered by interviewers in face-to-face encounters.
3. Surveys conducted by the telephone.

Mail Distribution and Return
 Research worker can either hand deliver questionnaires, requesting that respondents mail the
  completed questionnaires to the research office or after questionnaires are mailed, researcher
  visits homes to pick them up and check for completeness.
 Completion rate seems to be higher when research worker delivers the questionnaire, picks it
  up or both.
 Basic method of collecting data through mail is to send questionnaire, with a letter of
  explanation and a self addressed, stamped envelope for returning (for example, like the
  surveys we receive in our mailboxes, many often have the “postage paid” indicated in the
  corner of the envelope).
 The Main reason for not retuning questionnaires is that it’s TOO MUCH TROUBLE!

Acceptable Response Rates
 Response rates: the number of people participating in a survey divided by the number selected
  in the sample (percentage form).
 Inferential statistics used with survey analysis assumes that all members of the initial sample
  complete and return their questionnaires.
 Because that is very rare to happen, response bias is a concernresearchers often hopes for
  the possibility that the respondents look essentially like a random sample of the initial sample,
  and a somewhat smaller random sample of the total population.
 Overall response rate is one guide to the representative-ness of the sample respondentshigh
  response rates means less chance of significant response bias, with low response rates,
  correspondents are likely to differ from the respondents in ways other than their willingness to
  participate in the survey.
 Rough guides (with no statistical basis) to what is a good response rate: 50% is adequate for
  analysis and reporting, 60% is good and 70% is very good.

Interview Surveys
 Interviewers ask questions orally and record respondent’s answers.
 Usually done in a face-to-face encounter, but telephone interviewing follow similar guidelines.
 Most interviews require more than one interviewer, but small scale interviews can be done by
   one person.
 Researchers must assume that a questionnaire item will mean the same thing to every
   respondent and every given response must mean the same when given by different
 The interview’s presence should not affect a respondent's perception of a question or the
  answer given.
 To save time and money, a given interviewer is typically assigned to complete all the
  interviews in a particular geographical area.

General Guidelines for Survey Interviewing
 Appearance and Demeanour: Interviewers should dress in a fashion similar to that of the
  people they’ll be interviewing (cleanliness and neatness in modest apparel). Dress and
  grooming are typically regarded as signs of a person’s attitudes and orientations. Interviewers
  should be pleasant, must communicate interest in getting to know the respondent without
  appearing to spy (never too casual or clingy). Interview will be more successful if the
  interviewer can become the kind of person the respondent is comfortable with, respondents
  deserve the most enjoyable experience the researcher can provide.
 Familiarity with Questionnaire: Must be able to read questionnaire items to respondents
  without error and stumbling over words, lines must be read as though they are part of a natural
 Interviewer must be familiar with the specifications prepared in conjunction with the
 Some questions will not fit a given respondent's situation, interviewer must determine how the
  question should be interpreted in that situation.
 It would be better for the interviewer to leave a question unanswered than to spend a period of
  time searching though the specifications for clarifications or trying to interpret the relevant
 Following Question Wording Exactly
 Recording Responses Exactly
 Probing for Responses: Sometimes respondents will give an inappropriate/incomplete
  answer, request for an elaboration can be useful (Probe). Probes must be completely neutral,
  must not affect the nature of subsequent response.

Advantages of Doing an Interview Survey
 They typically attain high response rates (a properly designed and executed interview survey
  ought to achieve a completion rate of at least 80-85%).
 Presence of and interviewer decreases the number of “don’t knows:” and “no answers”, since
  clarification can easily be done on the spot.
 Interviewers can clarify matters if respondent clearly misunderstands the intent of the
 Can observe respondents as well as ask questions.

Telephone Surveys: Positive Factors
 Saves both money and time.
 May dress anyway you please without affecting the answers respondents give.
 Respondents may be more honest in giving socially disapproved answers (no eye to eye
 Interviewers can communicate a lot about themselves over the phone, even though they can’t
   be seen.
 Allow greater control over data collection if several interviewers are engaged in the project.

Telephone Surveys: Negative Factors
 Some phone numbers are unlisted (but this has been erased through a technique called random
   digit dialling).
 Bogus surveys-ones that are actually sales campaigns disguised as research.
 The ease in which people can hang up.
 Answering machines (using machines to screen calls)however research has showed that this
  had not yet had a significant effect on the ability of telephone researchers to contact
  prospective respondents.

Comparing the Three Different Methods

            Self Administered         Interview Surveys          Telephone Surveys
           Cheaper and quicker     Produce fewer incomplete      Cheaper and can be
                                        questionnaires          mounted and excited
         Costs no more to conduct     More effective for       Safer when interviewing
          a national survey than a   complicated surveys           high crime areas
           local one of the same
                sample size
            Require a small staff  Conduct survey based on Impact of interviewers on
                                    sample of addresses or  responses is somewhat
                                   phone numbers rather on lessened when they can’t
                                           names                    be seen

Computer Assisted Telephone Interviewing (CATI)
 Increasingly used by academic, government and commercial survey researchers.
 Central computer is programmed to select a telephone number at random and dials it.
 Interviewer introduces study and asks the question displayed on the screen and then types that
  answer into the computer terminal (depending on the question-open/close ended).
 Computer automatically prepares the data for analysis, or researcher can begin analysing data
  before the interview is complete (gaining and advanced view of how the analysis will turn

New Technologies and Survey Research
 CAPI (Computer assisted personal interviewing): face to face interviews rather than over
  the phone.
 CASI (Computer assisted self interviewing): respondent reads questions on the computer
  screen and enters his/her own answers.
 CSAQ (Computerized self administered questionnaire): respondent receives the
  questionnaire on a floppy disk, bulletin board, or other means and completes the questions,
  software then accepts the answers. Respondent then returns the data file.
 TDE (Touchtone data entry): initiates process by calling a number at the research
  organization, prompts a series of computerized questions, respondent answers by pressing the
  keys on the telephone pad.
 VR (Voice recognition): same as TDE, but the system accepts spoken responses.
 Internet and world wide web.

Dos and Don’ts for Conducting Online Surveys
 DO:
 Consistent wording between the invitation and the survey.
 Use plain simple language.
 Offer to share selected results from others who also have completed the survey.
 Plan the time of day and day of week to mail (when will respondents most likely be reading
  mail at home?).
 Be aware of technical limitations (Will respondents have programs needed to access).
 Test incentives, rewards and prize drawings to determine the optimal response.
 Limit studies to <15 minutes.
 DON’T:
 Use terms such as unique ID number in the invitation, then ask respondents to type
  “password” when they get to the survey.
 Force the respondent to scroll down the screen for the URL for the study location.

Secondary Analysis
 Survey research involves 3 steps: questionnaire construction, sample selection, and data
   collection (interviewing/self administered questionnaires).
 This is a form of research in which the data is collected and processed by one researcher are
   reanalyzed- often for a different purpose, by another.
 Example: General Social Survey (GSS).
 Advantage: cheaper and faster than doing original surveys, benefit from the work of top flight
   professionals, enhance possibility of meta-analysis (researcher brings together a body of past
   research on a topic).
 Disadvantage: question of validity (have no assurance that data collected will be appropriate
   for you research interests).


Internal Validity
 To be internally valid, the conclusions of the research must be supported by the data.
 Internal validity is judged according to the accuracy with which a description of particular
   events represents the data.
 The essence of internal validity for survey research is complete confidence that your
   conclusions come from the data.

External Validity
 External validity or generalizability for survey research is captured by the question, “How can
  one determine the extent to which the findings of a particular inquiry have applicability in
  other contexts or with other subjects?”
 Essentially, the extent to which the effect can be generalized to populations, settings,
  treatment variables, and measurement variables.
 This can also refer to “fit” or the degree to which the audience or reader of the report is able to
  transfer the research findings to contexts outside of the study situation to other settings.
 The researcher must supply a substantial amount of clear and detailed information or thick
  description about the issue/phenomenon studied and the setting in which that
  issue/phenomenon was found.
 The degree of transferability is a direct function of the similarity or “fittingness” between the
  two contexts.
        Eg: Alberta bureaucrat scenario.

 Reliability is generally concerned with replication: an account is considered to be reliable if
   the data are reproducible.
 If the analytic strategy were repeated by the same or different investigator, then the results
   should be the same.
Strategies for Ensuring Rigour
1. Verification Strategies
       The most important way to ensure that research is rigourous is to focus on verification
          during the study.
       Verification is the process of checking, confirming, making sure, being certain.
       Doing this as the study is conducted is key as the researcher then can identify and
          correct threats to reliability and validity as they surface.
       Strategies include:
             Investigator Responsiveness:
             Methodological Coherence:
             Sampling:
             Data Analysis:
             Thinking Theoretically:

2. Other Strategies
    There are other strategies that can be used during the research to contribute to rigour,
             Prolonged Engagement:
             Participant Checks:
             Journal Writing:
             Peer Review:
             Audit Trail:


Strengths for Survey Research
 Describes the characteristics of a large population.
 Allows a large sample of respondents.
 Surveys are flexible, allows flexibility in your analyses.
 Surveys have strength in measurement.
 Helps determine unemployment rates, and voting intensions.
 Helps examine official documents such as marriage, birth, or death records.
 Therefore, surveys are strong on reliability.

Weaknesses for Survey Research
 Lacks the context of social life.
 Appear superficial on coverage of complex topics.
 Subject to artificiality.
 Surveys can also be inflexible because the initial study design must remain the same.
 Therefore, surveys are weak on validity.


1. Determine the purpose of the survey
        Why is the research being conducted?
2. Determine respondent group
        Select respondents to whom the survey will be relevant.
        Select respondents who are willing to share information.
        Select a reliable sample of respondents.
3. Determine the questionnaire construction
         Questionnaires are used to ask people question as a way of getting data for analysis
          and interpretation.

Design Guidelines:
       a) Determine the type of questions to be asked to achieve the best results in achieving the
               i. Open-ended questions: The respondent provides an answer to the question.
                   Responses must then be coded by the researcher for analysis
              ii. Closed-ended questions: A list of answers is provided from which the
                   respondent selects a response. The list of answers must include all possible
                   responses and be carefully constructed to provide one best answer. Closed
                   ended questions are preferable because they provide a greater uniformity of
                   answers and the answers are more easily processed.
             iii. Combination of open and closed-ended questions:
       b) Include statements in the design.
                 - Statements add interest to the questionnaire and allow for greater flexibility in
                   the design.
                 - Statements can be used to summarize attitudes to which the respondents can
                   agree or not agree.
                 - The Likert Scale is often used in questionnaires where there are statements.
                         Eg. SA, A, D, SD, U
       c) Make sure questions and/or statements are short, clear and precise so there will be no
          misunderstanding in meaning.
       d) Avoid the use of “and” in questions and statements so they will not be “double-
          barreled” and lead to confusion.
       e) Avoid the use of negative words such as “no” and “not.” Items should be phrased in a
          positive manner.
       f) Items in a questionnaire should not reveal any biases. Avoid socially desirable
          questions and answers.

Format Guidelines:
      a) Researchers should provide a short introductory comment to help the respondent make
         sense of the questionnaire.
      b) The introduction should make comments about content and purpose of the
      c) Questionnaires should have a professional and uncluttered appearance. By spreading
         out questions, respondents will avoid errors, missed questions, misinterpretation and
         have adequate space to answer questions completely.
      d) Researchers need to provide adequate explanations and guidelines to provide adequate
         explanations and guidelines on what respondents are expected to do.
             i. Contingency questions: A survey question intended for only some
                 respondents, determined by their response to a previous question. Instructions
                 can be isolated and highlighted in some manner or provided in a flow chart.
            ii. Matrix questions: Several questions with the same set of answer categories.
                        Eg: Likert responses.
      e) Careful consideration must be given to the order of items in the questionnaire.
               - The order needs to be logical and non-threatening. An effective order of
                 questions will motivate respondents to complete the questionnaire without
                 getting frustrated.
               - Generally for self-administered questionnaires it is best to begin with the most
                 interesting set of items to get the respondents’ attention. Duller demographic
                   data can be obtained at the end. For interview surveys, it is best to establish a
                   rapport with the respondents by obtaining demographic dada first and the
                   proceeding to other questions.
               - Pre-testing, by having a small sample population complete the questionnaire
                   provides a good means by which to eliminate errors and determine an
                   effective order to the items in the questionnaire.
4. Administration of the questionnaire
       a) Decide on the most effective way to administer the questionnaire, keeping in mind the
          advantages and disadvantages of each method.
               - self-administered
               - face-to-face interviews
               - telephone surveys
               - other
5. Analysis and interpretation of data


Need More Information?
 Group Presentation Website:
 Group Presentation E-mail:

To top