I. THEORETICAL AND EPISTEMOLOGICAL BACKGROUND
In general, a survey involves the collection of information from a large group of people or a
population, commonly via:
TV Viewing Polls
…our focus is on survey research, which is conducted to advance scientific knowledge or
Survey research is frequently utilized within the Social Sciences, including disciplines like
Psychology, Marketing, and Organizational Behaviour.
Positivism has become a dominant institutional form in social research.
However, during the 1970's and 1980's prominent concerns were raised about the limits of
quantitative data and methods often associated with positivism; including survey research
Eg: Concerns explained.
Assumption: Objective world which science can “mirror” with privileged knowledge.
Key Focus: Search for contextual and organizational variables which cause organizational
Key Theories in Paradigm: Contingency theory, systems theory, population ecology,
transaction cost economics of organizing, dustbowl empiricism, etc.
Goal of Paradigm: Uncover truth and facts as quantitatively specified relations among
Criteria for Assessing Research:
Prediction=Explanation, Rigour; internal/external validity, reliability.
Unit of Analysis: The variable.
Research Methods and Types of Analysis:
Experiments, questionnaires, secondary data analysis, quantitatively coded
Quantitative: Regression, Likert scaling, structural equation modelling.
Qualitative: Grounded theory testing.
II. SURVEY RESEARCH
In survey research, the researcher selects a sample of respondents from a particular population
and administers a standardized questionnaire.
A survey can be administered to large or small populations.
Surveys are an effective tool to use when a population is too large to be observed directly.
Survey research is used to collect data for descriptive studies, explanatory and exploratory
In survey questionnaires, individuals or a representative of a group provide data that measures
attitudes, opinions and orientations of a population.
Surveys are not only used as a method of social inquiry, but also in public opinion polls, by
political parties and by telemarketers.
Major Features of the Method
Variables are often operationalized when researchers ask people questions as a way of getting
data for analysis and interpretation (the questions are either asked by an interviewer, or written
down and given to respondents for completion).
2 Forms of surveys:
Questionnaire: document containing questions and others types of items designed to
solicit information appropriate for analysis.
In-depth Interviewing: ask people questions in order to gather data.
Survey research is especially appropriate for making descriptive studies of large populations.
Data collected may be used for explanatory purposes.
The Beauty of Questionnaires
It provides a method of collecting Data by either asking people questions or asking them to
agree or disagree with statements representing different points of view.
Used primarily in survey research, but also in experiments, field research and other modes of
Questions can be open ended (respondents supply their own answer-similar to a short answer
question on an exam), answers can be in the form of writing on paper or verbally reporting
answers to an interviewer.
Example: what is your opinion on abortion?
It can also be closed ended (select from a list of answers provided) This type of
questionnaire is more common than open-ended ones because they provide greater uniformity
of responses, and are more easily processed.
Example: From a range of 1-5, how would you rate this hotel? With 1 being poor,
and 5 as excellent.
A Closer Look at Closed Ended Questions
Two structural requirements for closed ended questions:
1. Response categories provided should be exhaustive (should include all possible
responses that might be expected-even if answers maybe obscure to you and me).
Usually researchers try to ensure this by adding a category such as “other” and may
follow “please specify.”
2. The answer category must be mutually exclusive: respondent should not be compelled
to select more than one to ensure this, we must carefully consider each combination
of categories, and could the respondent reasonably choose more than one answer?
If our survey is constructed carefully, we usually do not need to add an instruction to tell
respondents to select the “best answer.”
Things to Note for when Asking Questions
Make Items Clear: Questionnaire items should be precise so that the respondent knows
exactly what the researcher is asking.
Avoid Double-Barrelled Questions: Researchers asking respondents for a single answer to
question that actually has multiple parts.
Respondents must be competent to answer: When asking respondents to provide
information, must keep in mind whether they can do so reliably.
Respondents must be willing to answer: Often, we would like to learn things from people
that they are unwilling to share with us.
Questions should be relevant: Questionnaire should be relevant to most respondents.
Shorts Items are the best: Respondents are often unwilling to study an item in order to
understand it, respondent should be able to read something quickly, understand the main point,
and select an answer without difficulty, answers should not be misinterpreted.
Avoid Negative Items/Negative words: Appearance of a negation in a questionnaire items
paves the way for easy misinterpretation.
Avoid Biased Items and Terms: Meaning of someone’s response to a question depends in
large part on its wording, some questions seem to encourage particular response more than do
Three Main Methods for Administering Survey Questionnaires
1. Self administered questionnaires: respondents are asked to complete the questionnaire
themselves (example: Mail survey), administer questionnaire to a group of respondents
gathered at the same place at the same time.
2. Surveys administered by interviewers in face-to-face encounters.
3. Surveys conducted by the telephone.
Mail Distribution and Return
Research worker can either hand deliver questionnaires, requesting that respondents mail the
completed questionnaires to the research office or after questionnaires are mailed, researcher
visits homes to pick them up and check for completeness.
Completion rate seems to be higher when research worker delivers the questionnaire, picks it
up or both.
Basic method of collecting data through mail is to send questionnaire, with a letter of
explanation and a self addressed, stamped envelope for returning (for example, like the
surveys we receive in our mailboxes, many often have the “postage paid” indicated in the
corner of the envelope).
The Main reason for not retuning questionnaires is that it’s TOO MUCH TROUBLE!
Acceptable Response Rates
Response rates: the number of people participating in a survey divided by the number selected
in the sample (percentage form).
Inferential statistics used with survey analysis assumes that all members of the initial sample
complete and return their questionnaires.
Because that is very rare to happen, response bias is a concernresearchers often hopes for
the possibility that the respondents look essentially like a random sample of the initial sample,
and a somewhat smaller random sample of the total population.
Overall response rate is one guide to the representative-ness of the sample respondentshigh
response rates means less chance of significant response bias, with low response rates,
correspondents are likely to differ from the respondents in ways other than their willingness to
participate in the survey.
Rough guides (with no statistical basis) to what is a good response rate: 50% is adequate for
analysis and reporting, 60% is good and 70% is very good.
Interviewers ask questions orally and record respondent’s answers.
Usually done in a face-to-face encounter, but telephone interviewing follow similar guidelines.
Most interviews require more than one interviewer, but small scale interviews can be done by
Researchers must assume that a questionnaire item will mean the same thing to every
respondent and every given response must mean the same when given by different
The interview’s presence should not affect a respondent's perception of a question or the
To save time and money, a given interviewer is typically assigned to complete all the
interviews in a particular geographical area.
General Guidelines for Survey Interviewing
Appearance and Demeanour: Interviewers should dress in a fashion similar to that of the
people they’ll be interviewing (cleanliness and neatness in modest apparel). Dress and
grooming are typically regarded as signs of a person’s attitudes and orientations. Interviewers
should be pleasant, must communicate interest in getting to know the respondent without
appearing to spy (never too casual or clingy). Interview will be more successful if the
interviewer can become the kind of person the respondent is comfortable with, respondents
deserve the most enjoyable experience the researcher can provide.
Familiarity with Questionnaire: Must be able to read questionnaire items to respondents
without error and stumbling over words, lines must be read as though they are part of a natural
Interviewer must be familiar with the specifications prepared in conjunction with the
Some questions will not fit a given respondent's situation, interviewer must determine how the
question should be interpreted in that situation.
It would be better for the interviewer to leave a question unanswered than to spend a period of
time searching though the specifications for clarifications or trying to interpret the relevant
Following Question Wording Exactly
Recording Responses Exactly
Probing for Responses: Sometimes respondents will give an inappropriate/incomplete
answer, request for an elaboration can be useful (Probe). Probes must be completely neutral,
must not affect the nature of subsequent response.
Advantages of Doing an Interview Survey
They typically attain high response rates (a properly designed and executed interview survey
ought to achieve a completion rate of at least 80-85%).
Presence of and interviewer decreases the number of “don’t knows:” and “no answers”, since
clarification can easily be done on the spot.
Interviewers can clarify matters if respondent clearly misunderstands the intent of the
Can observe respondents as well as ask questions.
Telephone Surveys: Positive Factors
Saves both money and time.
May dress anyway you please without affecting the answers respondents give.
Respondents may be more honest in giving socially disapproved answers (no eye to eye
Interviewers can communicate a lot about themselves over the phone, even though they can’t
Allow greater control over data collection if several interviewers are engaged in the project.
Telephone Surveys: Negative Factors
Some phone numbers are unlisted (but this has been erased through a technique called random
Bogus surveys-ones that are actually sales campaigns disguised as research.
The ease in which people can hang up.
Answering machines (using machines to screen calls)however research has showed that this
had not yet had a significant effect on the ability of telephone researchers to contact
Comparing the Three Different Methods
Self Administered Interview Surveys Telephone Surveys
Cheaper and quicker Produce fewer incomplete Cheaper and can be
questionnaires mounted and excited
Costs no more to conduct More effective for Safer when interviewing
a national survey than a complicated surveys high crime areas
local one of the same
Require a small staff Conduct survey based on Impact of interviewers on
sample of addresses or responses is somewhat
phone numbers rather on lessened when they can’t
names be seen
Computer Assisted Telephone Interviewing (CATI)
Increasingly used by academic, government and commercial survey researchers.
Central computer is programmed to select a telephone number at random and dials it.
Interviewer introduces study and asks the question displayed on the screen and then types that
answer into the computer terminal (depending on the question-open/close ended).
Computer automatically prepares the data for analysis, or researcher can begin analysing data
before the interview is complete (gaining and advanced view of how the analysis will turn
New Technologies and Survey Research
CAPI (Computer assisted personal interviewing): face to face interviews rather than over
CASI (Computer assisted self interviewing): respondent reads questions on the computer
screen and enters his/her own answers.
CSAQ (Computerized self administered questionnaire): respondent receives the
questionnaire on a floppy disk, bulletin board, or other means and completes the questions,
software then accepts the answers. Respondent then returns the data file.
TDE (Touchtone data entry): initiates process by calling a number at the research
organization, prompts a series of computerized questions, respondent answers by pressing the
keys on the telephone pad.
VR (Voice recognition): same as TDE, but the system accepts spoken responses.
Internet and world wide web.
Dos and Don’ts for Conducting Online Surveys
Consistent wording between the invitation and the survey.
Use plain simple language.
Offer to share selected results from others who also have completed the survey.
Plan the time of day and day of week to mail (when will respondents most likely be reading
mail at home?).
Be aware of technical limitations (Will respondents have programs needed to access).
Test incentives, rewards and prize drawings to determine the optimal response.
Limit studies to <15 minutes.
Use terms such as unique ID number in the invitation, then ask respondents to type
“password” when they get to the survey.
Force the respondent to scroll down the screen for the URL for the study location.
Survey research involves 3 steps: questionnaire construction, sample selection, and data
collection (interviewing/self administered questionnaires).
This is a form of research in which the data is collected and processed by one researcher are
reanalyzed- often for a different purpose, by another.
Example: General Social Survey (GSS).
Advantage: cheaper and faster than doing original surveys, benefit from the work of top flight
professionals, enhance possibility of meta-analysis (researcher brings together a body of past
research on a topic).
Disadvantage: question of validity (have no assurance that data collected will be appropriate
for you research interests).
III. MAJOR CRITERIA FOR RIGOUR
To be internally valid, the conclusions of the research must be supported by the data.
Internal validity is judged according to the accuracy with which a description of particular
events represents the data.
The essence of internal validity for survey research is complete confidence that your
conclusions come from the data.
External validity or generalizability for survey research is captured by the question, “How can
one determine the extent to which the findings of a particular inquiry have applicability in
other contexts or with other subjects?”
Essentially, the extent to which the effect can be generalized to populations, settings,
treatment variables, and measurement variables.
This can also refer to “fit” or the degree to which the audience or reader of the report is able to
transfer the research findings to contexts outside of the study situation to other settings.
The researcher must supply a substantial amount of clear and detailed information or thick
description about the issue/phenomenon studied and the setting in which that
issue/phenomenon was found.
The degree of transferability is a direct function of the similarity or “fittingness” between the
Eg: Alberta bureaucrat scenario.
Reliability is generally concerned with replication: an account is considered to be reliable if
the data are reproducible.
If the analytic strategy were repeated by the same or different investigator, then the results
should be the same.
Strategies for Ensuring Rigour
1. Verification Strategies
The most important way to ensure that research is rigourous is to focus on verification
during the study.
Verification is the process of checking, confirming, making sure, being certain.
Doing this as the study is conducted is key as the researcher then can identify and
correct threats to reliability and validity as they surface.
2. Other Strategies
There are other strategies that can be used during the research to contribute to rigour,
IV. STRENGTHS AND WEAKNESSES OF SURVEY RESEARCH
Strengths for Survey Research
Describes the characteristics of a large population.
Allows a large sample of respondents.
Surveys are flexible, allows flexibility in your analyses.
Surveys have strength in measurement.
Helps determine unemployment rates, and voting intensions.
Helps examine official documents such as marriage, birth, or death records.
Therefore, surveys are strong on reliability.
Weaknesses for Survey Research
Lacks the context of social life.
Appear superficial on coverage of complex topics.
Subject to artificiality.
Surveys can also be inflexible because the initial study design must remain the same.
Therefore, surveys are weak on validity.
V. SURVEY RESEARCH DESIGN
1. Determine the purpose of the survey
Why is the research being conducted?
2. Determine respondent group
Select respondents to whom the survey will be relevant.
Select respondents who are willing to share information.
Select a reliable sample of respondents.
3. Determine the questionnaire construction
Questionnaires are used to ask people question as a way of getting data for analysis
a) Determine the type of questions to be asked to achieve the best results in achieving the
i. Open-ended questions: The respondent provides an answer to the question.
Responses must then be coded by the researcher for analysis
ii. Closed-ended questions: A list of answers is provided from which the
respondent selects a response. The list of answers must include all possible
responses and be carefully constructed to provide one best answer. Closed
ended questions are preferable because they provide a greater uniformity of
answers and the answers are more easily processed.
iii. Combination of open and closed-ended questions:
b) Include statements in the design.
- Statements add interest to the questionnaire and allow for greater flexibility in
- Statements can be used to summarize attitudes to which the respondents can
agree or not agree.
- The Likert Scale is often used in questionnaires where there are statements.
Eg. SA, A, D, SD, U
c) Make sure questions and/or statements are short, clear and precise so there will be no
misunderstanding in meaning.
d) Avoid the use of “and” in questions and statements so they will not be “double-
barreled” and lead to confusion.
e) Avoid the use of negative words such as “no” and “not.” Items should be phrased in a
f) Items in a questionnaire should not reveal any biases. Avoid socially desirable
questions and answers.
a) Researchers should provide a short introductory comment to help the respondent make
sense of the questionnaire.
b) The introduction should make comments about content and purpose of the
c) Questionnaires should have a professional and uncluttered appearance. By spreading
out questions, respondents will avoid errors, missed questions, misinterpretation and
have adequate space to answer questions completely.
d) Researchers need to provide adequate explanations and guidelines to provide adequate
explanations and guidelines on what respondents are expected to do.
i. Contingency questions: A survey question intended for only some
respondents, determined by their response to a previous question. Instructions
can be isolated and highlighted in some manner or provided in a flow chart.
ii. Matrix questions: Several questions with the same set of answer categories.
Eg: Likert responses.
e) Careful consideration must be given to the order of items in the questionnaire.
- The order needs to be logical and non-threatening. An effective order of
questions will motivate respondents to complete the questionnaire without
- Generally for self-administered questionnaires it is best to begin with the most
interesting set of items to get the respondents’ attention. Duller demographic
data can be obtained at the end. For interview surveys, it is best to establish a
rapport with the respondents by obtaining demographic dada first and the
proceeding to other questions.
- Pre-testing, by having a small sample population complete the questionnaire
provides a good means by which to eliminate errors and determine an
effective order to the items in the questionnaire.
4. Administration of the questionnaire
a) Decide on the most effective way to administer the questionnaire, keeping in mind the
advantages and disadvantages of each method.
- face-to-face interviews
- telephone surveys
5. Analysis and interpretation of data
VI. ADDITIONAL RESOURCES
Need More Information?
Group Presentation Website: http://members.shaw.ca/kristopher.skinner/soc315
Group Presentation E-mail: firstname.lastname@example.org