Questionnaires and Surveys A quick and dirty overview

Document Sample
Questionnaires and Surveys A quick and dirty overview Powered By Docstoc
					Questionnaires and Surveys: A quick and dirty overview

 Purposes: To rapidly determine how a relatively large group of people respond to
predetermined issues and focused questions. Unlike interviews which allow the
researcher to probe a relatively small number of respondents in depth, surveys are less
flexible and lack the ability to ask the follow up question. Surveys are often a second
step after pilot interviews or open-ended surveys have been completed.

Content: may be on issues of fact or as a way to assess attitudes or opinions.

         Surveys about factual issues are easier to construct.

         Opinions are the expressions of deeper, guiding attitudes.

         Attitudes are convictions, fears, inclinations.

        Attitudes are expressed when “triggered” by some object or event. (e.g., response
to the Katrina fiasco).

         It’s relatively easy to assess opinions by using a simple questions and a rating

       Attitudes require sets of 5-20 related questions with which respondent has to
agree or disagree.

Survey Formats and getting people to respond: People need to see it as worthwhile –
altruistic or serving some need of their own.

                Surveys may be

                1.   mailed,

                2.   distributed electronically via email or forum,

                3.   given out in classes, or

                4.   conducted verbally (an interview) either in person or by telephone..

 Writing clear, concise questions that relate to the issues of interest is the most difficult
part of using survey methods.

Question formats:
                    open ended. No fixed answer.

Open ended questions are very difficult to evaluate and should be used sparingly once the
pilot stage has passed and categories of answers have been identified.

                     Close-ended: Have a fixed set of answers to respond to. May be
                     factual or attitudinal.

These are quick and easy to evaluate. However, providing answer categories can
introduce bias by giving people responses they may not have thought of, or, by not
including a person’s choice the person can become negatively influenced toward the

Close-Ended Questions:       Response Formats

 1. Fixed choice

e.g., three ways to structure responses to closed ended questions. Which do you prefer?

 How many time per week             Circle the best      Check the best           Circle the
 do you                                answer               answer              number of the
                                                                                 best answer
 meet each science class on

                                 2 or 3                ___    2 or 3       1.      2 or 3
                                 4                     ___    4            2.      4
                                 5                     ___    5            3.      5
                                 other                 ___    other        4.      other

2. Rating scales

          These are used to measure intensity of response on some set of ordered

                       Strongly disagree              disagree            agree
strongly agree         no opinion

I like ice cream               __                        __               __
__          __

Joe is doing a good job     1                          2                     3
4            0

                                   Too little                           About right
                     Too much

        The amount of reading is      I--------------------------------------I--------------------

3. Ranking schemes

        Respondent must prioritize a set of options. Important that directions be
clear to eliminate multiple scores of any number.

              e.g. Please tell what is most important to you in a teacher. Rank the most
              important item “1,” the next most important item “2,” and so on. Do not
              use a ranking more than once. Use only whole numbers.

              _____ Being prepared

              _____ Being fair

              _____ Knowing how to motivate students
                _____ Knowing subject matter

                _____ Good classroom management

Criteria for writing close-ended questions.

        Response categories must be:

       exhaustive,
        mutually exclusive,
       clearly defined in terms the respondent is likely to understand


           1. Not exhaustive

                      Sciences courses taken                  Chemistry _____

                                                             Physics      _____

                                                             Biology      _____

   2.   Not mutually exclusive

                Number of different preps per day              _____ 1-3

                                                              _____ 3-5

                                                              _____ 5-7

   3.   Not clearly understood

                How many science classrooms are in your school?                   ___ none
                                                                                ___ 1-3

                                                                                ___ 4-6

                                                                                ___ more
than 6

   4.    Violates all three:

                         What kind of science furnishings are in science classrooms at your

                         _____ tables

                         _____ desks

                         _____ lab benches

                         _____ stools

                         _____ chairs

                         _____ storage cabinets

                        _____ lockers

Sequencing the questions:

Use the inverted funnel model.

 Start with narrow questions, followed by broader ones. Go from facts to generalizations,
from specifics to abstract ideas.

The first questions:

         1 Are the ones most often responded to favorably. They also often shape the
         respondents attitude to the survey and subsequent responses.
       2. Should put the responder at ease because they are easy to answer and don’t
       touch on sensitive issues.

       3. Are usually closed ended

For example.........

       1.     What grade were you in when you first encountered science in school?

       2.     What were the science lessons like?

                a.     Did you use a text?

                b.     Did you have a lab?

                c.     Did you do investigations?

                d.     Did you write papers?

                e.     other?

       3.     In general, how effective do you think your early science experiences were?

       4.     In your opinion, what would a very good first science experience be like?

Writing Instructions

For every question or group of questions with similar response types, you need to write a
clear set of instructions.

            They can be simple. “Circle the one choice that best reflects your opinion”
            Or, they can be complex. See the rank ordering example above.

If your questionnaire is long, you may need to introduce different sections of the
questionnaire with separate instructions.

Telling the Purpose

If you are going to need parental permission or are going to mail the survey, you need a
cover letter telling the purpose of the survey. Otherwise, the survey needs a little
introduction at the beginning that helps to make the likelihood of people responding
higher. These guidelines are similar to information to be given before conducting

       1.     Identify who is doing the survey

       2.     Tell why the survey is being done. There may be a conflict between research
             needs and ethics here.

       3.     Tell why it is important that they respond to the survey (how they benefit)

       4.     Confidentiality and how it will be maintained (if appropriate).

  1.   Question placement on the list may bias the responses.

  2.   The wording may bias a person’s answers.

           a.   If they don’t understand the vocabulary
           b.   If words open to too much interpretation are used, like “liberal” or “republican”

  3.   Response set bias. Having all of a set of opinion questions be rated in the same direction.

       This is especially a problem in matrix formatted questions.

  4.   Leading questions.
          a. The responder can tell the way they are supposed to answer/

                E.g., Don’t you think the best teachers ……?

                      e.g., You wouldn’t say this text was too bad, would you?

                      e.g., Most people say that…., what do you think?

           b.   Questions that use emotionally loaded words can bias the way people respond.
                e.g., issues like starvation,

                 Food should be sent to starving people in Bosnia     SA         A
D          SD       N

    5.    Threatening questions: These touch on anxiety arousing topics like sex, drugs,
         religion, gambling. People tend to underreport their activities. Can increase by:
         Using a long explanatory introduction.

                                                                     Avoiding short

                                                                       Using open-ended
instead of closed questions

    6.   Double-barreled questions: These are two in one.

                     e.g. Would you say most science is inquiry based and fun?

If both elements are important, then sort out the questions and ask them separately.

Or, you could have people rate them.

                     Which of the two elements of a science program is most important?

                            1.   that it be inquiry based

                            2.   That it be fun

David Hopkins, A Teacher’s Guide to Classroom Research, Philadelphia: Open
University Press, 1985

Shared By: