questionnaire by 196T3f1

VIEWS: 694 PAGES: 31

									  Collecting primary
data: questionnaires
         Week 10 lecture 1




                             1
Agenda
   Questionnaire introduction
   When to use questionnaire
   Types of questionnaire
   Identify data to be collected by questionnaire
   Designing questionnaire
       Design individual questions in questionnaire
       Opening and closing of questionnaire
       Pilot testing
   Administering the questionnaire
                                                       2
Questionnaire
   Used extensively in survey research
   Can be used in experiment and case study
    research strategies
   Strict definition
       Surveys where the person answering the question
        actually records their own answers.
   Broad view
       Include all techniques of data collection in which
        each person is asked to respond to the same set
        of questions in a predetermined order.
                                                             3
Important design issues
   It is far harder to produce a good
    questionnaire than you might think
   Issues to maximize response rates, validity
    and reliability
       Careful design of individual questions (item)
       Clear layout of the questionnaire form
       Lucid explanation of the purpose of the
        questionnaire
       Pilot testing
                                                        4
When to use questionnaires
   Not particularly good for exploratory study or other
    research that requires large number of open-ended
    question
       What would be the most suitable data collection method for
        exploratory study?
       Is questionnaire a good way to collect user requirements?
   Work best with standardized questions that you can
    be confident will be interpreted the same way by all
    respondents.
       Descriptive or explanatory research

                                                                 5
When to use questionnaire
   Descriptive research
       Opinion, attitude and experience of something
       Eg. End-user satisfaction of a new system
   Explanatory research
       Cause-and-effect relationship between variables
         Eg. Would color scheme of an e-commerce website affect
          customers’ purchasing behavior
   Multi-method
       Eg. A questionnaire to discover customers’ attitudes
        followed by in-depth interviews to explore and understand
        these attitudes.

                                                                    6
When to use questionnaire
   To collect self-reported beliefs or behaviors.
       Behavior
       Attitudes/beliefs/opinions
       Expectations
       Self-classification
       Knowledge
   Require less skill and sensitivity to administer than
    interviews.
   Data are usually quantified and analysed by
    computer program
                                                            7
Types of questionnaire
   Self-administered questionnaires
       Completed by the respondents
         On-line questionnaires
         Postal or mail questionnaires
         Delivery and collection questionnaires

   Interviewer-administered questionnaires
       Responses are recorded by the interviewer
         Telephone questionnaires

   Structured interviews
       Interviewers physically meet respondents and ask the
        questions face to face.

                                                               8
The choice of questionnaire
   Characteristics of the respondents from whom you
    wish to collect data
   Importance of reaching a particular person as
    respondent
   Importance of respondents’ answers not being
    contaminated or distorted
   Size of sample you require for your analysis
   Types of questions you need to ask to collect your
    data
   Number of questions you need to ask to collect your
    data.
   Available resources
                                                          9
The choice of questionnaire
   It is difficult to know the actual respondents
    from online or postal questionnaire
   Possible contamination of respondents’
    answers
       In telephone or structured interview, interviewer
        might lead to some certain response
       In self-administered questionnaire, respondents
        might discuss answers with others


                                                            10
The choice of questionnaire
   Telephone or structure interview usually get
    high response rate (50%-70%)
   Postal or delivery and collection
    questionnaire can get moderate response
    rate (30%-50%)
   Online questionnaire usually get lowest
    response
       Hard to measure
       30% within organization; 10% or lower for public
        Internet survey
                                                           11
Deciding what data need to be
collected
   Example
       Problem: towards the end of your information system
        project, you need to measure the satisfaction of your client
       Naïve solution:
         Email your client a short questionnaire contains two
          questions
               1. Are you satisfied with the system we developed? YES/NO
               2. Are you going to use the system in the future? YES/NO
        Declare success to course coordinator because client
         answers “yes” to both questions
       What’s the problem?

                                                                            12
Deciding what data need to be
collected
   Theory
       Suggest constructs of questionnaire
         How do we measure End – User “satisfaction” of computer
          system?
               Content, Accuracy, Format, Ease of Use, Timeliness
           How do we measure perception of adoption?
               Voluntariness, Relative Advantage, Compatibility, Image, Ease of
                use, Result Demonstrability, Visibility, Trialability
       www.isworld.org is an excellent source for Information
        system survey instruments!
       http://www.isworld.org/surveyinstruments/tutor.htm

                                                                                   13
Type of data we collect
through questionnaire
   Attribute
       Respondents’ generally background or some
        special characteristics
   Opinion
       Eg. Do you think the workload of this course is too
        high?
   Behavior
       What do respondents do?
       Eg. How often do you visit your dentist?
                                                          14
Ensuring that essential data
are collected
   What question (s) you want to answer from the questionnaire?
     What might influence a consumer to purchase online?
    What constructs you might need to answer those questions
     Online payment, Internet product choice, Internet customer
      relation, …
   What variables you need to collect data for those constructs
     Customer relation
     Want to know customer’s opinion on:
         It is important to provide an easy return process
         It is important to provide quality after-sale service.
   How you might measure those variables
     Rating or scaling



                                                                   15
Using the data requirements
table
Research Objective: find student bloggers’ experience of using blog
Constructs    Variables required             Detail in which data     Check included in
                                             measured                 questionnaire
Authorship    Years of being a blogger
              Number of blogs authored

Activeness    Frequency of making new
              posts
              Frequency of visiting other
              blogs
              Frequency of commenting on
              other blogs
Content       What do you post?              Personal, technology,
              What do you read?              sports or…

…
                                                                                     16
Designing the questionnaire
   Designing individual questions (items)
       Adopt questions used in other questionnaires
           See survey instruments in ISworld
           Lots of instruments (questions) reported in academic
            papers
       Adapt questions used in other questionnaires
       Develop your own



                                                                   17
Individual question style
   Open-ended question
       Respondents answer in their own words
   Closed-ended question
       Respondents choose a response from those provided
   Examples
       Why do you blog? (OPEN)
       Which one of the reasons make you start your first blog?
         () share daily activities as well as pictures with family
          members
         () share information on a particular hobby
         () it makes me look cool
         () other reasons


                                                                      18
Open vs. Closed
   Open-ended question
       Great freedom for respondent to answer
       Responses may be ambiguous
       Coding is time-consuming and costly which usually results
        in some degree of error
       Entail more work from respondents
   Closed-ended question
       Require less effort and less facility with words
       Difficult to develop good closed questions
       Recommendation for designing closed-end question: use
        open questions in preliminary interviews or pretests

                                                                    19
Closed question style
   List
       The respondent is given a list of choices, any of which may
        be selected
   Category
       Only one response can be selected
   Rank
       Respondent is asked to place something in order
   Rating or Scale (Likert style)
       A rating device is used to record response
   Quantity
       Response is a number giving the amount
                                                                  20
List questions
   Eg.
       Please choose all communication tools you ever
        used in campus.
           Internal Message Board
           WebCT
           E-mail
           Other ___________




                                                         21
Category question
   Each respondent’s answer can only fit one
    category
   Eg.
       How often do you visit this shopping center
           Once a week
           Twice a week or more
           Less than once a week
       What’s wrong with this responses?


                                                      22
Ranking question
Rank-order the three most important things you want in the job you make your life’s work? (1 indicates
the most important one 4 the least important one)

______          Making a lot of money
______          Being creative
______          Being free from supervision
______          Having opportunities for advancement


   Should not ask respondent to rank more than four
    levels
        OK to offer more responses than being ranked
   In telephone questionnaire, should not ask
    respondent to rank more than four responses
                                                                                                         23
Rating or Scale question
   Likert-style question
       Ask respondents how strongly they disagree or agree with a
        statement or a series of statements
       Odd number or even number scales
           Odd number offers a neutral choice
           Even number force respondents to make choice



For the following statement please circle the number that matches your view most closely
 I feel that employee’s views have influenced the decisions taken by management.

        1                 2                 3                4                    5

    Strongly          Disagree           Neutral           Agree            Strongly Agree
    Disagree
                                                                                             24
Quantity question
   Eg.
       When did you start writing blog? ________




                                                    25
Opening and closing of
questionnaire
   A cover letter explaining the purpose of the questionnaire
       Include date, recipients’ name, address and salutation if available
       First set of messages
           What research is about why it is useful
       Second set of messages
           Why recipient’s response is important, how long it will take to complete
       Third set of messages
           Promise of confidentiality or anonymity
       Fourth set of messages
           How results will be used;
       Final set of messages
           Whom to contact if have any queries, who to return filled questionnaires to
            and date by which should be returned
       Closing remarks
           Thank recipient for their help
       Signature, name and title.


                                                                                          26
Opening and closing of
questionnaire
   Introducing the questionnaire
       In the first page of the questionnaire
         Title and/or subtitle
         Summary of key points in the cover letter

   Closing the questionnaire
       Explain what you want the respondents to do with their
        completed questionnaires
       Eg.
         Thank you for taking the time to complete this
          questionnaire. If you have any queries please do not
          hesitate to contact ….
          please return the completed questionnaire to …
                                                                 27
Pilot testing
   Purpose is to refine the questionnaire
       respondents will have no problems in answering them
       You will have no problems in recording the data
       Obtains some assessment of the questions’ validity and
        reliability of the data
   Steps
       Ask expert or group of experts to comment on the
        representativeness and suitability of the questions
       Administer the questionnaire to a group as similar as
        possible to the final population in your sample.

                                                                 28
Pilot testing
   Use pilot testing to find out
       How long the questionnaire took to com plete
       The clarity of instructions
       Which, if any, questions were unclear or ambiguous;
       Which, if any, questions the respondent felt uneasy about
        answering
       Whether in their opinion there were any major topic
        omissions
       Whether the layout was clear and attractive
       Any other comments.

                                                                    29
Administer the questionnaire
   Online questionnaire / postal questionnaires
       Pre-survey contact
       Contact the recipients by email/post
       Send out the first follow-up one week after the
        first contact
       Send out the second follow-up to people who
        have not responded after three weeks.
       Send out a third follow-up if time allows or the
        response rate is low

                                                           30
Administer the questionnaire
   Delivery and collection questionnaires
       Pre-survey contact
       Same day vs. different days
   Telephone questionnaires
       Try to make pre-survey contact
       Record date and time of call and whether or not
        the questionnaire was completed down
       Try a few more times for unsuccessful call

                                                          31

								
To top