questionnaire by 196T3f1

VIEWS: 694 PAGES: 31

									  Collecting primary
data: questionnaires
         Week 10 lecture 1

   Questionnaire introduction
   When to use questionnaire
   Types of questionnaire
   Identify data to be collected by questionnaire
   Designing questionnaire
       Design individual questions in questionnaire
       Opening and closing of questionnaire
       Pilot testing
   Administering the questionnaire
   Used extensively in survey research
   Can be used in experiment and case study
    research strategies
   Strict definition
       Surveys where the person answering the question
        actually records their own answers.
   Broad view
       Include all techniques of data collection in which
        each person is asked to respond to the same set
        of questions in a predetermined order.
Important design issues
   It is far harder to produce a good
    questionnaire than you might think
   Issues to maximize response rates, validity
    and reliability
       Careful design of individual questions (item)
       Clear layout of the questionnaire form
       Lucid explanation of the purpose of the
       Pilot testing
When to use questionnaires
   Not particularly good for exploratory study or other
    research that requires large number of open-ended
       What would be the most suitable data collection method for
        exploratory study?
       Is questionnaire a good way to collect user requirements?
   Work best with standardized questions that you can
    be confident will be interpreted the same way by all
       Descriptive or explanatory research

When to use questionnaire
   Descriptive research
       Opinion, attitude and experience of something
       Eg. End-user satisfaction of a new system
   Explanatory research
       Cause-and-effect relationship between variables
         Eg. Would color scheme of an e-commerce website affect
          customers’ purchasing behavior
   Multi-method
       Eg. A questionnaire to discover customers’ attitudes
        followed by in-depth interviews to explore and understand
        these attitudes.

When to use questionnaire
   To collect self-reported beliefs or behaviors.
       Behavior
       Attitudes/beliefs/opinions
       Expectations
       Self-classification
       Knowledge
   Require less skill and sensitivity to administer than
   Data are usually quantified and analysed by
    computer program
Types of questionnaire
   Self-administered questionnaires
       Completed by the respondents
         On-line questionnaires
         Postal or mail questionnaires
         Delivery and collection questionnaires

   Interviewer-administered questionnaires
       Responses are recorded by the interviewer
         Telephone questionnaires

   Structured interviews
       Interviewers physically meet respondents and ask the
        questions face to face.

The choice of questionnaire
   Characteristics of the respondents from whom you
    wish to collect data
   Importance of reaching a particular person as
   Importance of respondents’ answers not being
    contaminated or distorted
   Size of sample you require for your analysis
   Types of questions you need to ask to collect your
   Number of questions you need to ask to collect your
   Available resources
The choice of questionnaire
   It is difficult to know the actual respondents
    from online or postal questionnaire
   Possible contamination of respondents’
       In telephone or structured interview, interviewer
        might lead to some certain response
       In self-administered questionnaire, respondents
        might discuss answers with others

The choice of questionnaire
   Telephone or structure interview usually get
    high response rate (50%-70%)
   Postal or delivery and collection
    questionnaire can get moderate response
    rate (30%-50%)
   Online questionnaire usually get lowest
       Hard to measure
       30% within organization; 10% or lower for public
        Internet survey
Deciding what data need to be
   Example
       Problem: towards the end of your information system
        project, you need to measure the satisfaction of your client
       Naïve solution:
         Email your client a short questionnaire contains two
               1. Are you satisfied with the system we developed? YES/NO
               2. Are you going to use the system in the future? YES/NO
        Declare success to course coordinator because client
         answers “yes” to both questions
       What’s the problem?

Deciding what data need to be
   Theory
       Suggest constructs of questionnaire
         How do we measure End – User “satisfaction” of computer
               Content, Accuracy, Format, Ease of Use, Timeliness
           How do we measure perception of adoption?
               Voluntariness, Relative Advantage, Compatibility, Image, Ease of
                use, Result Demonstrability, Visibility, Trialability
     is an excellent source for Information
        system survey instruments!

Type of data we collect
through questionnaire
   Attribute
       Respondents’ generally background or some
        special characteristics
   Opinion
       Eg. Do you think the workload of this course is too
   Behavior
       What do respondents do?
       Eg. How often do you visit your dentist?
Ensuring that essential data
are collected
   What question (s) you want to answer from the questionnaire?
     What might influence a consumer to purchase online?
    What constructs you might need to answer those questions
     Online payment, Internet product choice, Internet customer
      relation, …
   What variables you need to collect data for those constructs
     Customer relation
     Want to know customer’s opinion on:
         It is important to provide an easy return process
         It is important to provide quality after-sale service.
   How you might measure those variables
     Rating or scaling

Using the data requirements
Research Objective: find student bloggers’ experience of using blog
Constructs    Variables required             Detail in which data     Check included in
                                             measured                 questionnaire
Authorship    Years of being a blogger
              Number of blogs authored

Activeness    Frequency of making new
              Frequency of visiting other
              Frequency of commenting on
              other blogs
Content       What do you post?              Personal, technology,
              What do you read?              sports or…

Designing the questionnaire
   Designing individual questions (items)
       Adopt questions used in other questionnaires
           See survey instruments in ISworld
           Lots of instruments (questions) reported in academic
       Adapt questions used in other questionnaires
       Develop your own

Individual question style
   Open-ended question
       Respondents answer in their own words
   Closed-ended question
       Respondents choose a response from those provided
   Examples
       Why do you blog? (OPEN)
       Which one of the reasons make you start your first blog?
         () share daily activities as well as pictures with family
         () share information on a particular hobby
         () it makes me look cool
         () other reasons

Open vs. Closed
   Open-ended question
       Great freedom for respondent to answer
       Responses may be ambiguous
       Coding is time-consuming and costly which usually results
        in some degree of error
       Entail more work from respondents
   Closed-ended question
       Require less effort and less facility with words
       Difficult to develop good closed questions
       Recommendation for designing closed-end question: use
        open questions in preliminary interviews or pretests

Closed question style
   List
       The respondent is given a list of choices, any of which may
        be selected
   Category
       Only one response can be selected
   Rank
       Respondent is asked to place something in order
   Rating or Scale (Likert style)
       A rating device is used to record response
   Quantity
       Response is a number giving the amount
List questions
   Eg.
       Please choose all communication tools you ever
        used in campus.
           Internal Message Board
           WebCT
           E-mail
           Other ___________

Category question
   Each respondent’s answer can only fit one
   Eg.
       How often do you visit this shopping center
           Once a week
           Twice a week or more
           Less than once a week
       What’s wrong with this responses?

Ranking question
Rank-order the three most important things you want in the job you make your life’s work? (1 indicates
the most important one 4 the least important one)

______          Making a lot of money
______          Being creative
______          Being free from supervision
______          Having opportunities for advancement

   Should not ask respondent to rank more than four
        OK to offer more responses than being ranked
   In telephone questionnaire, should not ask
    respondent to rank more than four responses
Rating or Scale question
   Likert-style question
       Ask respondents how strongly they disagree or agree with a
        statement or a series of statements
       Odd number or even number scales
           Odd number offers a neutral choice
           Even number force respondents to make choice

For the following statement please circle the number that matches your view most closely
 I feel that employee’s views have influenced the decisions taken by management.

        1                 2                 3                4                    5

    Strongly          Disagree           Neutral           Agree            Strongly Agree
Quantity question
   Eg.
       When did you start writing blog? ________

Opening and closing of
   A cover letter explaining the purpose of the questionnaire
       Include date, recipients’ name, address and salutation if available
       First set of messages
           What research is about why it is useful
       Second set of messages
           Why recipient’s response is important, how long it will take to complete
       Third set of messages
           Promise of confidentiality or anonymity
       Fourth set of messages
           How results will be used;
       Final set of messages
           Whom to contact if have any queries, who to return filled questionnaires to
            and date by which should be returned
       Closing remarks
           Thank recipient for their help
       Signature, name and title.

Opening and closing of
   Introducing the questionnaire
       In the first page of the questionnaire
         Title and/or subtitle
         Summary of key points in the cover letter

   Closing the questionnaire
       Explain what you want the respondents to do with their
        completed questionnaires
       Eg.
         Thank you for taking the time to complete this
          questionnaire. If you have any queries please do not
          hesitate to contact ….
          please return the completed questionnaire to …
Pilot testing
   Purpose is to refine the questionnaire
       respondents will have no problems in answering them
       You will have no problems in recording the data
       Obtains some assessment of the questions’ validity and
        reliability of the data
   Steps
       Ask expert or group of experts to comment on the
        representativeness and suitability of the questions
       Administer the questionnaire to a group as similar as
        possible to the final population in your sample.

Pilot testing
   Use pilot testing to find out
       How long the questionnaire took to com plete
       The clarity of instructions
       Which, if any, questions were unclear or ambiguous;
       Which, if any, questions the respondent felt uneasy about
       Whether in their opinion there were any major topic
       Whether the layout was clear and attractive
       Any other comments.

Administer the questionnaire
   Online questionnaire / postal questionnaires
       Pre-survey contact
       Contact the recipients by email/post
       Send out the first follow-up one week after the
        first contact
       Send out the second follow-up to people who
        have not responded after three weeks.
       Send out a third follow-up if time allows or the
        response rate is low

Administer the questionnaire
   Delivery and collection questionnaires
       Pre-survey contact
       Same day vs. different days
   Telephone questionnaires
       Try to make pre-survey contact
       Record date and time of call and whether or not
        the questionnaire was completed down
       Try a few more times for unsuccessful call


To top