How to Conduct Successful Market Research Projects by howardtheduck


									How to Conduct Successful Market
        Research Projects
          September 2000
Contents of the Library Research Kit

•   Survey Research Process
•   Survey Instrument
•   Mail Survey Research
•   Ballot Measure Feasibility Research
•   Qualitative Research
•   Appendices
    – A guide to creating an RFP
    – Layout options for mail survey
Survey Research Process
      Survey Research Process I
      Defining Research Objectives

• Prior to developing specific survey questions and
  the sampling frame.
• What needs to be accomplished by conducting
  the survey?
• Objectives: assess support level for a ballot
  measure vs. gather opinions about current and
  potential services.
     Survey Research Process II
      Developing the Sample Frame

• Primary reason for sampling: reliably
  representing the universe with a smaller group of
• Representativeness of a sample is dependent
  upon the degree of randomization of the sample:
  every individual in the universe has equal chance
  of being included.
     Survey Research Process II
      Developing the Sample Frame

• Sampling frame must be appropriate for the
  research objectives: RDD, voter, or card holder.
• Sampling frame must take into account all
  potential factors that might bias the sample.
• Sampling frame should be relatively easy to
• Sample size should be sufficient to provide an
  acceptable margin or error for important analyses.
     Survey Research Process II
      Developing the Sample Frame

• Random Digit Dial (RDD)
  – Drawn by determining the active phone exchanges in a
    given sampling area.
  – Appropriate for gathering information about an entire
  – Incidence rate.
  – Tendency to under-represent minorities and over-
    represent older women.
  – Telephone book not an acceptable substitute for RDD.
     Survey Research Process II
      Developing the Sample Frame

• Voter File Sample:
  – Appropriate for gathering information among
    registered voters.
  – Stratify and cluster by demographic characteristics.
  – Each cluster represents individuals sharing a similar
    demographic profile.
  – Generally no incidence rate issue.
  – Sample likely voters to increase prediction accuracy.
      Survey Research Process III
        Developing the Questionnaire

• Reliability and validity
• Sources of measurement error:
  –   Question wording effects.
  –   Question order effects.
  –   Response category effects.
  –   Interviewer effects.
  –   Panel effects.
  –   Learning effects.
    Survey Research Process IV
      Developing the Questionnaire

• General rules:
  – Keep the order of questions logical, with smooth
  – Move from general to specific.
  – Make sure respondents understand the wording.
  – Be conversational.
  – Avoid monotony.
  – Include exhaustive and non-overlapping response
     Survey Research Process V
       Pre-testing the Questionnaire

• Estimate the length of the questionnaire.
• Ensure that words, phrases, and subjects are easily
  understood by the respondents.
• Ensure that answer categories match with what the
  respondents have to say.
• Ensure that the questionnaire achieves the research
     Survey Research Process VI
        Language Translation

• Appropriate when the universe contains a
  significant number of linguistically isolated
• Appropriate when the universe contains a
  significant number of bi-lingual respondents who
  are more comfortable in their first language.
• Overriding goal: translation maintains the exact
  meaning of the questions.
    Survey Research Process VII
          Data Collection

• Telephone surveys:
  – Telephone interviewing is efficient and relatively cost-
  – Conducted on weekdays (evening) and weekends (late
  – CATI programming facilitates interviewing process.
    Survey Research Process VII
          Data Collection

• Volunteers useful for:
  – Short, straightforward questionnaires.
  – Cost savings.
  – Potential bias.
• Disadvantages of using volunteers:
  – Higher refusal rate.
  – Not monitored and more interviewer errors.
  – Not equipped with CATI programming technologies
    and skills.
       Survey Research Process VII
             Data Collection

• Criteria for selecting a high-quality data collection
   –   Demographics of the interviewing staff.
   –   Supervisors’ and project managers’ experience.
   –   Ability to monitor remotely.
   –   Multi-lingual interviewing ability.
   –   Proficiency working with different types of samples.
   –   CATI and turnaround time.
    Survey Research Process VII
          Data Collection

• Mail surveys:
  – Appropriate for reaching certain individuals (e.g.,
    individuals without telephones or without identifiable
  – Disadvantages: lack of control over respondents, low
    response rate, long data collection period, and bias
    based on respondents’ self-selection.
  – Need for incentives.
     Survey Research Process VII
           Data Collection

• Intercept surveys:
   – Administered face-to-face.
   – Effective for reaching individuals concentrated at a
     specific location.
   – Attempt to contact every ‘nth’ person.
   – Higher response rate than mail surveys but a sound
     sampling design for an intercept survey can be
     Survey Research Process VII
           Data Collection

• Internet surveys:
   – Represent only a select group of individuals: those with
     a known email address or visitors to a site.
   – HTML questionnaire at a selected URL address.
   – Sending the URL to potential respondents via email.
   – Respondents click on the URL link and complete the
   – Efficient, accurate, and cost-effective.
   Survey Research Process VIII
         Data Processing
• When using a professional data collection
  – Data normally are provided as a ‘flat’ ASCII file.
  – Data map is needed to locate variable values and
  – An example of a ‘flat’ ASCII file is provided on the
    next slide.
Survey Research Process VIII
      Data Processing
              0   0   1   2   0   2   0   5   6
              0   0   2   4   0   3   2   5   2
              0   0   3   5   0   5   2   6   2
              0   0   4   3   1   1   3   2   1
              0   0   5   3   0   5   5   3   2
              0   0   6   2   0   5   5   3   3
              0   0   7   5   0   4   6   0   3
              0   0   8   5   0   4   2   0   2
              0   0   9   2   1   1   0   5   5
 Respondent   0   1   0   1   0   4   0   2   2
    # 10      0   1   2   0   0   0   1   0   5
              0   1   3   2   0   0   2   5   5
              0   1   4   2   0   5   1   6   6
              0   1   5   5   0   3   2   6   0

                              Column 6
   Survey Research Process VIII
         Data Processing
• When conducting the survey with volunteers:
  – Record all responses on paper.
  – Keypunch responses into computer for data
  – Software packages to use for keypunching: Excel,
    SPSS, or SNAP.
  – Each column is a variable and each row is a
   Survey Research Process VIII
         Data Processing
• Tasks involved:
  – Merging multiple data sets into one.
  – Checking and cleaning data.
  – Coding and re-coding verbatim responses into
    numerical values.
  – Aggregating all responses to interpret the
    population as a whole.
  – Deriving variables to perform necessary analyses.
      Survey Research Process VIII
            Data Processing
• Weighting data:
  –   Adjust a known bias in the sample.
  –   Ensure the representativeness of the sample.
  –   Weight data on a single dimension.
  –   Weight data on multiple dimensions.
    Survey Research Process IX
           Data Analysis
• Statistical software packages to use: SPSS,
• Statistical techniques: mean, mode, median,
  crosstabulation, correlation, factor analysis,
  and regression analysis.
        Survey Research Process X
              Report Writing
• Recap of objectives and methodology
• Detailed question-by-question analysis.
   – Topline results.
   – Segmentation of key audiences.
• Key findings.
• Conclusions and recommendations.

  Note: Examples of GRA’s final reports are provided on the CD-ROM.
Sample Questions: Customer
              Survey Instrument
• Brief introduction and do not reveal the specific purpose of
  the survey.

• Qi. For statistical reasons, I would like to speak to the
  youngest male currently at home that is at least 18 years
  ASK): Ok, then I’d like to speak to the youngest female
  currently at home that is at least 18 years of age (IF NO

• Qii. In what city do you live?
              Survey Instrument
• Q1. How long have you lived in <City that library

• Q2. What information sources do you use to find out about
  <City name> news, information and events? (DON’T

• Q3. In the last three months have you ________?

• Example of skip pattern: If Respondents indicated ‘2’ for
  Q3A, then ask Q4, otherwise, skip to Q5.
              Survey Instrument
• Q4. What is the main reason you have not used your
  public library recently? (DON’T READ LIST, ONE

• Q5. Your local library offers many programs and services.
  I’m going to read several of these and I’d like you to tell
  me if you are aware that they are available. Are you aware
  that ____ (is/are) available at your local branch library?
            Survey Instrument
• Q6. In your opinion, how important is it for libraries to
  ______? Is it very important, somewhat important, not too
  important, or not at all important? (CONTINUE UNTIL ALL

• Q7. Are you satisfied or dissatisfied with your local
  library’s efforts to _____? (GET ANSWER, THEN ASK): Is
  that very (satisfied/dissatisfied) or somewhat
             Survey Instrument
• Q8 & Q9. If your library: ______, would you be more or
  less likely to use the library? (GET ANSWER, THEN ASK):
  Is that much (more/less) likely or somewhat (more/less)
• Q10. Did you know the <library name> maintains its own
• Q11. Have you ever visited your local library’s website?
• Q12.What have you used the website for?
• Q13. Are you generally satisfied or dissatisfied with the
  resources available on the library’s website?
             Survey Instrument
• Q14. If your library _________, would you very interested,
  somewhat interested, not too interested, or not at all
  interested in this service?

• Q15. If you knew that, with a personalized online library
  account, you could _________, would you be very
  interested, somewhat interested, not too interested, or not
  at all interested in using this service?

• Demographic questions
Using Mail Surveys
     Mail Survey Research I
  When Are Mail Surveys Useful?
• When a particular group is difficult or
  impossible to reach by telephone.
• When the group of interest is only library
  users: intercept combined with mail surveys.
• When there is not an immediate deadline.
       Mail Survey Research II
        Acquiring the Sample
• A complete list of potential respondents.
• Contact information including, at minimum, the
  mailing address should be provided.
• Additional information preferred: name,
  geographic location, gender, and etc.
      Mail Survey Research III
           Graphic Design
• Visually appealing and reader-friendly.
• Use graphic design to increase response rate.
• Professional graphic designers.
      Mail Survey Research IV
     Envelope and Cover Letter
• Envelop is the first attempt to get potential
  respondents to open the survey.
• Distinguish the survey from other direct mail or
• If possible, specify the name of the recipient.
• Take advantage of a cover letter to introduce
  potential respondents to the survey.
        Mail Survey Research V
• First class mail:
  –   33 cents per mail piece.
  –   1-6 days delivery time.
  –   Zip codes not required for delivery.
  –   Mail forwarding free for 6 to 12 months.
• Standard rate mail:
  – 18 cents per piece.
  – Up to 21 days delivery time.
  – Valid zip codes required.
      Mail Survey Research VI
• To increase response rate.
• An effective yet economical incentive: eligibility
  in a drawing for a prize such as a gift
  certificate or a cash prize.
      Mail Survey Research VII
Pre-notification and Follow-up Letters
 • Two other approaches to increase response
 • Pre-notification letters notify respondents of
   the survey and explains the objectives.
 • Follow-up letters are sent one week after the
   surveys have been mailed out to kindly
   encourage respondents to complete and return
   the survey in time.
Ballot Measure Research &
Ballot Measure Feasibility Research I
  Developing the Feasibility Survey
 • Determine the level of support for a new tax
   and at what tax amount the support is
 • Identify the spending projects that resonate
   best with voters.
 • Determine the arguments for the measure
   that are most effective in persuading voters to
   support the measure as well as those that
   could weaken the support.
Ballot Measure Feasibility Research II
      Writing the Ballot Question
 • No longer than 75 words.
 • Basic approach: state the original problem,
   followed by features of the measure that will
   contribute to the proposed solution.
 • Features should be limited to the compelling
   ones found in the survey.
 • Tax level is mentioned as the necessary
   means to achieve the goal.
Ballot Measure Feasibility Research III
      The Information Campaign
  • Identify and turnout voters whose support for
    the measure was greatest.
  • Be offensive with benefits and supporting
  • Avoid reiterating detracting arguments even
    in an attempt to refute those arguments.
  • Seek assistance from a research consultant
    and a political consultant.
Qualitative Research
     Qualitative Research I
  What Is Qualitative Research?
• Utilizes open-ended interviewing
• Explore and understand opinions and
• Not representative.
• Very small sample size.
• Examples: focus groups and executive
       Qualitative Research II
           Common Uses
• Pre-survey: developing hypotheses to be tested
  in a telephone or mail survey.
• Post-survey: in-depth exploration of previous
  quantitative findings.
• Exploration of potential service and program
• Identifying strengths and weaknesses.
• Exploring communication messages.
• Understanding usage motivations.
       Qualitative Research III
            Focus Groups
• 8 to 10 participants.
• Typically takes two hours to conduct.
• Appropriate when research objectives require
  qualitative, in-depth responses from a select
  group of participants.
• Findings can not be generalized to the larger
• Recruiting, incentives, and facility rental.
       Qualitative Research IV
        Executive Interviews
• Similar to focus groups, to explore issues and
  attitudes through in-depth discussions.
• One-on-one interviews.
• In-person or via telephone.
Creating an Effective RFP
           Creating an RFP I
         Developing Objectives
•   Desired margin of error
•   Preferred survey type
•   Desired completion date
•   Project budget
•   Expected language of survey
        Creating an RFP II
Request a Fixed Fee Price Proposal

Mail Survey           Telephone Survey        Intercept Survey
Random Sample         Random Sample           Material Printing
Material Printing     Questionnaire Copying   Incentive Costs
Incentive Costs       or CATI Programming     Data Collection
Mail House Costs      Data Collection         Data Processing
Postage               Data Processing         Project Management
Data Processing       Project Management      Research Fee
Project Management    Research Fee            Miscellaneous Costs
Research Fee          Miscellaneous costs
Miscellaneous Costs
             Creating an RFP III
              Cover the Details
• Request in RFP:
  –   Project Contact/Firm Address and Telephone
  –   Work Plan
  –   Firm Background
  –   Personnel Background
  –   Project Organizational Chart
  –   References/Project Descriptions
  –   Cost Proposal
         Creating an RFP IV
     Personalize to the Industry
• Specific questions to ask:
  – What benefits can your firm offer?
  – Briefly state the best way to achieve the research
  – What is your firm’s competitive advantage?
  – How much does your firm know about the general
    topics covered by the research?
  – What are the biggest research challenges?
  – How will you create a random sample?
      What you can find in CD-ROM

•   Quantitative Report
•   PowerPoint Presentation
•   Focus Group Report
•   Executive Interview Report
•   Questionnaire Template
•   Margin of Error Calculator
•   Mail Survey Template I and II
•   Template User Instructions

To top