A Media Guide to Survey Research - University of Nebraska–Lincoln

Document Sample
A Media Guide to Survey Research - University of Nebraska–Lincoln Powered By Docstoc
					                                A Media Guide to Survey Research

        The World Association for Public Opinion Research (WAPOR) is dedicated to developing
the science of survey research and promoting its appropriate use around the world. It specifically
seeks to "promote international cooperation and exchange" with journalists and inform them about
"the appropriate forms of publishing poll results."1 To further these goals, WAPOR here provides
information to journalists on learning about survey research and about what facts they should know
when using surveys in a news story.

                                  Learning about Survey Research

        Just as journalists who cover politics need to be familiar with the political system and
reporters covering legal cases need to be versed in judicial procedures, journalists using surveys
need at least a basic grounding in survey research. As Katherine Wallman, president of the
American Statistical Association (ASA) has noted, "statistical literacy" is required to report stories
using surveys and other statistical data (Wallman, Katherine K., "Enhancing Statistical Literacy:
Enriching Our Society," Journal of the American Statistical Association, 88 (March, 1993), 1-8).
        First, the best way for a journalist to gain competency about surveys is to complete an
advanced degree in statistics or in quantitative social science. For a list of some graduate programs
specializing in survey research see http://www.aapor.org/. Of course this possibility is not
practical for many journalists.
        Second, a number of organizations offer short courses ranging from a few hours to a few
weeks in survey methods. These include courses at:

          the Essex Summer School in Social Science Data Analysis and Collection

          the Interuniversity Consortium for Political and Social Research Summer Program in
          Quantitative Methods at the University of Michigan

          the Joint Program in Survey Methodology at                the   University of     Maryland

          the Summer Institute in Survey Research Technique at the University of Michigan

          the annual meetings of the American Association for Public Opinion Research

      WAPOR Constitution, Article 2, Sections 1 and 2.

        Third, one can attend the general sessions of the annual conferences of the leading survey-
research associations:

       the World Association for Public Opinion Research (held in alternating years in Europe and
       North America) (http://www.unl.edu/wapor)

       The World Association of Research Professionals (http://www.esomar.org/)

       the various national, survey-research associations such as those listed by ESOMAR

       In addition, WAPOR frequently holds regional conferences around the world

       Fourth, there are a number of publications specially designed to inform journalists about

       Council of American Survey Research Organizations, "CASRO Media Kit," 1999-2002 at

       Friend, Cecilia; Challenger, Don; and McAdams, Katherine C., Contemporary Editing.
       Lincolnwood, IL: NTC/Contemporary, 2000.

       Gawiser, Seldon R. and Witt, G. Evans, A Journalist's Guide to Public Opinion Polls.
       Westport, CT: Praeger, 1994. See http://www.greenwood.com

       Gawiser, Sheldon R. and Witt, G. Evans, 20 Questions a Journalist Should Ask About Poll
       Results. 2nd edition
       National Council for Public Polls at http://www.ncpp.org/qajsa.htm

       Meyer, Philip, The New Precision Journalism. 4th edition. Lanham, MD: Rowman &
       Littlefield, 2002. See http://www.rowmanlittlefield.com/

       Worcester, Robert M., Journalists' Guide to the Publication of Opinion Survey Results.
       London: Market & Opinion Research International, 1987.

       Fifth, there are similar materials aimed at informing general audiences about surveys.
While not specifically geared to journalists, they also provide highly useful information about

       American Association for Public Opinion Research, Best Practices for Survey and Public
       Opinion                     Research.                      n.d.                     at

       American Association for Public Opinion Research, Code of Professional Ethics and
       Standards, 1986, at

       American Statistical Association, "ASA Series: What                                                   is     a      Survey?"      at

       Asher, Herbert, Polling and the Public: What Every Citizen Should Know. 5th edition.
       Washington, DC: CQ Press, 2001.
       See http://www.cqpress.com/

       Council of American Survey Research Organizations, "CASRO Guidelines for Survey
       Research Quality," 1998 at http://www.casro.org/guidelines.cfm

       Council of American Survey Research Organizations, "Code of Standards and Ethnic for
       Survey Research," 1999-2002 at http://www.casro.org/codeofstandards.cfm

       European Society for Opinion and Marketing Research, ESOMAR Guide to Opinion Polls.
       1999 at http://www.esomar.org

       European Society for Opinion and Marketing Research, "International Code of Marketing
       and Social Research Practice," 1998 at http://www.esomar.org/

       Lavrakas, Paul J. and Traugott, Michael W., eds., Elections Polls, the News Media, and
       Democracy. New York: Chatham House, 2000. See http://www.sevenbridgespress.com/

       McDonald, Colin and Vangelder, Phyllis, eds., ESOMAR Handbook of Market and
       Opinion Research. 4th edition. Amsterdam: ESOMAR, 1998. See http://www.esomar.org/

       National Council on Public Polls,                                      "Statement           of      Disclosure,"           n.d.   at

       Traugott, Michael W. and Paul J. Lavrakas, The Voter's Guide to Election Polls. 2nd
       edition.   Chatham,     NJ:     Chatham     House      Publishing,    1999.     See

        Finally, there are various professional journals that can be consulted for the latest
developments in survey research. These include the top two professional journals, WAPOR's
International Journal of Public Opinion Research and AAPOR's Public Opinion Quarterly. Other
important journals include the International Journal of Market Research, the Journal of Marketing
Research, the Journal of Official Statistics, the Polling Report, Public Perspective, Research World,
and Survey Methodology.

                                 The Needed Facts about Surveys

        Whenever using a survey in a story, journalists need to obtain basic methodological
information on the data. The WAPOR Code of Professional Ethics and Practices
(http://www.unl.edu/wapor/ethics.html) lists essential facts that should be included in all reports
on surveys and therefore known about a survey that is used in a news story. In abbreviated form
they are:

       1. who commissioned the survey
       2. who conducted the survey
       3. the purpose of the survey
       4. the universe the survey covers
       5. sampling method and procedures
       6. non-response rate
       7. sample size (number of cases)
       8. weighting procedures (if used)
       9. data collection method
       10. when data collected
       11. results
       12. characteristics of interviewers and coders and their training
       13. copy of questionnaire
       14. results for sub-samples vs. whole sample
       15. precision of findings and sampling error when applicable
       16. standard, scientific use of technical terms

        Moreover, there is a high degree of agreement amongst survey-research organizations about
what elements of surveys are crucial to report (Table 1). In nine documents from five survey-
research organizations (World Association for Public Opinion Research - WAPOR; American
Association for Public Opinion Research - AAPOR; Council of American Survey Research
Organizations - CASRO; European Society for Opinion and Marketing Research - ESOMAR;
National Council on Public Polls - NCPP) there is complete agreement that the following
information about surveys should be disclosed:

       Who Conducted the Survey
       Sample Design
       Sample Size
       Mode of Data Collection
       When Collected/Dates
       Question Wording

       Sample Population
       Response Rate

In addition, eight of nine documents (and all five organizations)
also mentioned:

       Who Sponsored the Survey
       Sampling Error/Confidence Intervals
       Question Order

       Other elements mentioned by all survey-research organizations, but in fewer documents,

       Purpose of Survey

        As many of these essential facts as possible should be included in news reports using
surveys. Typically all of this information can be covered in two-to-three sentences. When it is not
possible to include all of the information, journalists should be prepared to provide it upon request.

                                                              Table 1

                                       Disclosure Standards of Survey-Research Organizations

Organizations                               Standards

                             1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

 WAPOR                       x x x x x x x x x x x x x x x
 Code                        x x x x x x x x . x x x x
 Best Practices              x x x x x x x x x x x     x
 Code                        X X X X x X x X x x x x              X
 Guidelines                  X X x X x X X x x x x x .            x
 Code                        x x x x x x x x x x x x          x
 Guide                       x . x x x x x x x x
 Statement            .      X X x X X x X x X x x X
 20 Questions                x x x x x x x x x x x x x

. = Indicates some mention of this element, but reporting requirement is either limited or unclear.
x = Indicates required. When both lower (x) and upper (X) cases Xs are used, this means that higher priority or greater importance is
assigned to the element marker the upper case X.


1=Who Conducted
2=Who Sponsored
3=Sample Size

4=Sample Design
6=When Conducted
7=Sampling Error
8=Question Wording
9=Question Order
10=Sample Population
12=Response Rate

Sources: AAPOR, 1986; AAPOR, n.d.; CASRO, 1998; CASRO, 1999-2002b; NCPP, n.d.; WAPOR, 2002


Shared By: