Census 2000 Topic Report #6 Census

Document Sample
Census        2000 Topic Report #6 Census Powered By Docstoc
					Census 2000 Topic Report No. 6                                                    Issued February 2004

Census 2000 Testing, Experimentation,                                             TR-6



and Evaluation Program




Evaluations
of the
Census 2000
Partnership
and Marketing
Program

                                        U.S. Department of Commerce
                                        Economics and Statistics Administration
                                        U.S. CENSUS BUREAU
Acknowledgments

The Census 2000 Evaluations Executive Steering              George A. Sledge, Mary Ann Sykes, and Cassandra
Committee provided oversight for the Census 2000            H. Thomas provided coordination support. Florence
Testing, Experimentation, and Evaluations (TXE)             H. Abramson provided editorial review.
Program. Members included Cynthia Z. F. Clark,
Associate Director for Methodology and Standards;           This report was prepared under contract by W.
Preston J. Waite, Associate Director for Decennial          Sherman Edwards and Michael J. Wilson of Westat,
Census; Carol M. Van Horn, Chief of Staff; Teresa           Inc. The contract manager was George A. Sledge and
Angueira, Chief of the Decennial Management                 the technical expert was Emilda B. Rivers of the
Division; Robert E. Fay III, Senior Mathematical            Planning, Research and Evaluation Division. The fol­
Statistician; Howard R. Hogan, (former) Chief of the        lowing authors and project managers prepared Census
Decennial Statistical Studies Division; Ruth Ann            2000 experiments and evaluations that contributed to
Killion, Chief of the Planning, Research and Evaluation     this report:
Division; Susan M. Miskura, (former) Chief of the           Decennial Statistical Studies Division:
Decennial Management Division; Rajendra P. Singh,               Sarah E. Brady
Chief of the Decennial Statistical Studies Division;            Darlene A. Moul
Elizabeth Ann Martin, Senior Survey Methodologist;              Herbert F. Stackhouse
Alan R. Tupek, Chief of the Demographic Statistical
Methods Division; Deborah E. Bolton, Assistant              Planning, Research and Evaluation Division:
Division Chief for Program Coordination of the                   Sherri J. Norris
Planning, Research and Evaluation Division; Jon R.               James M. Poyer
Clark, Assistant Division Chief for Census Design of
the Decennial Statistical Studies Division; David L.        Independent contractors:
Hubble, (former) Assistant Division Chief for                    Bob Calder, National Opinion Research Center
Evaluations of the Planning, Research and Evaluation             Jennifer Crafts, Westat, Inc.
Division; Fay F. Nash, (former) Assistant Division Chief
                                                                 Ed Malthouse, National Opinion Research Center
for Statistical Design/Special Census Programs of the
Decennial Management Division; James B. Treat,                   Richard Mantovani, Macro International
Assistant Division Chief for Evaluations of the Planning,        Sally Murphy, National Opinion Research Center
Research and Evaluation Division; and Violeta                    Steven Pedlow, National Opinion Research Center
Vazquez of the Decennial Management Division.                    Javier Porras, National Opinion Research Center
                                                                 Joann Sorra, Westat, Inc.
As an integral part of the Census 2000 TXE Program,
the Evaluations Executive Steering Committee char­               Kirk Wolter, National Opinion Research Center
tered a team to develop and administer the Census
                                                            Greg Carroll and Everett L. Dove of the Admin­
2000 Quality Assurance Process for reports. Past and
present members of this team include: Deborah E.            istrative and Customer Services Division, and Walter
Bolton, Assistant Division Chief for Program                C. Odom, Chief, provided publications and printing
Coordination of the Planning, Research and Evaluation       management, graphic design and composition, and edi­
Division; Jon R. Clark, Assistant Division Chief for        torial review for print and electronic media. General
Census Design of the Decennial Statistical Studies          direction and production management were provided
Division; David L. Hubble, (former) Assistant Division      by James R. Clark, Assistant Division Chief, and
Chief for Evaluations and James B. Treat, Assistant
                                                            Susan L. Rappa, Chief, Publications Services Branch.
Division Chief for Evaluations of the Planning, Research
and Evaluation Division; Florence H. Abramson,
Linda S. Brudvig, Jason D. Machowski, and
Randall J. Neugebauer of the Planning, Research
and Evaluation Division; Violeta Vazquez of the
Decennial Management Division; and Frank A.
Vitrano (formerly) of the Planning, Research and
Evaluation Division.
The Census 2000 TXE Program was coordinated by the
Planning, Research and Evaluation Division: Ruth Ann
Killion, Division Chief; Deborah E. Bolton, Assistant
Division Chief; and Randall J. Neugebauer and
George Francis Train III, Staff Group Leaders. Keith
A. Bennett, Linda S. Brudvig, Kathleen Hays
Guevara, Christine Louise Hough, Jason D.
Machowski, Monica Parrott Jones, Joyce A. Price,
Tammie M. Shanks, Kevin A. Shaw,
Census 2000 Topic Report No. 6                                 Issued February 2004

       Census 2000 Testing, Experimentation,                   TR-6


                    and Evaluation Program




        EVALUATIONS OF THE
    CENSUS 2000 PARTNERSHIP
    AND MARKETING PROGRAM




                       U.S. Department of Commerce
                                   Donald L. Evans,
                                            Secretary

                                   Samuel W. Bodman,
                                      Deputy Secretary

                Economics and Statistics Administration
                                    Kathleen B. Cooper,
                      Under Secretary for Economic Affairs

                                      U.S. CENSUS BUREAU
                                  Charles Louis Kincannon,
                                                    Director
          Suggested Citation
           W. Sherman Edwards and
                    Michael J. Wilson
               Census 2000 Testing,
    Experimentation, and Evaluation
  Program Topic Report No. 6, TR-6,
     Evaluations of the Census 2000
Partnership and Marketing Program,
                U. S. Census Bureau,
                                             ECONOMICS

             Washington, DC 20233
                                          AND STATISTICS

                                         ADMINISTRATION




                                        Economics and Statistics
                                        Administration

                                        Kathleen B. Cooper,
                                        Under Secretary for Economic Affairs




                                        U.S. CENSUS BUREAU
                                        Charles Louis Kincannon,
                                        Director

                                        Hermann Habermann,
                                        Deputy Director and Chief Operating Officer

                                        Cynthia Z. F. Clark,
                                        Associate Director for Methodology and Standards

                                        Preston J. Waite,
                                        Associate Director for Decennial Census

                                        Teresa Angueira,
                                        Chief, Decennial Management Division

                                        Ruth Ann Killion,
                                        Chief, Planning, Research and Evaluation Division


                                        For sale by the Superintendent of Documents, U.S. Government Printing Office
                                        Internet: bookstore.gpo.gov Phone: toll-free 866-512-1800; DC area 202-512-1800
                                        Fax: 202-512-2250 Mail: Stop SSOP, Washington, DC 20402-0001
                     Contents



                                 Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

                                 1.	 Introduction and Background . . . . . . . . . . . . . . . . . . . . . .                ..1
                                     1.1 The Partnership and Marketing Program . . . . . . . . . . .                        ..1
                                     1.2 Partnership and Marketing Program evaluation activities                             .3
                                     1.3 Previous evaluations of marketing and outreach efforts                             ..4
                                     1.4 Other related surveys . . . . . . . . . . . . . . . . . . . . . . . .              ..4

                                 2.	 Summary and Assessment of Evaluation Activities . . . . .                          ....5
                                     2.1 Description of the evaluation studies . . . . . . . . . . .                    ....5
                                     2.2 Findings of the evaluation studies . . . . . . . . . . . . .                   ....6
                                     2.3 Discussion of individual evaluation studies' strengths
                                         and limitations . . . . . . . . . . . . . . . . . . . . . . . . . . .          . . . 13
                                     2.4 Cross-study evaluation topics . . . . . . . . . . . . . . . . .                . . . 15

                                 3. Summary of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

                                 4. Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

                                 References       . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

                                 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

                                 LIST OF TABLES

                                 Table 1. 	Diverse America likelihood spectrum: attitudes and
                                           role of advertising by segment . . . . . . . . . . . . . . . . . . .1

                                 Table 2. General advertising plans by phase                      ...............2

                                 Table 3. 	Significant differences (Chi-square) in census beliefs
                                           by recent awareness of the census . . . . . . . . . . . . . . . . 9

                                 Table 4. 	Significant differences (Chi-square) in self-reported
                                           census participation by whether positive beliefs
                                           reported . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

                                 Table 5. 	Final return rates for PMPE Wave 2 survey respondents by
                                           sample and for corresponding groups in the general
                                           population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

                                 FIGURE

                                 1. Schedule of Census 2000 Activities . . . . . . . . . . . . . . . . . . .25




U.S. Census Bureau               Evaluations of the Census 2000 Partnership and Marketing Program iii
This page intentionally left blank.
               Foreword
   The Census 2000 Testing, Experimentation, and Evaluation Program
                           provides measures of effectiveness for the Census 2000 design,
                           operations, systems, and processes and provides information on
                           the value of new or different methodologies. By providing measures
                           of how well Census 2000 was conducted, this program fully sup-
                           ports the Census Bureau’s strategy to integrate the 2010 planning
                           process with ongoing Master Address File/TIGER enhancements and
                           the American Community Survey. The purpose of the report that
                           follows is to integrate findings and provide context and background
                           for interpretation of related Census 2000 evaluations, experiments,
                           and other assessments to make recommendations for planning
                           the 2010 Census. Census 2000 Testing, Experimentation, and
                           Evaluation reports are available on the Census Bureau’s Internet site
                           at: www.census.gov/pred/www/.




U.S. Census Bureau              Evaluations of the Census 2000 Partnership and Marketing Program v
This page intentionally left blank.
1. Introduction and Background


1.1 The Partnership and                          expensive and (2) responses                    model of the United States popula/
Marketing Program                                received through the mail appear               tion, called the Likelihood
                                                 to be more complete and accurate               SpectrumTM . Table 1 shows the
In response to declining mail
                                                 than those obtained in followup                relationship between the model
return rates (down from 87 percent
                                                 efforts. PMP activities were also              and the goals of the advertising
in 1970, the first census with a
                                                 intended to increase the level and             campaign. Y&R took as a proxy
large scale mailout/mailback oper/
                                                 rate of cooperation with the                   measure for this likelihood the
ation, to 74 percent in 1990 ), the
                                                 Nonresponse Followup (NRFU)                    number of civic activities an indi/
U. S. Census Bureau implemented a
                                                 phase of Census 2000. Ultimately,              vidual engaged in: most likely to
number of changes in design and
                                                 a goal of the PMP was to help                  respond are those participating in
operations for Census 2000. These
                                                 reduce the differential undercount             five or more civic activities,
included a greatly expanded out-
                                                 across population groups.                      undecided or passive are those
reach and promotion campaign,
                                                                                                with one to four activities, and
called the Partnership and                       The PMP included the following
                                                                                                least likely are those with no civic
Marketing Program (PMP), which                   components:
                                                                                                activities.
for the first time included paid
                                                 • A paid advertising campaign;
advertising and an enhanced                                                                     The Y&R campaign was further
Partnership Program, in an attempt               • The Partnership Program;                     segmented by race and ethnic
to increase public awareness of the                                                             group, in particular targeting tradi/
                                                 • Promotions and Special Events;               tionally harder to enumerate popu/
Census, to promote positive atti/
tudes about the Census, and to                   • A media relations program; and               lations: African-Americans,
increase or at least slow the                                                                   Hispanics, Asians, American
decline in mail return rates, partic/            • ;The Direct Mail Pieces                      Indians/Alaska Natives, and Native
ularly among segments of the pop/                   component.                                  Hawaiians and other Pacific
ulation traditionally more difficult                                                            Islanders. The primary slogan for
                                                 Each of these components was
to enumerate. Two primary con/                                                                  the campaign, selected to promote
                                                 new, expanded, or significantly
cerns about the mail return rate                                                                beliefs of personal and community
                                                 modified from 1990. The paid
made the expanded PMP appear                                                                    benefits and stimulate return of
                                                 advertising campaign, developed
worthwhile: (1) followup of nonre/                                                              the census form, was: "This is your
                                                 by Young and Rubicam (Y&R), was
sponse to the mail Census is very                                                               future. Don't leave it blank." There
                                                 based on a likelihood to respond



   Table 1.
   Y&R Likelihood SpectrumTM: Attitudes and Role of Advertising by Segment
                                    Least likely to respond        Undecided/Passive                 Most likely to respond

   Attitudes Towards                Fear                           Apathetic                         Familiar
   Census                           Distrust                       Not very familiar                 Intend to participate
                                    Completely unaware
   Role of Advertising              Lower resistance to pave way   Provide information               Reinforce positive behavior
                                    for community programs         Provide reason to complete        Instill sense of urgency
                                    Motivate                       Motivate                          Motivate
                                    Educate                        Educate                           Remind
                                    Remind                         Remind

      Adapted from Wolter et. al., 2002




U.S. Census Bureau                                Evaluations of the Census 2000 Partnership and Marketing Program 1
  Table 2.
  General Media Plans by Phase*
                                   Education phase                    Motivation phase                   Nonresponse followup phase

  Vehicles                         Print                              Print                              Radio
                                   Radio                              Radio                              Television
                                   Television                         Television
                                                                      Out of home**
  Time Period                      November 1 to January 30           February 28 to April 9             April 17 to June 5
  Activity Weeks                   Broadcast: 9 weeks                 Broadcast: 6 weeks                 Broadcast: 7 weeks
                                   Print: 2 months                    Print: 2 months
                                                                      Out of home: 2 months

     *For the Diverse America audience, those most likely to respond were not targeted during the education and nonresponse followup
  phases.
     **Out of home media included posters, outdoor advertisements, and transit advertisements Adapted from Wolter et. al., 2002.



were variations of this slogan for              about 140,000 partner organiza/                   take materials home and/or com/
different race and ethnic groups.               tions.                                            municate with their parents about
                                                                                                  the importance of participating in
The advertising campaign was                    The Census Bureau provided mate-
                                                                                                  the census (Macro International,
divided into three phases, as                   rials to partners to help publicize
                                                                                                  2002). Other major components
shown in Table 2. Each phase was                the census and to educate and
                                                                                                  of Promotions and Special Events
intended to have its own set of                 motivate partners' constituents,
                                                                                                  were "How America Knows What
messages, in keeping with the                   including posters and fact sheets,
                                                                                                  America Needs," which assisted
goals shown in Table 2. A primary               videos, articles for newsletters,                 local elected officials in encourag/
focus was to demonstrate the ben/               press releases, sample forms,                     ing their communities to partici/
efits to the individual and commu/              graphics, and promotional items.                  pate in the census, and the Census
nity of participation, and the cost             The Census Bureau made materials                  2000 Road Tour, in which twelve
of not participating (U.S. Census               available in a number of different                Census Bureau vehicles traveled
Bureau, Decennial Management                    languages; in addition, many part/                around the country during
Division, undated).                             ners developed in-language educa/                 February through April 2000, set/
                                                tional and informational materials                ting up exhibits in local "high traf/
The Partnership Program, greatly
                                                for their constituents. The Census                fic" areas. The Census Bureau also
expanded for Census 2000,
                                                Bureau also provided instructional                focused on media relations during
involved Census Bureau partner-
                                                manuals for partners in a variety of              the census period to complement
ship specialists working with state,
                                                settings to help them design pro-                 the other components of the PMP    .
local and tribal governments, com/
                                                grams to meet shared goals.                       The goal was to ensure that posi/
munity groups, nongovernmental
                                                Census Bureau staff also participat/              tive and educational stories about
organizations, local media, and pri/
                                                ed in partner activities. Partners, in            the census would appear in print
vate sector industries. The objec/
                                                turn, helped to publicize the cen/                and electronic media.
tives, as for the PMP overall, were
                                                sus through a variety of media,
(1) to increase the overall response                                                              The Direct Mail component incor/
                                                organized educational and motiva/
rate for Census 2000, (2) to reduce                                                               porated some significant changes
                                                tional community activities, and
the undercount of historically hard-                                                              from 1990 to 2000, based on
                                                provided assistance in some cen/
to-enumerate populations, and (3)                                                                 methodological research in the
                                                sus operations (Westat, 2001).
to communicate a consistent mes/                                                                  intervening years. For Census
sage to all Americans that re-                  The Census in Schools (CIS)                       2000, both the Mailout/Mailback
enforced the paid advertising                   Program was a significant compo/                  and Update/Leave universes
message and, in effect, closed the              nent of Promotions and Special                    received advance letters, telling
sale (U.S. Census Bureau, Decennial             Events, with the goal of teaching                 them that the Census 2000 ques/
Management Division, undated).                  students about the census. A vari/                tionnaire would be coming; 1990
The Partnership Program employed                ety of teaching materials were                    census operations did not include
some 690 partnership specialists                made available to teachers, with                  an advance letter. A 1992
around the country, working with                the intention that students would                 Implementation Test of the effects


2 Evaluations of the Census 2000 Partnership and Marketing Program                                                            U.S. Census Bureau
of sending advance letters, includ/    percent in low response areas          The PMPE survey Wave 1 was field/
ing a stamp on the return enve/        (Dillman et. al., 1993).               ed before the education phase of
lope, and mailing reminder post                                               the advertising campaign to meas/
                                       1.2 Partnership and
cards found that each of these                                                ure "baseline" awareness and atti/
                                       Marketing Program
additional mail contacts with                                                 tudes; Wave 2 was largely between
                                       evaluation activities
households resulted in higher over-                                           the education and motivation phas/
all response rates - 6.4 percent,      The Census Bureau commissioned         es, and the start of Wave 3 coincid/
2.6 percent, and 8.0 percent,          three major research evaluations of    ed with the NRFU phase of both
respectively - and that the effects    PMP activities:                        Census 2000 operations and the
were additive within the test sam/                                            advertising campaign. The CMS
                                       • ;The Partnership and Marketing
ples. The improvements for 1990                                               was conducted essentially between
                                          Program Evaluation (PMPE), a
low response rate areas were                                                  Waves 2 and 3 of the PMPE survey.
                                          series of three general popula/
somewhat smaller - 4.2 percent,
                                          tion surveys conducted and ana/     Partnership activities had a longer
1.6 percent, and 5.7 percent
                                          lyzed by the National Opinion       time frame than the advertising
(Clark et. al., 1993). Both the 1990
                                          Research Center (NORC), which       campaign. The planning and edu/
and 2000 censuses included
                                          was intended to evaluate the        cation phases, focusing on devel/
reminder post cards; the stamp
                                          effects of most of the PMP com/     oping the partnerships, stretched
was used in neither, although both
                                          ponents;                            from late 1996 through late 1999.
included prepaid return envelopes.                                            Motivation activities began in late
                                       • ;The Survey of Partners, a sam/
Another major change between                                                  1999 or early 2000, reached a
                                          ple survey of organizations
1990 and 2000 was prompted by a                                               peak between the mailout and
                                          enlisted as partners for Census
1993 test of the effects of various                                           Census Day (April 1), and contin/
                                          2000, focusing on the Partner-
kinds of motivational messages on                                             ued through the NRFU. The
                                          ship Program and conducted
response rates. Including the state/                                          Partnership Evaluation field period
                                          and analyzed by Westat; and
ment "Your Response is Required                                               was October 2000 through March
by Law" in a box on the outer          • ;An evaluation of the Census in      2001, considerably after PMP activ/
envelope increased response rates         Schools Program, based on a         ities had concluded.
by 9 to 11 percentage points over-        survey of primary and second/
                                                                               The Census in Schools Program
all as compared with approaches           ary school teachers, conducted
                                                                              conducted mailings to teachers
not using that phrase on the enve/        and analyzed by Macro
                                                                              and principals March through
lope, and 7 to 8 percentage points        International.
                                                                              September 1999. All schools
in 1990 low response areas             Another important evaluation           received at least one teaching kit,
(Dillman et. al., 1996). This state/   study was the Census Monitoring        and invitational packets were sent
ment was included on the Census        Survey (CMS), a weekly survey of       to elementary school teachers and
2000 outer envelope; it had not        the general population conducted       secondary school math and science
been used previously in a decenni/     just before and during the Census      teachers in historically hard-to-enu/
al census.                             2000 mailout/mailback by               merate (HTE) areas. During the
                                       InterSurvey (now known as              census period, take-home packets
Finally, the census questionnaire
                                       Knowledge Networks). The CMS           were mailed to all elementary
itself was redesigned to be more
                                       was privately commissioned             school teachers and middle school
"respondent-friendly," using gener/
                                       and funded.                            social studies teachers. The CIS
ally accepted design principles and
                                                                              survey was conducted in the
focus group testing. This redesign     Figure 1 in the Appendix shows
                                                                              Spring of 2000.
was made possible by the use of        the timing of Census 2000 opera/
the new technologies of optical        tions, paid advertising, and evalua/   The advertising campaign was test/
scanning and character recogni/        tion study activities. The advertis/   ed during the Census 2000 Dress
tion. A Simplified Questionnaire       ing campaign was timed so as to        Rehearsal in 1998, in Sacramento
Test in 1992 found that the            achieve the objectives outlined in     CA, Columbia SC and eleven sur/
respondent-friendly design             Table 1, and the PMPE and CMS          rounding counties, and Menominee
increased the return rate by 3.4       surveys were timed to assess the       WI. A pre-/post-test survey to
percent overall as compared with       effects of the advertising campaign    assess the effects of the campaign
the 1990 Short Form, and by 7.5        (as well as other PMP activities).     in Sacramento and South Carolina


U.S. Census Bureau                      Evaluations of the Census 2000 Partnership and Marketing Program 3
was conducted by Westat and ana/        tion studies, so interpretation of    the mailout/mailback operations.
lyzed by Roper-Starch.                  differences in results must be        Timing of Wave 3 of the PMPE is
                                        viewed cautiously. The last two       comparable to that of Wave 2 of
Besides these major evaluations by
                                        rows of Figure 1 show the approxi/    the 1990 OES.
contractors and other non-Census
                                        mate dates of the 1980 KAP and
groups, there were a variety of                                               1.4 Other related surveys
                                        1990 OES.
other evaluation activities conduct/
ed both by Census Bureau and con-       Wave 1 (the "pre" phase) of each of   Besides the surveys conducted to
tractor staff that looked at specific   these surveys occurred in late        evaluate marketing and outreach
components of the PMP Some of
                         .              January and early February. Wave 1    efforts, the Census Bureau commis/
these activities will be cited in the   of the PMPE was much earlier          sioned other research to examine
remainder of the report.                because the advertising campaign      issues of attitudes and behaviors
                                        in 2000 included an education         related to participation in the cen/
1.3 Previous evaluations of             phase that happened much earlier      sus. In 1999, NORC conducted a
marketing and outreach                  than such efforts in 1990 or 1980.    Knowledge, Attitudes, and
efforts                                 While the three Wave 1 surveys are    Perceptions Survey, known as
The PMPE and Census 2000 Dress          roughly comparable in their rela/     KAP-1. During the 1990s, there
Rehearsal surveys follow the gen/       tionship to planned outreach and      were several surveys measuring
eral form of two previous               publicity efforts, they may not be    public attitudes about privacy and
"pre/post" evaluation surveys: the      comparable for some measures          confidentiality around the census,
Knowledge, Attitudes, and               because of the differences in tim/    reviewed in a companion Topic
Practices (KAP) Survey in 1980, and     ing and because of external events    Report by Eleanor Singer. The
the 1990 Outreach Evaluation            such as news reports related to the   Survey of 1990 Census
Survey (OES). Each was designed to      census.                               Participation (SCP), conducted by
assess the effects of census mar/                                             NORC in June and July following
                                        Wave 2 of the 1980 KAP occurred
keting and outreach efforts on the                                            census operations, was designed
                                        after the outreach and publicity
American public's awareness,                                                  to assess the reasons for the
                                        campaign was well under way, but
knowledge and attitudes, and                                                  decline in response from 1980 to
                                        concluded just before the
behavior with regard to the                                                   1990. A roughly comparable sur/
                                        mailout/mailback operation. In
Decennial Census.                                                             vey was the 1980 Applied Behavior
                                        contrast, Wave 2 of the 1990 OES
This report will cite findings from     was conducted after the               Analysis Survey (ABAS), conducted
these two surveys by way of his/        mailout/mailback and during the       in April during census operations.
torical comparison, where compa/        NRFU. The 2000 PMPE's Wave 2          Limited results from these surveys
rable items were used. It should be     was conducted before the              will be cited to demonstrate histor/
noted that there are significant dif/   mailout/mailback, so is somewhat      ical trends or provide reinforce/
ferences in the timing of survey        comparable to Wave 2 of the 1980      ment or contrast to findings from
waves across the various evalua/            ,
                                        KAP although it is not as close to    the 2000 evaluation studies.




4 Evaluations of the Census 2000 Partnership and Marketing Program                                U.S. Census Bureau
2.	 Summary and Assessment of
    Evaluation Activities

By most outcome measures,             among Whites by at least 1.5 per/         Taken as a whole, the evaluation
Census 2000 was a success. The        centage points. The gap between           study data suggest certain conclu/
mail return rate, defined as the      long and short forms was smaller          sions with regard to the effective/
number of mail returns received       by the final return rate (9.6 per/                        ,
                                                                                ness of the PMP but are far from
before the cutoff date for the NRFU   centage points overall), but the          conclusive on any front. An
divided by the number of occupied     gap was reduced less among all            attempt to put all of the pieces
housing units in mailback areas,      non-White races, except Asians,           together in multivariate models did
was 74.1 percent, almost identical    than among Whites (Stackhouse             not show any significant main
to that of the 1990 Census (1990      and Brady, 2002).                         effect of PMP activities on actual
rates cited in Stackhouse and                                                   mail return behavior.
                                      PMP evaluation studies were
Brady, 2002 ) and ending the sharp
                                      intended to measure the effective/        In the remainder of this chapter,
decline between 1970 and 1990.
                                      ness of PMP components and activ/         we describe the Census 2000 eval/
The final mail response rate,
                                      ities - to try to attribute the contri/   uation studies and discuss their
defined as the percentage of the
                                      bution of each to the relative            results. Then, we discuss the
NRFU-eligible households returning
                                      success, as it turns out, of Census       strengths and weaknesses of the
forms, was 67 percent, up from 65
                                      2000. Without an experimental             study designs and implementa/
percent in 1990 and well above        design, it was not easy, and per-         tions. Chapter 3 summarizes the
the expected rate of 61 percent       haps not possible, to measure             discussions and make recommen/
(U.S. Census Bureau,                  these contributions directly, so the      dations for evaluating future
http://rates.census.gov). The NRFU    evaluation analysis strategy relied       Decennial Census Partnership and
effort finished almost two weeks      on a simple behavioral model, the         Marketing Programs.
ahead of schedule. Finally, in 1990   one underlying the Y&R advertising
the net undercount of the U.S. pop/   strategy: in order to participate,        2.1 Description of the
ulation was estimated at 1.6 per-     individuals must first be aware of        evaluation studies
cent overall, and up to 5 percent     Census 2000, they must have posi/         The PMPE survey was conducted in
for various racial and ethnic         tive attitudes about it, and they
                                                                                three waves, combining telephone
groups (U.S. Census Bureau,           must be motivated to fill out the
                                                                                and in-person interviews. The sur/
http://www.census.gov/dmd/www         Census 2000 form. Attitudes and
                                                                                vey was intended to capture cen/
/pdf/underus.pdf). For 2000, vari/    motivation, in turn, are a function
                                                                                sus awareness and other factors
ous estimates (U.S. Census Bureau,    of the information individuals have
                                                                                thought to be associated with
2003) indicate a net over count of    about the Decennial Census. The
                                                                                cooperation, as well as exposure
0.36 to 1.12 percent, with no         PMP attempted to convey the right
                                                                                to messages about the census
undercount of a racial or ethnic      message, to the right people, at
                                                                                from a wide variety of sources.
group larger than 2.5 percent.        the right time to convince them to
                                                                                Wave 1, with 3,002 completed
                                      respond to the census.
However, the mail return rate for                                               interviews from a random-digit-dial
the long form (63.0 percent) was      As we shall see, the evaluations          sample frame, occurred (largely)
considerably lower than that for      largely support the links between         before the education phase of the
the short form (76.4 percent).        awareness and positive attitudes          advertising campaign. Wave 2 com/
These rates compare with 70.4         or beliefs, between positive beliefs      prised 2,716 completed interviews
percent for the long form and 74.9    and intended or reported participa/       with a sample selected from the
percent for the short form in 1990.   tion, and between intended or             Decennial Master Address File
The gap of 13.4 percentage points     reported and actual participation.        (DMAF) and was conducted during
overall between short and long        Evaluation data indicate that the         the motivation phase and before
form return rates was greater         presence and strength of these            the mailout of Census 2000 forms.
among all non-White races than        links vary by population group.           Wave 3 was conducted after the


U.S. Census Bureau                     Evaluations of the Census 2000 Partnership and Marketing Program 5
mailout, during and after the            The Survey of Partners was con/         mail, with telephone followup, of a
NRFU; the sample was also select/        ducted well after PMP and Census        sample of some 4,000 primary and
ed from the DMAF, and 4,247 inter-       2000 activities were over, relying      secondary school teachers. The
views were completed. The design         on the recall of designated con/        sample was selected from two
of each wave included oversam/           tacts at partner organizations. It      sources: a commercial list of teach/
ples of hard-to-enumerate popula/        was intended to assess the success      ers and a list of teachers who
tion groups: Hispanics, non/             of the partnership efforts and iden/    ordered CIS Program materials. The
Hispanic African-Americans,              tify the benefits and contributions     sample was stratified to allow sep/
Asians, Native Hawaiians, and            of partners and the Census Bureau.      arate estimates for HTE areas and
American Indians. Survey instru/         The survey was conducted through        non-HTE areas. Some 1,046 sur/
ments included items on "media           the mail, with telephone followup,      veys were completed and accepted
                                         of a stratified random sample of        for analysis.
use; awareness of government
agencies and programs; awareness         partner organizations drawn from
                                                                                 2.2 Findings of the
of community agencies and pro/           the May 2000 Contact Profile and
                                                                                 evaluation studies
grams; recall of exposure to the         Usage Management System
                                         (CPUMS), including: national and        This section will summarize the
mass media; recall of exposure to
                                         local Federal government organiza/      findings of the evaluation studies
partnership-sponsored activities;
                                         tions; media organizations, sub-        by topic, following the behavioral
recall about sources of informa/
                                         classified by race/ethnicity of their   model of participation described
tion; knowledge and attitudes
                                         target audiences; national and all      earlier. We will begin by describing
about the Decennial Census; aided
                                         other for-profit private businesses;    findings with regard to the reach
recall of specific advertising; aided
                                         national non-government organiza/       of the PMP activities that were
recall of specific partnership activi/
                                         tions (NGOs); local NGOs, sub-clas/     evaluated, then discuss awareness
ties; Census 2000 mailback form
                                         sified by race/ethnicity of their       of Census 2000 and exposure to
receipt, handling, and mailback                                                  PMP activities, then describe find/
                                         constituencies; and state, local and
behavior; and demographic infor/                                                 ings with regard to attitudes
                                         tribal government organizations. A
mation" (Wolter et. al., 2002).                                                  towards the Census and govern/
                                         total of 9,057 interviews was com/
The 2000 CMS was conducted               pleted. The survey instrument           ment in general, and finally cover
weekly during the motivation             included items on: materials the        findings with regard to
                                         partners received and used, includ/     intended/reported and actual par/
phase of the advertising campaign
                                         ing how helpful the materials were      ticipation. For each topic, we will
and through the mailout/mailback
                                         and the timeliness of their receipt;    describe results for the general
period. It was intended to provide
                                         the relative importance of various      population and also for the target/
immediate feedback on marketing
                                         partnership goals; the kinds of         ed race and ethnic groups. Finally,
and mailout activities. The sample
                                         activities partners engaged in; the     we will discuss findings with
was drawn from the InterSurvey
                                         kinds of assistance Census staff        regard to the relative effects of dif/
(now known as Knowledge
                                                                                 ferent PMP components and activi/
Networks) panel, and was conduct/        provided; costs associated with
                                                                                 ties across these topics.
ed through interactive Web TV. The       Census 2000-related activities; and
five weekly surveys had sample           characteristics of the partner          2.2.1 The reach of marketing
sizes (completed interviews) of          organization.                           activities
993; 973; 719; 1,004; and 948            The CIS survey was conducted dur/       Several evaluation and operational
respondents, respectively. Survey        ing the Spring of 2000, at the end      reports describe aspects of the
content included exposure to and         of the school year in which teach/      reach of PMP activities. The
reaction to advertising, census atti/    ers would have used materials they      Partnership Program, the Census in
tudes and awareness, perceptions         received as part of the program.        Schools Program, the paid advertis/
of, experience with, and action          The survey focused on whether           ing campaign, the media relations
taken with respect to the Census         and how teachers learned about          campaign, Promotions and Special
2000 mailback form, reasons for          the CIS Program, obtained materi/       Events and the Direct Mail Pieces
nonresponse, response intentions,        als, and used those materials in        component all helped to spread
NRFU experience, and recognition         class, and asked teachers to assess     the word about Census 2000.
of ads played back during the            the program and its materials. The      Several of these components par/
interview (Nie and Junn, 2000).          survey was conducted through the        ticularly targeted historically hard-


6 Evaluations of the Census 2000 Partnership and Marketing Program                                    U.S. Census Bureau
to-enumerate population groups                  sible to estimate with any preci/      school social studies and math
and/or geographic areas.                        sion how many individuals may          teachers (30 percent)3.
                                                have been reached by these part/
The Census Bureau enlisted about                                                       An important goal of the CIS
                                                ners' activities, but the number of
140,000 organizations in its                                                           Program was to reach families
                                                active partners is substantial.
Partnership Program1 . Most of                                                         through their children, increasing
these were local in scope, with                 About 16 percent of partner            awareness and knowledge of the
almost 40 percent operating at a                organizations spent non-Census         census, and ultimately increasing
city level and another 26 percent               funds, either their own or from        participation. Take-Home Packets
at a county level. The partners                 another source, to promote Census      were sent to all elementary school
were about equally divided                      2000. While there was consider-        teachers and to middle school
between governments and non-                    able nonresponse in the Survey of      social studies teachers. About 19
governmental organizations.                     Partners on questions asking for       percent of elementary school
Partnering governments included                 the amount of funds expended,          teachers and 8 percent of second/
local, state, and tribal entities.                                                     ary school social studies and math
                                                respondents reported some $168
Partnering non-governmental                                                            teachers sent materials home with
                                                million spent, which is probably a
organizations included community-                                                      their students. Teachers in HTE
                                                lower bound given the item nonre/
based organizations (29 percent),                                                      areas were about as likely (14 per-
                                                sponse. About one-third of partner
businesses (22 percent), religious                                                     cent versus 12 percent) to send
                                                organizations reported making in-
organizations (16 percent), and                                                        materials home as those in
                                                kind contributions, such as staff
educational organizations (15 per-                                                     other areas3.
                                                time, office space, and equipment
cent). More than half of partners               usage. The dollar value of these       A Census Bureau assessment of
did not target any specific race or             contributions was estimated at         the "How America Knows What
ethnic group, while 23 percent tar/             about $374 million (Westat, 2001).     America Needs" campaign (Sha and
geted Hispanics, 22 percent                                                            Collins, forthcoming) reported that
African-Americans, 11 percent                   According to the CIS Evaluation
                                                                                       local and national media coverage
Asians, 7 percent American                      report, some 56 percent of teach/      of Census 2000 more than doubled
Indians/Alaska Natives, and 4 per-              ers nationally had heard of the CIS    that of the 1990 census (in terms
cent Native Hawaiians/Pacific                   Program. Among elementary              of sheer number of news stories).
Islanders (Westat, 2001).                       school teachers in HTE areas, a tar/   However, according to an inde/
                                                geted group, 68 percent had heard      pendent media analysis commis/
At least 70 percent of partner
                                                of the CIS Program, compared with      sioned by the Census Bureau
organizations reported conducting
                                                62 percent in other areas. Among       (Douglas Gould and Co, 2001),
one or more kinds of activities to
                                                secondary school social studies        print media coverage of Census
publicize Census 2000, educate
                                                and math teachers, 44 percent in       2000 across nine major outlets
and motivate constituents, or sup-
                                                HTE areas had heard of the CIS         was down from the level of the
port Census 2000 operations.
                                                Program, compared with 59 per-         1990 Census. The Gould report
Almost one-quarter reported con/
                                                cent in other areas (Macro             speculated that the decline
ducting more than ten different
                                                International, 2002). Thus, it         occurred because Census 2000
kinds of activities. On average,
                                                appears that the mailing of invita/    was less controversial than the
each state2 had more than 1,100
                                                tional packets directly to teachers    1990 census and because of inter/
active and about 400 very active
                                                in HTE areas increased awareness       est in the Presidential campaign.
partner organizations. It is not pos-
                                                of the CIS Program among elemen/       Important exceptions to the "less
                                                tary school teachers, but not          controversial" observation were
     1
       The Census Bureau counted some
140,000 partners enlisted throughout the        among secondary school social          partisan wrangling over adjust/
Partnership Program. The Survey of Partners     studies and math teachers. About       ment and comments by some
estimated that almost 86,000 organizations
were eligible for the survey after accounting   37 percent of teachers in HTE          politicians about the intrusiveness
for duplicates in the master list and organi/   areas and 34 percent of those in       of the long form. The discrepancy
zations reporting that they were not part/
ners. Note that the survey occurred some        other areas reported actually          between the Census findings and
months after partner activities had ended, so
                                                receiving materials. Elementary
the survey estimate may be affected by
changes in some partner organizations in        school teachers were much more              3
                                                                                              These proportions were not presented
the interim.                                                                           in the CIS Evaluation report; the authors cal/
                                                likely to report receiving materials
     2
       Included were the 50 states, the                                                culated them from tables presented in the
District of Columbia, and Puerto Rico.          (46 percent) than were secondary       report's Appendix.



U.S. Census Bureau                               Evaluations of the Census 2000 Partnership and Marketing Program 7
those of the Gould report may indi/    1980, the 2000 PMPE found some-        2.2.3 Attitudes towards the Census
cate greatly increased coverage of     what lower levels among non/
                                                                              Overall, it appears that positive
Census 2000 at the local level. The    Hispanic African-Americans and
                                                                              attitudes towards the census
Gould report also noted that opin/     Whites, but higher levels among
                                                                              increased significantly during the
ion pieces were largely positive,      Hispanics (Wolter et. al., 2002,
                                                                              PMP campaign, but that this
and also that outreach efforts were    drawing data from Bates and
                                                                              increase was tempered by receipt
the subject of 19 percent of the       Whitford, 1991, and Moore, 1982) .
                                                                              of the census forms, particularly
sample articles overall and 36 per-    The latter difference may be due to
                                                                              among those who received the
cent of pre-Census Day articles, up    early education efforts aimed at
                                                                              long form. Among race and ethnic
from only 5 percent in 1990.           Hispanics in 2000. Wave 1 aware/
                                                                              groups other than non-Hispanic
                                       ness was much higher in 1990
2.2.2 Awareness and exposure                                                  Whites, those who had recently
                                       than either 1980 or 2000, which
                                                                              heard about the census were more
It is clear that awareness of Census   may in part be attributable to the
                                                                              likely to hold positive beliefs about
2000 rose dramatically throughout      1990 Census Awareness and
                                                                              it than those who hadn't. The fol/
the marketing campaign, at least       Products Program, which started
                                                                              lowing paragraphs provide details
well into the NRFU phase. During       earlier than similar efforts in 1980
                                                                              of these findings.
Wave 1 of the PMPE survey, about       (Fay et. al., 1991) and to the tim/
65 percent of respondents report/      ing of Wave 1 in 1990 as compared      The PMPE survey included eight
ed having heard nothing about          with 2000. It may, of course, also     items asking about respondents'
Census 2000 and fewer than 10          be related to other factors, such as   beliefs about the census. Using
percent reported having heard a        a higher level of news coverage of     factor analysis to construct and
great deal. In Wave 2, about 25        census issues.                         analyze a composite of these
percent reported hearing nothing                                              items, Wolter et. al. (2002) showed
                                       The 2000 Dress Rehearsal survey
and about the same percentage                                                 that there was some significant
                                       reached a level of "heard recently"
reported hearing a great deal. By                                             movement of beliefs in the positive
                                       similar to that of the PMPE (more
Wave 3, only about 15 percent of                                              direction between Wave 1 and
                                       than 80 percent), but the pre-cam/
respondents reported having heard                                             Wave 2, but not between Wave 2
                                       paign 2000 Dress Rehearsal survey
nothing, and almost half reported                                             and Wave 3. Every race and ethnic
                                       awareness levels were lower (28
having heard a great deal, almost a                                           group examined except American
                                       and 29 percent in Sacramento and
complete reversal from Wave 1.                                                Indians showed some increase in
                                       South Carolina, respectively) than
                                                                              positive beliefs over the three
Awareness increased significantly      in Wave 1 of the PMPE survey (35
                                                                              waves.4
between each wave for each of the      percent) (Roper-Starch, 1999). It
oversampled populations, except        seems reasonable that ambient          The PMPE survey findings are simi/
that awareness levels for              information about the census           lar to those of the 1980 KAP sur/
Hispanics may have leveled off         would be lower two years before        vey and 1990 OES. In 1980 (com/
between Wave 2 and Wave 3.             the census.                            parable to the PMPE Wave 1 to
(Wolter et. al., 2002)                                                        Wave 2 comparison), favorable
                                       The CMS, conducted essentially
                                                                              responses to three attitude items
Despite these dramatic increases in    between Waves 2 and 3 of the
                                                                              included in both 1980 and 1990
awareness measured during              PMPE survey, showed that the pro-
                                                                              increased between Wave 1 and
Census 2000, peak awareness (per/      portion of respondents reporting
                                                                              Wave 2. In 1990, there was mixed
centage having "heard recently")       they had seen or heard "a lot"
                                                                              movement between Wave 1 and
was lower than during the 1990         about Census 2000 from TV com/
                                                                              Wave 2 across six items, with
Census among Hispanics and non/        mercials rose from 30 percent dur/
                                                                              some items increasing and others
Hispanic Whites, although it was       ing the first week of March to 70
                                                                              decreasing. Because recent aware/
higher among non-Hispanic              percent 3 weeks later. Smaller per/
                                                                              ness of the census was relatively
African-Americans. Awareness as        centages, but similar proportionate
                                                                              high in Wave 1 of the OES, it may
measured in Wave 2 of the PMPE         increases, were reported for radio
                                                                              be that the appropriate comparison
and the 1980 KAP were fairly com/      and newspaper advertising (Nie
parable, although again higher in      and Junn, 2000). Clearly, the
2000 among non-Hispanic African-       intense motivation phase campaign           4
                                                                                     The report includes no item-by-item
                                                                              analysis of movement over time. In retro/
Americans. Comparing the Wave 1        and the mailout had a substantial      spect, it would have been useful to have this
results from 2000 with those from      effect on awareness.                   information available.



8 Evaluations of the Census 2000 Partnership and Marketing Program                                     U.S. Census Bureau
   Table 3.
   Significant Differences (Chi-square) in Census Beliefs by Recent Awareness of
   The Census
                                                                            Total             African-                       American       Native
                                Belief	
                                                                            Pop.    Hispanic American    White       Asian     Indian     Hawaiian

   Filling out the census will let the government know what
   my community needs                                                   *                       *        *           *                        *

   The census counts citizens and non-citizens alike                                  *                                                       *
   It is important for as many people as possible to participate
   in the census                                                                                *

   My answers to the census could be used against me                                  *         *                    *                        *
   Answering and sending back the census matters for my
   family and my community                                                            *         *                                *

   The Census Bureau promise of confidentiality can be
   trusted                                                                                      *
   I just dont see that it matters much if I personally fill out
   the Census or not                                                                            *                    *                        *

   Sending back your census form could personally benefit or
   harm you in any way                                                                                               *           *

      Based on findings reported by Wolter et. al., 2002
      *p <.10


is with Wave 2 to Wave 3 of the                       from data presented by Nie and                     suggests two interpretations: 1)
PMPE, where there was no net                          Junn - almost half of those receiv/                that this negative belief was not
movement.                                             ing the long form said that the                    addressed or was not addressed
                                                      controversy made them feel less
The CMS included five belief items.                                                                      convincingly in the marketing cam/
                                                      like returning their form5. It seems
The level of agreement with three                                                                        paign; 2) that receipt of the census
                                                      very likely, then, from the findings
positive items stayed reasonably                                                                         mailing alone may not have been
                                                      of both Martin (2000) and Nie and
constant over five weeks of inter-                                                                       the cause of the increase in agree/
                                                      Junn (2000) that the long form
viewing, while agreement with two
                                                      controversy had a negative effect                  ment with the statement found in
negative items - the census is an
                                                      on census returns, at least among                  the CMS6. Non-Hispanic African-
invasion of privacy, and my
                                                      those receiving the long form,                     Americans were significantly more
answers could be used against me
                                                      although neither source links hear/
- increased (that is, beliefs moved                                                                      likely than non-Hispanic Whites to
                                                      ing about the controversy directly
in a negative direction) over that                                                                       agree with each of the three nega/
                                                      with behavior.
period. Martin (2000) modeled the                                                                        tive beliefs after the marketing
association between hearing about                     The Census 2000 Dress Rehearsal                    campaign, in both sites. In
the controversy and negative                          study included nine belief items,
                                                                                                         Sacramento, Hispanics were more
beliefs, and concluded that the                       with some overlap with both the
                                                                                                         likely to agree with two of the
controversy did increase negative                     PMPE and CMS. Eight of the 9
beliefs, as did receipt of the long                   items showed some increase in                      three negative beliefs than non/
form, and that the effects of these                   positive views from before the                     Hispanic Whites7.
two factors were largely                              marketing campaign to after, most
independent.                                          significantly, in both Sacramento                        6
                                                                                                                 The level of agreement is also interest/
                                                                                                         ing. In the Census 2000 Dress Rehearsal sur/
                                                      and South Carolina. Notably, there                 vey and the 1999 KAP1, the levels of agree/
About 44 percent of respondents
                                                      was no change in either site in the                ment were at 20 percent or higher. Early in
in week 5 of the CMS had heard of                                                                        the CMS they were at 10 percent and went
                                                      proportion agreeing strongly or                    to 20 percent, the "historical norm."
the controversy, and 9 percent of
                                                      somewhat that the census is an                     Published findings of the 1990 OES show
those said that it made them feel                                                                        only the percentage giving positive respons/
                                                      invasion of privacy. This finding                  es, which declined between Wave 1 and
less like returning their Census                                                                         Wave 2 (after receipt of the mailout pack-
2000 form. Virtually all of this 9                                                                       age).
                                                            5
                                                              In the 1990 OES, almost twice as many            7
                                                                                                                 Neither the PMPE nor the CMS report
percent were people who had                            long form as short form recipients thought        includes comparable information by
received the long form, it appears                     that the census was an invasion of privacy.       race/ethnicity.



U.S. Census Bureau                                      Evaluations of the Census 2000 Partnership and Marketing Program 9
For each of the eight PMPE survey      all groups between Wave 1 and                    Analysis of the 1990 OES and SCP
items, among some (but not all)        Wave 2,8 suggesting beneficial                   found that knowing that the cen/
race and ethnic groups, people         effects from the PMP.                            sus was required by law was the
who had recently heard about the                                                        only knowledge item tested that
                                       Intended, self-reported, and
Census were significantly more                                                          was a significantly better predictor
                                       actual cooperation. Wave 3 of
likely to hold positive views than                                                      of return than others (Fay et. al.,
                                       the PMPE survey asked whether the
those who hadn't recently heard.                                                        1991a).
                                       respondent's household had
Table 3 summarizes the significant
                                       returned the census forms, and for               Table 4 summarizes the relation-
differences in percentage of
                                       both Waves 2 and 3 information on                ship between beliefs about the
respondents holding positive
                                       actual return of the census form                 census and self-reported participa/
beliefs about the census by
                                       was obtained. The correlation                    tion in Wave 3 of the PMPE survey.
whether they reported recent
                                       between intended and actual9                     For most race and ethnic groups,
awareness. While the recently
                                       cooperation was low, ranging from                there is a similar pattern of associ/
aware had more positive beliefs                                                         ation between beliefs and self-
                                       0.05 (all correlation values are
among almost all groups for                                                             reported behavior. For Hispanics,
                                       weighted) for American Indians to
almost all items, the patterns of                                                       however, the only belief signifi/
                                       0.27 for Hispanics. Reported coop/
statistical significance vary inter/                                                    cantly associated with self-reported
                                       eration in Wave 3 was more highly
estingly by race and ethnic group.                                                      participation is that the census
                                       correlated with actual cooperation,
Non-Hispanic African-Americans                                                          counts both citizens and non-citi/
                                       ranging from 0.30 for African-
showed the largest number of sta/                                                       zens. For American Indians, the
                                       Americans to 0.42 for American
tistical differences, and non/                                                          relationship between the belief
                                       Indians, but the correlations are
Hispanic Whites the fewest.                                                             that answers to the census "won't
                                       still fairly low. About three-quarters
                                       of Wave 2 respondents who said                   be used against me" and self-
Three of the belief items were
                                       they "definitely will" return the                reported cooperation is highly sig/
included in the 1990 OES, and two
                                       form actually did so, and about the              nificant, while it is not significant
of these were in the 1980 KAP sur/
                                       same proportion of Wave 3 respon/                among any other group.
vey. The two common items
(whether the census promise of         dents who said they had returned
                                                                                        Note that the three items related to
confidentiality can be trusted and     the form had their claim verified.
                                                                                        privacy and confidentiality in Table
that the census data can not be        The relationship between                         4 ("My answers to the census could
used against you) showed increas/      beliefs and cooperation. The                     be used against me," "The Census
es over time within all three sur/     CMS asked respondents whether                    Bureau promise of confidentiality
veys; the largest increases (12 and    they felt particular messages were               can be trusted," and "Sending back
8 percentage points) were between      "persuasive reasons to fill out the              your census for could personally
Wave 1 and Wave 2 in 2000.             census form." Allocation of federal              harm you in any way [sic]") show
                                       dollars to communities was viewed                no significant association with par/
2.2.4 Intended cooperation
                                       as persuasive by about three-quar/               ticipation, with one exception-
The PMPE survey allows compari/        ters of respondents, peaking the                 American Indians were less likely
son of the change in intended          week of March 23. Allocation of                  to report returning the census form
cooperation from before the start      Congressional seats as a persua/                 if they believed the census could
of the paid advertising campaign       sive reason also peaked the same                 be used against them. Fay et. al.
(Wave 1) to after the campaign's       week, at 60 percent of respon/                   (1991a) examined the relationship
education phase (Wave 2). Among        dents. Identification of mandatory               between an index of three priva/
all groups except Hispanics (who       participation as a persuasive rea/               cy/confidentiality items and return
started at a high level and stayed     son doubled from March 3 to                      of the census form using the 1990
there) and American Indians (who       March 23 (after the mailout, which                                  .
                                                                                        OES and 1980 KAP In 1990, those
started at a low level and rose        had that message on the enve/
slightly), the level of intended       lope), reaching a peak of 46 per-                     10
                                                                                                In the Dress Rehearsal survey, knowl/
cooperation rose significantly         cent in the week of April 710.                   edge that the census is mandatory also dou/
                                                                                        bled between the pre- and post-campaign
between Wave 1 and Wave 2. The
                                                                                        survey rounds, and the levels of knowledge
correlation between awareness of            8
                                              The increase was statistically signifi/   at each period were very close to the early
                                       cant for all groups except American Indians.     and late levels in the CMS. The 1999 KAP
Census 2000 communications and              9
                                              That is, returning a census form by       also found that about 22 percent of people
intended cooperation rose among        April 18.                                        knew the census was mandatory.



10 Evaluations of the Census 2000 Partnership and Marketing Program                                              U.S. Census Bureau
   Table 4.
   Significant Differences (Chi-square) in Self-Reported Census Participation by Whether
   Positive Beliefs Reported
                                                                                          African-                     American     Native
                                Belief
                                                                    Total Pop   Hispanic American     White    Asian     Indian   Hawaiian

   Filling out the census will let the government know what
   my community needs                                                  *                    *         *        *          *             *

   The census counts citizens and non-citizens alike                   *          *                   *        *
   It is important for as many people as possible to participate
   in the census                                                       *                    *         *        *          *         *

   My answers to the census could be used against me                                                                      *
   Answering and sending back the census matters for my
   family and my community                                             *                              *        *          *             *

   The Census Bureau promise of confidentiality can be
   trusted
   I just don't see that it matters much if I personally fill out
   the Census or not                                                   *                    *         *        *          *             *

   Sending back your census form could personally benefit or
   harm you in any way

      Based on findings reported by Wolter et. al., 2002
      *p <.10


with positive attitudes on all three                  2.2.5 The overall relationship                 and the innumerable influences on
items were significantly more likely                  between census communications                  individuals' mood and behavior
to return the census form (by both                    and actual cooperation                         that arise every day), and measure/
self-report and matching actual                                                                      ment of the explanatory factors
                                                      Wolter et. al. (2002) fitted a series
behavior, while there was no differ/                                                                 may be subject to various types of
                                                      of multivariate models in an
ence in 1980 (only self report was                                                                   error (e.g., comprehension of sur/
                                                      attempt to show the relationship
examined). Similar findings were                                                                     vey questions, recall of messages
                                                      between PMP activities and actual
reported by Kulka et. al. (1991).                                                                    or behavior, identification of the
                                                      mail return behavior. They conclud/
Martin (2000), in an analysis of                                                                     correct household respondent).
                                                      ed, "the . . . data are consistent
2000 CMS and 1990 OES data,                                                                          Thus, the multivariate analysis
                                                      with the hypothesis that mass
found "several indications that                                                                      should not necessarily be taken as
                                                      media and community-based com/
Census 2000 engendered more                                                                          evidence that there is not a rela/
                                                      munications had no effect on the
sensitivity and a more diverse pri/                                                                  tionship between mail return
                                                      odds of a mail return for the Asian,
vacy reaction than the previous                                                                      behavior and PMP activities, just as
                                                      American Indian, and Native
census." Martin noted, as had Fay                                                                    an indication that this particular
                                                      Hawaiian populations. . . . The
et. al. (1991b), that privacy and                                                                    method of exploring the relation-
                                                      data support a conclusion that cen/
confidentiality are multi-faceted                                                                    ship (assuming it exists) is not
                                                      sus communications were less
concepts for the public, with com/                                                                   finely tuned enough to detect it.
                                                      effective for the other-language
plex inter-relationships that may                     population than for the English                Bentley (2003) conducted an auxil/
change over time and in response                      population." These conclusions                 iary evaluation of the relationship
to particular stimuli (such as                        could certainly be termed disap/               between PMP activities and Census
receipt of the census form or hear/                   pointing. However, the incremental             2000 return rates. Using county-
ing about the long form controver/                    increase in return rates overall due           level return rates, a variety of indi/
sy in 2000). Thus, the relationship                   to the PMP is likely to be relatively          cators of PMP activities by geo/
between these concerns and return                     small (on the order of 5 percentage            graphic area, and other control and
of the census form in 2000 may                        points or less), there are many con-           explanatory variables, Bentley con/
warrant further exploration before                    founding factors that were unable              structed a series of statistical mod/
simply accepting the findings for                     to be included in the models (e.g.,            els in the same spirit as those
these three items as summarized                       news stories on the census, con/               developed by Wolter et. al.
in Table 4.                                           versations with friends and family,            Essentially replicating the Wolter


U.S. Census Bureau                                    Evaluations of the Census 2000 Partnership and Marketing Program 11
et. al. results, he detected no rela/    for the mail pieces is presented by    "a lot" about the census in TV com/
tionship between the intensity or        Dillman et. al. (1996). The 1994       mercials as in radio or newspaper
nature of PMP activities and coun/       field test combined three motiva/      advertisements. Reported exposure
ty-level return rates. Bentley cites a   tional appeals: a statement that       more than doubled for each medi/
number of limitations of his analy/      response was mandatory, state/         um between the weeks of March 3
sis, notably the lack of an experi/      ments about the benefits of partici/   and March 31, then leveled off for
mental design in the PMP Again,
                           .             pation, and varying levels of assur/   the final week.
the fact that no relationship was        ance of the confidentiality of
                                                                                The PMPE did not distinguish
detected does not demonstrate            responses. The latter two kinds of
                                                                                between advertising and news
that a relationship does not exist.      appeal could be thought of as ele/
                                                                                when asking about mass media,
                                         ments in the general category of
Neither of these modeling efforts                                               but the CMS did, although it is not
                                         "census climate" features, although
could include consideration of the                                              clear whether respondents would
                                         weak ones. The effects of the
effects of the advance letter and                                               consistently be able to make this
                                         mandatory statement dwarfed any
mandatory notice on the question/                                               distinction. Exposure to news sto/
                                         effects the other appeals had on
naire outer envelope, since these                                               ries about the census on television,
                                         return rates.
features were included for all                                                  on the radio, and in newspapers
households. Tortora et. al. (1993),      2.2.6 Disaggregating the effects of    was somewhat lower (in the case
commenting on the likely effects         Partnership and Marketing              of television almost half) than to
on return rates in Census 2000 of        Program component activities           advertising in the same sources. As
the various mail piece innovations,                                             with advertising, news exposure
                                         If a direct link between the PMP       increased steadily for the first four
noted:
                                         and mail return behavior cannot be     weeks, then leveled off for the
  In the past, the Census Bureau         demonstrated statistically from the    final week.
  has obtained somewhat lower            available evaluation data, one
  response rates in noncensus            would certainly not expect to find     In the PMPE, community-based
  years than in census years . . .       evidence of the direct effects of      sources that reached about the
  The usual explanation for this         individual PMP components. The         same levels of awareness as some
  difference is 'census climate', a      evaluation studies do provide          of the mass media were informal
  succinct explanation of the com­       some indirect insights into the rel/   conversations, census job
  bination of media attention,           ative success of the components,       announcements, signs or posters
  advertising, and cultural              however.                               inside buildings, and articles.
  sense of participation that                                                   Awareness of each of these
                                         The PMPE measured awareness of         sources increased between Wave 1
  seems to build during each cen­
                                         census information from a variety      and Wave 2, but only informal con/
  sus year . . . We do not know
                                         of mass media and community-           versations and signs or posters
  whether the existence of a 'cen­
                                         based sources in each of its three     increased significantly between
  sus climate' will substitute for
                                         waves. Awareness of mass media         Wave 2 and Wave 3. Very similar
  the effects of these elements or
                                         (television, magazine, radio, news-    patterns of awareness were found
  add to the response likely to be
                                         paper, and billboard) messages         among all race and ethnic groups.
  obtained in a census year.
                                         increased significantly each wave      None of these sources was includ/
In fact, this is the question that       among most race and ethnic             ed in the CMS. Speeches by gov/
remains unanswerable. The model/         groups, with the exception of mag/     ernment or local leaders had the
ing efforts of Wolter et. al. and        azine awareness, which did not         highest exposure rate among the
Bentley tried to correlate return        increase significantly between         community-based sources included
rates and different levels of inten/     Wave 2 and Wave 3. Television had      in the CMS, and their exposure
sity of "census climate" as stoked       the highest mean awareness             increased steadily over the survey
by the PMP, and could not detect a       among all groups, with radio and       period. Religious groups, local
relationship. The mail piece inno/       newspapers next. The Census            community or government organi/
vations may be the key elements          2000 Dress Rehearsal study found       zations, "things children brought
that stemmed the ebbing tide of          the same ranking among mass            from school," and school-related
return rates, but there is no direct     media in respondent awareness. In      activities had lower and fairly con/
statistical evidence to support that     the CMS, more than twice as many       stant levels of exposure during the
contention. One small argument           people reported seeing or hearing      survey period. While awareness of


12 Evaluations of the Census 2000 Partnership and Marketing Program                                 U.S. Census Bureau
messages from these sources was           Americans, non-Hispanic Whites,         2.3 Discussion of
also at lower levels in the PMPE          and Native Hawaiians. For commu/        individual evaluation
survey, awareness increased signif/       nity-based communications, the          studies' strengths and
icantly between Wave 1 and Wave           correlations increased for all of       limitations
2 for each of them, less so               the same groups except Hispanics.
between Wave 2 and Wave 3. The                                                    The studies commissioned to eval/
                                          The correlations between aware/         uate the Partnership and Marketing
remaining sources included in the
                                          ness of both kinds of communica/        Program for Census 2000 repre/
PMPE survey, conference exhibit
                                          tion and intended participation         sent the most comprehensive
booths, the Internet, paycheck or
utility bill inserts, and participation   rose for Asians, but not significant/   effort of this kind to date. The
on complete-count committees,             ly, and did not rise for American       three major studies complement
had the lowest levels of awareness        Indians. The CMS asked whether          each other well. The PMPE provides
throughout and exhibited some             the advertising (or news) exposure      the largest amount of information
increases across waves.                   "make me feel more like taking          related to the final outcomes of
                                                                                  interest - increasing or at least
                                          part" in the census. For each, "yes"
In Wave 3 of the PMPE survey,                                                     slowing the decrease in mail return
                                          responses rose sharply between
about 12 percent of respondents                                                   rates, and reducing the differential
said they had heard "a little" about      March 3 and March 10, then
                                                                                  undercount of various race and
Census 2000 from materials their          dropped off slowly. Overall, about
                                                                                  ethnic groups. The privately-fund/
children brought home from                half of respondents said "yes" for      ed CMS provides a "pulse-taking"
school, and about 3 percent report/       advertising and for news reports.       look at what happened during the
ed hearing "a lot." While it is                                                   peak period of the PMP week by
                                          The Survey of Partners identified
impossible to compare estimates                                                   week, which the PMPE was not
of the reach of the CIS Program           what activities partnering organiza/
                                                                                  designed to do. The Survey of
between the PMPE and the CIS              tions conducted, but most of these
                                                                                  Partners supports more of a
Evaluation with any precision,            activities cannot be directly related
                                                                                  process evaluation for one compo/
these numbers are not inconsistent        to items in the PMPE survey or          nent of the PMP than an outcome
with the CIS Evaluation figures           CMS. Vehicles for communication         evaluation. We will discuss each
cited earlier. In the Census 2000         with target audiences included          study in turn. The following sec/
Dress Rehearsal survey, non/              printing and distributing materials     tion discusses evaluation issues
Hispanic African-Americans (in            (37 percent of partners), using         that span the studies.
South Carolina) and Asians/Pacific
                                          print media (34 percent), holding
Islanders (in Sacramento) reported                                                2.3.1 Survey of Partners
                                          public and in-house meetings (33
more exposure to school-based
                                          percent), distributing census pro-      Of the three Census-commissioned
sources (their own or their chil/
                                          motional items at meetings and          evaluations, this study was most
dren's schools) than did non/
                                          events (30 percent), sponsoring         process-oriented. It provides a
Hispanic Whites. In the PMPE sur/
                                                                                  good quantitative assessment of
vey, all other race and ethnic            local media coverage (19 percent),
                                                                                  the number and level of involve/
groups reported more exposure to          and including messages in utility
                                                                                  ment of active partners. The
school-based sources than did non/        bills, phone cards, etc. (13 percent)
                                                                                  Survey of Partners did not attempt
Hispanic Whites, although the sta/        (Westat, 2001).
                                                                                  to measure person-level outcomes
tistical significance of the differ/
                                          Finally, Wolter et. al. examined the    in terms of awareness, beliefs, or
ence cannot be determined from
                                          association between awareness of        participation in Census 2000, even
the published report.
                                                                                  though the ultimate goals of the
                                          various information sources and
Wolter et. al. also compared the                                                  Partnership Program were very
                                          actual return behavior across the
correlation between intended par/                                                 much linked with those of the
                                          1980 KAP survey, 1990 OES, and
ticipation and awareness of mass                                                  advertising campaign and other
                                          the PMPE survey's Wave 2 and
media and community-based com/                                                    PMP activities. In fact, one of the
munications in Wave 1 and Wave 2.         Wave 3. They found no consistent        objectives of the PMP model was
For mass media, the correlations          pattern of association across the       to reduce resistance to the census
increased significantly between           three studies and four data points,     among the hardest to enumerate
waves for the general population,         although there was variation within     to "pave the way for community
Hispanics, non-Hispanic African-          each.                                   programs," which would likely


U.S. Census Bureau                        Evaluations of the Census 2000 Partnership and Marketing Program 13
have come via the Partnership         Limitations of the design include        • ;It would have been instructive
Program. The decision not to          issues of sampling error, nonre/            for the analysis of the relation-
attempt to measure such outcomes      sponse, frame undercoverage, and            ship between beliefs about the
retrospectively was probably a        response error. Appropriate adjust/         census and reported participa/
good one. However, it might be        ments and caveats were used in              tion to have been extended to
possible in future evaluation         the analysis and reporting, and are         (or replaced by) the same
efforts to measure the "reach" of     noted in this report where applica/         assessment using actual partici/
partners prospectively, using a       ble. For one of the more problem/           pation.
community case study approach.        atic populations, American Indians,      • ;The factor analysis of PMPE
                                      the conclusion drawn is that                belief items and the use of a
2.3.2 Census in Schools Evaluation
                                      undercoverage "…should be com/              composite belief variable was
This survey-based evaluation pro/     parable to that achieved for this           illuminating. However, for com/
vided more detail on the effective/   population in the Census 2000               parison with previous research
ness of one major component of        itself." (Wolter et. al., 2002, p. 10)      further item-by-item analysis
the Partnership and Marketing                                                     would have been helpful. It is
                                      One limitation of the evaluation
Program, an attempt to introduce                                                  also not clear whether the lack
                                      was particularly highlighted by the
Census 2000 into the classrooms                                                   of movement in the belief com/
                                      authors: the inability to clearly and
of the country's elementary and                                                   posite between Wave 2 and
                                      separately quantify the effects of
secondary schools. Like the Survey                                                Wave 3 could have been due to
                                      the mass-media campaign and
of Partners, it too is more of a                                                  the performance of particular
                                      effects of the partnership program.
process than an outcome evalua/                                                   items, notably the negative
                                      Due to the variety of potential
                                                                                  beliefs stimulated by the nega/
tion, useful for assessing the        influences possible from the two            tive long form publicity.
implementation of the program         programs, as well as other census-
rather than its effect on Census      related stimuli (e.g., news reports,     2.3.4 Census Monitoring Survey
2000 return rates.                    etc.), the time periods covered,         While not a census-sponsored eval/
2.3.3 Partnership and Marketing       and known problems in respon/            uation activity, this study provided
Program Evaluation Survey             dent recall precision, the separa/       very interesting and useful week-
                                      tion of effects was seen to be a         by-week snapshots of the popula/
The three-wave (baseline, pre-cen/    daunting task. For this quantifica/      tions' reactions to the PMP and the
sus, post-census) survey design       tion, an experimental approach           mailout/mailback operation. The
was employed to address the tar-      was recommended in future evalu/         CMS provided the flexibility to
get populations in manners most       ations.                                  respond to breaking events, such
appropriate to their circumstances.                                            as the controversy over the long
                                      Some specific comments on the
A mixture of telephone and in-per-                                             form, perhaps its most valuable
                                      PMPE (others will be incorporated
son survey modes was used to                                                   contribution in 2000.
                                      in topic areas below):
ameliorate the difficulties in sam/                                            Unfortunately, two planned compo/
pling and surveying some hard-to-     • ;While the PMPE survey included        nents of this effort, additional
count populations. In the first          questions specific to messages        rounds of interviewing into the
wave, the expected sample frame,         in the advertising campaign,          NRFU period and collection of actu/
the Decennial Master Address File        analysis of these items was not       al return behavior, apparently did
(DMAF), was not available so a ran/      included in the report. There         not happen. These features would
dom-digit-dial (RDD) approach was        were also intended to be some-        have provided additional useful
implemented for surveying the            what different messages in dif/       insights.
general population and oversam/          ferent phases of the campaign–it      The CMS is a complement to the
ples of Hispanics, non-Hispanic          is not clear from the materials       PMPE, and an entirely different
African Americans, and native            provided what these variations        kind of survey. While the effective
Hawaiians. As described in more          were, nor whether the variations      response rates are somewhat lower
detail below, the Wave 1 response        were covered in the PMPE ques/        to considerably lower than those
rate was lower than the Wave 2           tionnaire, which had the same         of the PMPE survey waves, its tem/
and 3 administrations, which used        content regarding messages in         porally fine-grained view of the
the DMAF.                                each Wave.                            effects of the advertising campaign


14 Evaluations of the Census 2000 Partnership and Marketing Program                                U.S. Census Bureau
can't be replicated in the PMPE        problematic if not actually impossi/     the mailout/mailback operation.
style of survey design without         ble. It is difficult to imagine suc/     The NRFU effort and the associated
compromising the response rate,        cessful control of media outlets in      advertising campaign were active
which depends on repeated con/         a manner that would allow selec/         while Wave 3 was in the field. The
tacts over a period of several         tion of differential doses for select/   quantity and volume of ambient
weeks.                                 ed groups in the same way that           information about the census also
                                       soil Ph and moisture might be con-       peaked during the period between
2.4 Cross-study evaluation
                                       trolled in a classical experiment.       Wave 2 and Wave 3, and into the
topics
                                                                                Wave 3 field period. Finally, the
                                       Designing an experiment evaluat/
2.4.1 Experimental Design                                                       Wave 3 field period extended
                                       ing the effects of the Partnership
                                                                                almost a month after the end of
The studies evaluating the effects     Program is more conceivable,
                                                                                NRFU activities, which may have
of the Partnership and Marketing       although any such design would
                                                                                resulted in increased recall error
Program and the Census 2000            likely still be confounded by the
                                                                                for the later interviews. It is thus
Dress Rehearsal were observation/      mass media campaign.
                                                                                impossible for the PMPE to disen/
al. The limitations of observational   Partnership activities could be
designs in the establishment of                                                 tangle the effects of these different
                                       restricted to selected geographic
cause and effect are clearly stated                                             events, but it does provide useful
                                       areas so that experimental and
in the NORC report on the PMPE                                                  information on the cumulative
                                       control groups could be estab/
(also see Cook and Campbell,           lished. This would not establish         effects of all of the influences on
1979). (NORC's final recommenda/       the dose or mix of treatments            census behavior.
tion is to include an experimental     received, however, for surveyed          The CMS fit nicely into the gap
design in future evaluations.)         individuals (and recall, as noted        between the PMPE's Wave 2 and
From an evaluative perspective, it     above, can be a poor indicator).         Wave 3 to offset the limitation just
certainly would be easier and less     Although perhaps somewhat more           described. The week-by-week
ambiguous if it were possible to       tractable, it would also be difficult    design allowed tracking during the
partition people into real-world       to select initially comparable           peak period of PMP activity.
groups receiving measured doses        experimental groups for treat/           Unfortunately, since the planned
of advertising and/or exposure to      ments. At the least, such groups         last two weeks of the CMS, during
partnership activities. Classical      would have to have comparable            the NRFU, apparently did not hap-
(e.g., agricultural) experimental      baseline attitudes and knowledge         pen, there is no way to isolate the
designs have the power of control-     of the census and similar patterns       effects of the NRFU phase of the
ling sources of variation and so       of census form return.                   advertising campaign.
allow the establishment of causa/
tion and the quantification of         2.4.2 Timing of the evaluation           The major shortcoming of the
effects. An experiment might elim/     activities
                                                                                Survey of Partners was the timing
inate possible confounding of                                                   of the field period, many months
                                       The PMP was carefully designed to
mass media and Partnership activi/                                              after Census 2000 operations.
                                       build over time to a peak just
ties. Indeed, people included in                                                Some contact persons at partner
                                       before and during the mailout
such an experiment would not                                                    organizations had left the organi/
                                       /mailback operation, and to re/
even have to report exposure to                                                 zation, and others had difficulty
                                       energize for the NRFU operation.
such activities. Activity and expo-                                             recalling details of their involve/
                                       The evaluation activities were
sure levels would be set and                                                    ment. Thus, the survey probably
                                       designed around the PMP and
known.                                                                          underestimates somewhat both the
                                       Census 2000 operations. Wave 1
We do not think, however, that it      and Wave 2 of the PMPE were well         number of active partners and the
would be feasible to design a          positioned to capture the effects of     level of involvement of active part/
meaningful experiment for the          the Education phase of the paid          ners. About one-quarter of sam/
evaluation of the effects of mass      advertising campaign. Wave 3's           pled organizations from the
media on awareness and behavior.       position does not allow such a           Census Bureau's database were
By its very nature, mass media is      clean look at a particular part of       never contacted during the survey,
"out there" for all to see so the      the PMP: between Wave 2 and              despite multiple attempts both
control of its reach and frequency     Wave 3 was the Motivation phase          through the mail and over the tele/
for selected groups is at least        of the advertising campaign and          phone. The timing of the survey


U.S. Census Bureau                     Evaluations of the Census 2000 Partnership and Marketing Program 15
undoubtedly contributed substan/                  waves and any of the findings with      Table 5 shows the final weighted
tially to this low rate of contact.               regard to awareness, beliefs, or        and unweighted return rates for
                                                  behavior.                               participants in Wave 2 and Wave 3
2.4.3 Civic activities
                                                                                          of the PMPE survey and correspon/
                                                  The few findings presented in the       ding final return rates12 for the
Because the Y&R model of
                                                  PMPE report relating to civic partic/   population as a whole. For Whites
response likelihood was based on
                                                  ipation provide only very limited       and American Indians, the rates are
the assumption that general civic
                                                  support for the Y&R likelihood          reasonably comparable, but for
participation would be correlated
                                                  model. There were significant dif/      Hispanics, African-Americans, and
with returning the census form, it
                                                  ferences (in the expected direction)    Asians the survey respondent
might have been useful for the
PMPE to have put somewhat more                    in recent census awareness by           return rates are 10 or more per/
emphasis on civic participation in                level of civic participation for the    centage points higher than the
its design and analysis. The PMPE                 general population, for non/            population rates. Some of this dif/
questionnaire for each wave                       Hispanic African-Americans, for         ference may be due to the effects
included seven questions on civic                 Asians, and for American Indians.       of the survey, in essence acting as
participation. Given the distribu/                There were similar but non-signifi/     a motivating factor. However, by
tion of responses in the PMPE sur/                cant differences for Hispanics, non/    extrapolation, within these race
vey (Wolter et. al., 2002, Figure 16)             Hispanic Whites, Asians, and            and ethnic groups survey nonre/
                                                  Native Hawaiians. None of the final     spondents had a 50 percent or
compared with the Y&R estimates
                                                  multivariate models described in        lower final census return rate, indi/
(Wolter et. al., 2002, Figure 1) it
                                                  the report found a significant rela/    cating that there is some other sys/
seems likely that the Y&R model
                                                  tionship between civic participa/       tematic difference between respon/
was based on more items. Thus, it
                                                  tion and intended or actual partici/    dents and nonrespondents in these
is probably not appropriate to
                                                                                          groups. While it is impossible to
compare the two sets of estimates.                pation (when controlling for other
                                                                                          know what the bias would be, we
Two points are worth noting, how-                 factors). While there may have
                                                                                          surmise that some of the differ/
ever. First, the level of no civic par/           been a positive correlation
                                                                                          ences between the White and other
ticipation in the general population              between civic participation and
                                                                                          populations in awareness, beliefs,
in the Y&R model (17 percent) is                  returning the census form, other
                                                                                          etc., would have been even greater
almost identical to that found in                 factors would appear to explain
                                                                                          had there been no survey nonre/
the PMPE survey in Wave 1 and                     that correlation if it exists.
                                                                                          sponse. Note that the weighted
Wave 2. Second, the distribution of
                                                  2.4.4 Survey nonresponse                return rates for survey participants
amount of civic participation drops
                                                                                          are generally higher than the
in Wave 3, with those with no                     The response rates for PMPE Wave        unweighted rates, so the weighting
activities increasing to about 25                 1, Wave 2, and Wave 3 were 48.4         does not ameliorate the
percent and those with three or                   percent, 64.5 percent, and 67.7         apparent bias.
more activities dropping from                     percent, respectively. Wolter et. al.
about 24 percent to about 10 per-                                                         Martin (2001) discussed the
                                                  acknowledge that nonresponse is a
cent11. The decrease in civic partici/                                                    response rates and potential for
                                                  potential source of bias in the find/
pation (of those reporting at least                                                       bias in the CMS. The response
                                                  ings. Generally, nonresponse bias
one activity) between Wave 1 and                                                          rates for the five weekly CMS sur/
                                                  is more likely when variables of
Wave 3 is statistically significant                                                       veys ranged from 58 percent to 83
                                                  interest are correlated with the
for the general population, for non/                                                      percent. However, the sample
                                                  propensity to respond to the sur/
Hispanic African-Americans, and                                                           frame for these surveys was the
                                                  vey. Intuitively, one would expect a
for non-Hispanic Whites. The PMPE                                                         InterSurvey panel, recruited using
                                                  correlation between responding to       an RDD sample design. The com/
report does not explore any possi/
                                                  a telephone or in-person survey         bined response rate thus averaged
ble relationship between this dif/
                                                  and mailing back the census form.       around 30 percent, considerably
ference in civic participation across
                                                  Thus, there is reason to be con/        lower than the rates for the PMPE.
     11
        Wolter et. al. suggest that seasonality   cerned about nonresponse bias in
may explain this difference, perhaps with a       the PMPE survey and others that
reduction in PTA participation as the summer                                                   12
                                                                                                  We use final return rates rather than
approached, but were unable to test this          seek to evaluate the effects of         the rates through April 18 because of dis/
hypothesis. Since the questionnaire asks                                                  crepancies in the difference between the
                                                  PMP or other activities on the cen/
about participation in the past 12 months,                                                4/18 and final rates in the Wolter et. al.
this explanation seems unlikely.                  sus return rate.                        report and for the U.S. population.



16 Evaluations of the Census 2000 Partnership and Marketing Program                                                 U.S. Census Bureau
 Table 5.
 Final Return Rates for PMPE Wave 2 Survey Respondents by Sample and for Correspond
 ing Groups in the General Population
                                                                                                                                            Difference††
                                                             Final return rate
                                                                                                                         Estimated final    between and
                                                                                        Census 2000                       return rate for   respondents’
                                                                                          final return   PMPE Survey        survey non- non-respondents
                                                         Unweighted*        Weighted*           rate** response rate*†   respondents*** final return rates

 Wave 2
 Total Population. . . . . . . . . . . . . .                    81.1             84.4                            57.8
 Hispanic. . . . . . . . . . . . . . . . . . . . . .            81.9             81.7           69.2                               51.8              30.1
 African-American . . . . . . . . . . . . . .                   74.8             76.6           64.3                               49.9              24.9
 White . . . . . . . . . . . . . . . . . . . . . . . .          87.1             87.1           86.8                               86.4               0.7
 Other . . . . . . . . . . . . . . . . . . . . . . . .          79.0             79.3
 Asian . . . . . . . . . . . . . . . . . . . . . . . .          88.5             89.8           74.6             72.2              38.6              49.9
 American Indian. . . . . . . . . . . . . . .                   72.3             74.7           70.7             71.2              66.8               5.5
 Native Hawaiian. . . . . . . . . . . . . . .                   76.8             79.2           59.4             73.0
 Wave 3
 Total Population. . . . . . . . . . . . . .                    80.1             80.9                            64.6
 Hispanic. . . . . . . . . . . . . . . . . . . . . .            83.5             89.2           69.2                               47.6              35.9
 African-American . . . . . . . . . . . . . .                   73.5             74.6           64.3                               50.4              23.0
 White . . . . . . . . . . . . . . . . . . . . . . . .          83.9             81.7           86.8                               91.1              –7.2
 Other . . . . . . . . . . . . . . . . . . . . . . . .          79.0             60.9
 Asian . . . . . . . . . . . . . . . . . . . . . . . .          88.3             85.3           74.6             55.6              35.5              52.9
 American Indian. . . . . . . . . . . . . . .                   69.2             71.1           70.7             74.6              74.3              –5.1
 Native Hawaiian. . . . . . . . . . . . . . .                   76.1             78.0        †††59.4             72.6

      † Unweighted response rate 3 from Appendix B.
      ***Calculated from unweighted return rate, Census 2000 final return rate, and PMPE survey response rate.
      ††Difference in percentage points.
      ††† U.S. population value is for Pacific Islanders.
      *Source: Wolter et. al. (2002)
      **Source: Stackhouse and Brady (2002)



Martin made the following obser/                                   • ;Any biases that are present are             Nonresponse could be a serious
vations about the potential for bias                                  more likely to affect estimates             issue for the CIS Evaluation survey,
in the CMS:                                                           of levels, and less likely to               although it perhaps is of less con/
                                                                      affect trend estimates, since the           cern from a process evaluation per/
• ;The demographic composition                                                                                    spective than from the perspective
                                                                      bias should be relatively con/
   of the samples match reason-                                                                                   of producing population estimates.
                                                                      stant over the surveys.
   ably well with that of the                                                                                     Only 28 percent of the sample
   Current Population Survey,                                      The Survey of Partners achieved an             returned questionnaires, and 26
   except that the CMS under-rep/                                  overall response rate of 68 per-               percent were ultimately included in
   resents those with less than a                                  cent. The refusal rate was only                the analysis. As noted by the
                                                                   about 7 percent; most nonre/                   report's authors, "Our expectation
   high school education and over-
                                                                   sponse was due to difficulty in                is that teachers who did not hear
   represents voters;
                                                                   contacting partner organizations.              of the program or did not use the
• ;Likely biases are in Census 2000                                                                               CIS materials were less willing to
                                                                   As noted earlier, this difficulty was
   participation rates (participation                                                                             respond to the survey." This intu/
                                                                   likely due in part to the timing of
                                                                                                                  itively reasonable observation sug/
   is highly correlated with voting)                               the survey, and probably resulted
                                                                                                                  gests that the survey may over-
   and concerns about privacy                                      in under-estimating partner partici/
                                                                                                                  estimate awareness of the CIS
   (nontelephone households are                                    pation in Census 2000 activities,              Program and use of its materials.
   not included in the CMS sam/                                    since, for example, some contact               The fact that the response rate
   ples, and households with                                       staff responsible for census-related           among teachers sampled from the
   unlisted numbers are under-                                     activities were no longer with the             orderers' list (32 percent) was only
   represented);                                                   partner organization.                          a little higher than that among


U.S. Census Bureau                                                Evaluations of the Census 2000 Partnership and Marketing Program 17
those sampled from the commer/               Total nonresponse due to language       participation increased from
cial list (27 percent) provides some         and incapacity in Wave 1 ranged         Wave 1 to Wave 2 about the
reassurance about this potential             from about 2.5 to 3.5 percent in        same for English, Spanish, and
for bias, however. Nonetheless,              the core, American Indian, and          Other language spoken at home;
one should interpret level esti/             Asian samples, and was about 9
                                                                                   • ;The correlation between aware/
mates with caution. It is less clear         percent in the Native Hawaiian
                                                                                      ness of community-based com/
how non-response may have                    sample. In Wave 2, it was between
                                                                                      munications and intended par/
                                             2 and 2.5 percent for the core,
affected comparisons, such as                                                         ticipation did not increase
                                             Asian, and Native Hawaiian sam/
between HTE and non-HTE areas.                                                        significantly for Other, while it
                                             ples. In Wave 3, it was about 2.5
                                                                                      did for English and Spanish in
2.4.5 Language spoken at home                percent for the core, 4 percent for
                                                                                      the core sample and for English
                                             the Native Hawaiian sample, and 9
One important area where nonre/                                                       in the Native Hawaiian sample;
                                             percent for the Asian sample.
sponse may be an issue is with lin/          There was virtually no nonre/         • ;In the multivariate analysis,
guistically isolated individuals. Of         sponse for these reasons in the          households speaking an Other
the various studies evaluating the           American Indian sample in Waves 2        language at home were signifi/
PMP, only the PMPE survey has                and 3.                                   cantly more likely to return their
much to say about such people.                                                        census forms than were English-
                                             "Language spoken at home" was
The survey was conducted in                                                           at-home households.
                                             used several times as an explana/
English and Spanish. Some number
                                             tory variable in the PMPE report,     The last finding in particular is
of households were not screened,
                                             broken down as English, Spanish,      counter-intuitive; one would expect
and some number of interviews
                                             and Other. The findings include:      linguistically isolated households
were not conducted, because of
                                                                                   to have a lower return rate. The
language barriers13. We cannot tell          • ;Awareness of census communi/
                                                                                   authors discount the findings
whether translators or proxy inter-             cations increased significantly
                                                                                   about Other-language households
views were allowed for languages                for all languages between Wave
                                                                                   because of small sample sizes.
other than Spanish, but some num/               1 and Wave 3, but less so for
                                                                                   However, nonresponse bias may be
ber of interviews were conducted                Spanish and Other than for
                                                                                   particularly acute here-truly lin/
where a language other than                     English;
                                                                                   guistically isolated households
English was spoken at home.                  • ;The correlation between aware/     (other than those speaking
                                                ness of census communications      Spanish) would apparently not
     13
        Language problems and "incapacita/
                                                generally, and of mass media       have been able to complete
tion" were counted together in the PMPE
report, Appendix B.                             communications, and intended       the survey.




18 Evaluations of the Census 2000 Partnership and Marketing Program                                    U.S. Census Bureau
3. Summary of Results


Through its continuing program of         impact in terms of number of         The following statements are indi/
methodological research and inno/         individuals reached or increases     rectly supported by the evaluation
vation to improve the census, the         in participation rates.              data or other research:
Census Bureau was able in Census
                                        • ;The PMP, and the paid advertis/     • ;News stories about politicians
2000 to reverse the downward
                                          ing program in particular, dra/         saying that the long form is an
trend in mail return rates and
                                          matically increased awareness           invasion of privacy had a nega/
reduce the differential undercount.
                                                                                  tive effect on return rates for
The combination of several major          of the census among the gener/
                                                                                  the long form; CMS data are per-
evaluation efforts for the Census         al population, and among cer/
                                                                                  suasive on this point.
2000 Partnership and Marketing            tain traditionally hard-to-enu/
Program provides much insight             merate race and ethnic groups.       • ;The advance letter is likely to
                           ,
into the effects of the PMP and the                                               have had a positive effect on
connection between the PMP and          • ;Print media coverage of Census
                                                                                  response rates.
the other successes is intuitively        2000 nationally was much
compelling. However, the grand            broader than in 1990, and prob/      • ;The respondent-friendly ques/
prize of the evaluation activities, a     ably more positive in tone over-        tionnaire design is also likely to
direct connection between the PMP         all.                                    have had a positive effect, and
interventions and return rates,                                                   the effect may have been
remains elusive.                        • ;The PMP increased the propor/          greater in hard-to-enumerate
                                          tion of positive beliefs about the      areas.
Here is what we believe we can            census up to the time of the
say with confidence from the eval/                                             • ;Receipt of the mailout package,
                                          mailout.
uation data we have examined:                                                     particularly the long form, may
                                        • ;The effects on proportion of           have increased some negative
• ;The mandatory notice on the                                                    beliefs about the census.
                                          positive beliefs vary by belief
   questionnaire's outer envelope
                                          and by race and ethnic group.        • ;Nonresponse to the PMPE and
   had a positive effect on return
   rates.                               • ;Some positive beliefs about the        CMS probably results in an
                                                                                  underestimate of the differences
• ;The Partnership Program and            census are associated with
                                                                                  between the non-Hispanic White
   the Census in Schools Program          increased reported participation,
                                                                                  population and the Hispanic,
   were relatively successful in          and the association varies some-
                                                                                  non-Hispanic African-American,
   reaching out to hard-to-enumer/        what by race and ethnic group.
                                                                                  and Asian populations.
   ate populations, as evidenced
                                        • ;Organizations including many
   by the kinds of constituencies                                              • ;Nonresponse to the PMPE and
                                          targeting traditionally hard-to-        CMS also results in an underesti/
   active partners reported in the
   Survey of Partners and by the          enumerate populations, con/             mate of the differences between
   levels of awareness and use of         ducted a large number of activi/        English-speaking and linguisti/
   materials reported in the CIS          ties in support of Census 2000,         cally isolated households in
   evaluation survey, although it is      with cash and in-kind contribu/         awareness and behavior with
   not possible to quantify their         tions exceeding $500 million.           regard to the census.




U.S. Census Bureau                      Evaluations of the Census 2000 Partnership and Marketing Program 19
This page intentionally left blank.
4. Recommendations


It is very hard, if not impossible,     The data accumulated over three         rather than extending beyond it),
from the statistical evidence to        censuses should be able to provide      although a truncated field period
attribute the relative success of       guidance on what beliefs about the      would undoubtedly lower the
Census 2000 to the PMP or its           census (a) are associated with          response rate somewhat.
components. The fact remains that       return propensity and (b) can be
                                                                                While desirable, it is unlikely that
Census 2000 was much more suc/          influenced by marketing and com/
                                                                                the next survey would be any
cessful than predicted, and more        munication efforts, differentially by
                                                                                more successful than the 2000
successful than the 1990 Census.        race and ethnicity. We recommend
                                                                                PMPE at connecting the PMP or its
It seems reasonable to take the         that the next marketing and com/
                                                                                components directly with return
view that a number of changes           munication campaign review these
                                                                                behavior. We recommend that the
were made, many incorporated            findings to identify particular mes/
                                                                                analysis of the next PMPE survey
             ,
into the PMP and overall they were      sages to include for particular seg/
                                                                                focus more on the marketing strat/
successful. Thus, it seems reason-      ments of the population.
                                                                                egy itself and its messages, with
able that this successful strategy
                                        Given that general civic participa/     an eye toward continuous improve/
should be continued, with refine/
                                        tion does not seem to be a primary      ment.
ments that appear warranted in the
spirit of continuous improvement.       factor in return propensity
                                                                                The Survey of Partners was a use/
                                        (although the two may be correlat/
                                                                                ful process evaluation, and should
It is very likely that including the    ed), we recommend a review of the
                                                                                provide some guidance in planning
message that participation is           Y&R model to see how its segmen/
                                                                                and implementing a future
required by law on the mailout          tation strategy might be different if
                                                                                Partnership Program. If the survey
envelope had a positive effect on       other factors were used as the
                                                                                is repeated, it should be integrated
return rates, and should be contin/     basis.
                                                                                into the overall schedule so as to
ued. An experimental test of this
                                        We recommend a review of PMP            be fielded as soon after partner
message in a noncensus year
                                        activities, including the segmenta/     activities subside as possible. It
increased returns by about 10 per/
                                        tion strategy, to assess how they       would also be helpful to provide
centage points overall. Otherwise,
                                        might better reach younger adults       partners with some of the evalua/
we do not feel that the data war-
rant recommendations about con/         and those who speak languages           tion questions during the Program
tinuing specific aspects of the PMP     other than English and Spanish at       activities, so more (and more reli/
or not.                                 home. New formative work for            able) quantitative information
                                        these groups may also be                would be available for the survey.
The lack of an experimental design      warranted.
was a principal reason for not                                                  If funds are available, new kinds of
being able to associate the return      Regarding evaluation activities for     evaluation activities might be use/
rate success of Census 2000 with        the next census, we recommend           ful. For example, it may be possi/
PMP components. It does not seem        continuing PMPE- and CMS-type           ble to do more comprehensive
sensible to embed an experimental       surveys. We recommend extending         evaluations at a local level, using
design in a decennial census, but it    the CMS to go into the NRFU peri/       ethnographic and/or other qualita/
may be possible to do so in a           od, assuming that activities target/    tive methods as well as surveys
dress rehearsal or field test carried   ed to the NRFU remain a part of         and other quantitative methods14.
out in discrete geographic loca/                 ,
                                        the PMP and obtaining actual par/       A locally-based "case study"
tions. In particular, it would be       ticipation information on CMS           approach could begin to quantify,
instructive to vary the PMP or "cen/    households to include in the analy/
sus climate" factors as a whole         sis. It would be desirable for Wave
                                                                                     14
                                                                                        Census 2000 activities included
against different features of the       3 of the PMPE to coincide more
                                                                                ethnographic research, but not specifically to
mail pieces.                            closely with the NRFU period (i.e.,     help evaluate the PMP.



U.S. Census Bureau                      Evaluations of the Census 2000 Partnership and Marketing Program 21
for example, the reach of local and    households more efficiently than a    make causative connections not­
national partner activities or local   national survey. Combining qualita-   possible with broad quantitative­
media relations efforts. It could      tive and quantitative methods         data alone.­
also assess influences on the          locally would also provide more­
behavior of linguistically isolated    depth to the analysis, helping to­




22 Evaluations of the Census 2000 Partnership and Marketing Program                              U.S. Census Bureau
References                              Douglas Gould and Co. (2001),           Martin, E., and E. Rivers (2001), "A
                                        "Media Analysis of Census 2000,"        Look at Some Preliminary Results
Bates and Whitford (1991),
                                        Washington, D.C., U.S. Census           of Evaluating the Census 2000
"Reaching Everyone: Encouraging
                                        Bureau, February 28.                    Integrated Marketing Strategy,"
Participation in the 1990 Census,"
                                                                                paper prepared for presentation at
Proceedings of the Social Statistics    Fay, R.E., N. Bates, and J. Moore
                                                                                the American Association for Public
Section, American Statistical           (1991a), "Lower Mail Response in
                                                                                Opinion Research.
Association, 507-512.                   the 1990 Census: A Preliminary
                                        Interpretation," Annual Research        Martin, B., J. Moore, N. Bates, H.
Bentley, M. (2003), Census 2000
                                        Conference Proceedings,                 Woltman, D. Hubble, and E. Rivers
Auxiliary Evaluation: Evaluation of
                                        Washington, D.C., U.S. Census           (undated), "Project Overview-
Partnership and Marketing on
                                        Bureau, 3-32.                           Census Tracking Study,"
Improving Census 2000 Mail
                                                                                Washington, D.C., U.S. Census
Return Rates, Washington, D.C.,         Fay, R. E., Carter, W., Dowd, K.
                                                                                Bureau, unpublished document.
U.S. Census Bureau, April 3.            (1991b) "Multiple causes of nonre/
                                        sponse: Analysis of the Survey of       Moul, D.A. (2002), Census 2000
Citro, C.F., D.L. Cork, and J.L.
                                        Census Participation." Proceedings      Evaluation H.5: Nonresponse
Norwood, Eds. (2001), The 2000
                                        of the Social Statistics Section,       Followup for Census 2000,
Census: Interim Assessment,
                                        American Statistical Association,       Washington, D.C., United States
National Academy Press,
                                        525-530.                                Census Bureau.
Washington, D.C.
                                        Kulka, R. A., Holt, N. A, Carter, W.,   Nie, N., and J. Junn (2000), "Census
Clark, J.R., D.A. Dillman, and M.D.
                                        and Dowd, K. (1991) "Self-reports       Monitoring Study Summary
Sinclair (1993), "How Prenotice
                                        of time pressures, concerns for pri/    Findings," press release, May 4,
Letters, Stamped Return Envelopes
                                        vacy, and participation in the 1990     2000. InterSurvey, Inc.
and Reminder Postcards Affect
                                        mail census." Annual Research
Mailback Response Rates for                                                     Roper-Starch (1999), Promotion
                                        Conference Proceedings,
Census Questionnaires," Annual                                                  Evaluation: Evaluating the
                                        Washington, D.C., U.S. Census
Research Conference Proceedings,                                                Effectiveness of Paid Advertising in
                                        Bureau, 33-54.
Bureau of the Census, 37-48.                                                    the Census 2000 Dress Rehearsal
                                        Macro International (2002), Census      in Sacramento and South Carolina:
Cook, T.D. and Campbell, D.T.
                                        2000 Evaluation D.2: Evaluation of      Survey Findings, report prepared
(1979). Quasi-Experimentation -
                                        the Census in Schools Program:          for the U.S. Census Bureau.
Design & Analysis Issues for Field
                                        Materials and Distribution,
Settings, Rand McNally College                                                  Sha, B., and L.V. Collins (forthcom/
                                        Washington, D.C., U.S. Census
Publishing Company, Chicago.                                                    ing), "Communicating the
                                        Bureau Planning, Research, and
                                                                                Importance of Civic Participation in
Dillman, D.A., M. Sinclair, and J.R.    Evaluation Division, July 10.
                                                                                Census 2000: How America Knows
Clark (1993), "Effects of
                                        Martin, E. (2000), "Changes in pub/     What America Needs," draft chap/
Questionnaire Length, Respondent-
                                        lic opinion during the census."         ter for L.L. Kaid, D.G. Bystrom, M.S.
friendly Design, and a Difficult
                                        Paper presented at the Census           McKinney, and D.B. Carlin, eds.,
Question on Response Rates for
                                        Advisory Committee of                   Communicating Politics: Engaging
Occupant-addressed Census Mail
                                        Professional Associations, Virginia,    the Public in Democratic Life,
Surveys," Public Opinion Quarterly
                                        Oct. 19, 2000.                          in press.
57: 289-304.
                                        Martin, E. (2001), "Privacy             Stackhouse, H.F., and S. Brady
Dillman, D.A., E. Singer, J.R. Clark,
                                        Concerns and the Census Long            (2002), Census 2000 Evaluation
and J.B. Treat (1996), "Effects of
                                        Form: Some Evidence from Census         A.7.b: Census 2000 Mail Return
Benefits Appeals, Mandatory
                                        2000," Proceedings of the Section       Rates, Washington, D.C., United
Appeals, and Variations in
                                        on Survey Methods Research,             States Census Bureau.
Statements on Confidentiality on
                                        American Statistical Association.
Completion Rates for Census
Questionnaires," Public Opinion
Quarterly 60: 376-389.




U.S. Census Bureau                      Evaluations of the Census 2000 Partnership and Marketing Program 23
Tortora, R.D., S.M. Miskura, D.A.      Wolter, K., B. Calder, E. Malthouse,   U.S. Census Bureau (2003),
Dillman (1993), "Onward Towards a      S. Murphy, S. Pedlow, and J. Porras    "Decision on Intercensal Population
2000 Census Design: Research           (2002), "Census 2000 Evaluation:       Estimates," March 12, 2003,
Results," Proceedings of the Section   Partnership and Marketing Program      Washington, D.C., report.
on Survey Research Methods,            Evaluation," Washington, D.C., U.S.
                                                                              U.S. Census Bureau, Decennial
American Statistical Association,      Census Bureau Planning, Research,
                                                                              Management Division (undated),
120-128.                               and Evaluation Division, July 17.
                                                                              "Census 2000 Partnership and
Westat (2001), Census 2000             U.S. Census Bureau (2001),             Marketing Program: Program
Evaluation D3: Report of Survey of     Regional Partnership Report:           Master Plan," Washington, D.C.,
Partners, Washington, D.C., U.S.       Portrait of America. Washington,       unpublished document.
Census Bureau Planning, Research,      D.C., U.S. Census Bureau.
and Evaluation Division,
November 19.




24 Evaluations of the Census 2000 Partnership and Marketing Program                              U.S. Census Bureau
Appendix





        Figure 1.
        Schedule of Census 2000 Operations, Paid Advertising Campaign,
        and Evaluation Activities

                                                                    1999                                           2000
                                                    Sept      Oct       Nov     Dec       Jan      Feb       Mar      Apr          May      Jun


          Census 2000 Operations

             Advance Letter Mailout                                                                      ▲▼
                                                                                                         3/1 3/8

             Mailout/Mailback                                                                                  ▲      ▼
                                                                                                             3/15     4/18
             Nonresponse Followup                                                                                           ▲                 ▼
                                                                                                                            4/27            6/26
          Paid Advertising
          Campaign

             Education Phase                                        ▲                         ▼
                                                                    11/1                    1/30
             Motivation Phase                                                                            ▲            ▼
                                                                                                      2/28            4/9
             NRFU Phase                                                                                                   ▲          ▼
                                                                                                                      4/17           5/24

          2000 PMPE Survey                      ▲                          ▼              ▲                  ▼            ▲                 ▼
                                                9/1        Wave 1     11/13              1/17     Wave 2 3/11             4/17    Wave 3 6/17
          2000 Census Monitoring                                                                          ▲           ▼
          Survey                                                                                         3/3          4/17
          Census in Schools                                                                                                 ▲                ▼
          Program Evaluation                                                                                               4/27             6/23

          1990 OES                                                                        ▲       ▼                  ▲           ▼
                                                                                        1/13 W1 2/9                 4/9 W1 5/9
          1980 KAP                                                                        ▲       ▼        ▲ ▼
                                                                                        1/13 W1 2/9 3/8 W1 3/28



        Note: The Partnership Program began well before and extended throughout the time frame shown here.
        The Survey of Partners is not included in this figure since it occurred substantially later than this time frame.




U.S. Census Bureau                                 Evaluations of the Census 2000 Partnership and Marketing Program 25


				
DOCUMENT INFO
Shared By:
Stats:
views:4
posted:9/25/2010
language:English
pages:34