POQ Cell Phones

Document Sample
POQ Cell Phones Powered By Docstoc
					Public Opinion Quarterly, Vol. 73, No. 5 2009, pp. 845–865




CELL-PHONE-ONLY VOTERS IN THE 2008 EXIT POLL
AND IMPLICATIONS FOR FUTURE NONCOVERAGE
BIAS

MICHAEL MOKRZYCKI∗
SCOTT KEETER
COURTNEY KENNEDY




         Abstract Amid growing concern about potential noncoverage bias in
         random digit dial survey samples that exclude cell phones, a national
         face-to-face exit poll provided an opportunity to reach November 2008
         Election Day voters regardless of telephone status and to evaluate how
         the cell-only subgroup has changed since the 2004 election. The National
         Election Pool’s survey found a sharp increase in cell-only incidence,
         comparable to trends for the general public in government surveys, with
         cell-only status approaching the norm for voters under age 30. But vot-
         ers age 30 and older actually abandoned landlines at a faster rate, and
         the difference in presidential vote preference between the cell-only and
         landline-accessible voters in this age group was even greater than for
         younger voters. This suggests that typical poststratification weighting
         adjustments for age may be less likely to mitigate noncoverage bias in
         future landline-only RDD surveys.


In the past five years the survey research profession has devoted extensive
study to the question of whether landline random digit dial (RDD) samples
may suffer noncoverage bias by omitting people who only have cellular phones
(see e.g., American Association for Public Opinion Research Cell Phone Task
Force 2008).1 The potential problem attracted considerable attention from
news reporters and commentators in the 2004 presidential election and this

MICHAEL MOKRZYCKI  is an independent consultant, 168 Middle Street, West Newbury, MA 01985–
1926, USA. SCOTT KEETER is with Pew Research Center, 1615 L Street NW, Suite 700, Washington,
DC 20036, USA. COURTNEY KENNEDY is with the University of Michigan, Program in Survey
Methodology, 426 Thompson Street, Room 4050, Ann Arbor, MI 48104, USA. Address corre-
spondence to Michael Mokrzycki; e-mail: mike@mikemokr.com.
1. At least twenty-five papers presented at the 2009 annual AAPOR meeting focused entirely or
largely on aspects of cell phones in survey research, according to the conference program. And a
special 2007 issue of Public Opinion Quarterly 71(5) was devoted to the subject.
doi:10.1093/poq/nfp081
C The Author 2009. Published by Oxford University Press on behalf of the American Association for Public Opinion Research.

All rights reserved. For permissions, please e-mail: journals.permissions@oxfordjournals.org
846                                                    Mokrzycki, Keeter, and Kennedy


continued in 2008.2 However, until 2008, empirical evidence largely reassured
researchers the impact of cell phones on survey quality was minimal for most
topics. While the number of cell-phone-only Americans had steadily increased,
it remained a small fraction of the general population. Certain subgroups were
much more likely to be cell-only (most notably younger people) but differences
on demographic and attitudinal measures were small between them and their
cohorts who still had landlines. A 2006 study found that typical poststratifica-
tion weighting techniques in landline RDD samples produced survey estimates
that hardly differed nominally, much less to a statistically significant degree,
from blended landline and cell phone samples (Keeter et al. 2007). And analysis
of the 2004 Election Day exit poll found that cell-only and landline-reachable
voters differed little within age cohorts (Keeter 2006).
    Yet, substitution of wireless for landline telephone service has increased
unrelentingly since those studies were conducted. By the second half of 2008,
fully one in five U.S. households was cell-only according to the in-person
National Health Interview Survey (NHIS) (Blumberg and Luke 2009). Concern
about potential noncoverage bias prompted a number of national pollsters
to include cell phone samples in their pre-election surveys during the 2008
campaign. And these more recent dual-frame telephone studies began to suggest
these concerns may have been warranted. For example, in September 2008 the
Pew Research Center discerned a pattern in its national dual-frame telephone
polls on preference in the presidential race: including a sample of cell phone
interviews produced a 2- to 3-point shift in the margin toward Democrat Barack
Obama and away from Republican John McCain in all three surveys (Keeter,
Dimock and Christian 2008a). Other survey firms found directionally similar
results (Benford et al. 2009, Langer et al. 2009) although the differences were
just shy of statistical significance. A later Pew analysis of combined data
from its six pre-election polls from September through the weekend before the
election found a statistically significant but small 2.4-percentage point decrease
in Obama’s lead when cell phone interviews were excluded (Keeter, Dimock
and Christian 2008b).
    Fortunately, we find that final pre-election polling in 2008 was generally ac-
curate in the end, with surveys that excluded cell phones and those that included
them performing equally well compared to actual vote for president. Despite
the sizeable coverage gap, weighting adjustments appeared to have corrected
for the growing bias. But new evidence presented in this study indicates this

2. A search of the keyword combination of “cell phones,” “pollsters” and “polls” in the Nexis
database for the four weeks before the 2008 election yielded 160 stories in major newspapers,
news magazines or broadcast media; the same search of Factiva for the comparable time period
in 2004 found 150 stories (Keeter 2006). In both cases very tight search criteria likely produced
conservative estimates, and the search parameters excluded blogs. In 2008 some media references
may have been reports of results of surveys that began including a cell phone frame, but many
reports questioned whether the omission of cell phones from RDD samples might bias survey
estimates.
Cell-Phone-Only Voters in the 2008 Exit Poll                                  847


situation is not likely to last indefinitely. The seeming discrepancy between the
small noncoverage bias identified in recent dual-frame studies and their gener-
ally good performance in comparison with the single-frame final pre-election
polls is a puzzle with many possible solutions, but not a cause for future com-
placency. And aside from how accurately they predict vote preference, surveys
suffer a growing validity challenge on their face when they exclude one-fifth
or more of the population of interest.
   The national exit poll—conducted mostly in person outside voting places on
November 4, 2008—offered an opportunity to reach a sample of all Election
Day voters regardless of their telephone status. As in 2004, the news media
consortium that commissions the exit polls included a question on the 2008
national exit poll that allows analysis of attitudinal and demographic differences
between voters who only have cell phones and those who have landline service.
The sharp increase in the incidence of cell-only Election Day voters—from 7
percent in 2004 to 20 percent in 2008—enhances the statistical power of the
latest subgroup analysis (with nearly 1,500 cell-only respondents). These exit
poll data allow us to examine how the relationship between telephone service
and voting behavior has changed since 2004, as well as whether demographic
weighting to adjust for noncoverage of the cell-only voters remains effective.
   We proceed by first assessing the impact of cell-phone-only voters in the pre-
election polls, comparing the performance of dual-frame telephone polls with
single-frame polls sampling only landline numbers. We then present findings
from respondent-level analysis of our primary data source, the Election Day
national exit poll. We find evidence that the risk of noncoverage error in landline
surveys has increased since the 2004 election because the voting behavior
of landline and cell-only respondents within age groups is less similar than
it was four years ago. Somewhat surprisingly, the difference in presidential
vote preference between cell-only and landline-accessible voters was greater
among older voters than among younger voters in 2008. We close by drawing
implications for future telephone surveys.



The Predictive Accuracy of Dual- versus Single-frame 2008
Pre-election Polls
To begin to understand the consequences of cell phone noncoverage in 2008,
we reviewed fifteen national telephone polls with RDD sampling conducted
during the final week of the presidential campaign. More than half of these
polls were conducted for news media outlets; the rest were conducted by
private or academic survey organizations for marketing or analytic purposes.
Results for these polls were released to the public and summarized on political
polling websites including Pollster.com and RealClearPolitics.com. Six used
a dual-frame design and nine sampled landlines only. All the polls projected
the correct winner, but given Obama’s 7-point victory margin, this is perhaps
848                                                 Mokrzycki, Keeter, and Kennedy


Table 1. Accuracy of Pre-Election Polls by Sample Design
                                                  Obama      McCain         A        s.e.(A)
Election result                                      52.9       45.7
Landline telephone RDD
  FOX News/Opinion Dynamics                          50         43         0.00       0.07
  CNN/Opinion Research Corp.                         53         46         0.00       0.08
  American Research Group                            53         45         0.02       0.06
  IBD/TIPP                                           52         44         0.02       0.07
  Rasmussen                                          52         46        −0.02       0.04
  George Wash. U. (Lake/Tarrance)                    49         44        −0.04       0.08
  Diageo/Hotline                                     50         45        −0.04       0.07
  Research 2000                                      51         46        −0.04       0.06
  Marist College                                     52         43         0.04       0.07
Mean A for landline RDD                                                    0.007
Mean absolute value (A) for landline RDD                                   0.026

Landline and cell telephone RDD
  Ipsos/McClatchy                                    53         46         0.00       0.07
  Pew Research Center                                52         46        −0.02       0.04
  NBC News/Wall Street Journal                       51         43         0.02       0.07
  ABC News/Washington Post                           53         44         0.04       0.04
  CBS News/New York Times                            51         42         0.05       0.08
  Gallup                                             55         44         0.08       0.04
Mean A for dual-frame RDD                                                  0.027
Mean absolute value (A) for dual-frame RDD                                 0.036

  NOTE.—Means are based on A values prior to rounding.
  SOURCE.—Pre-election poll estimates compiled by Pollster.com and RealClearPolitics.com.


to be expected. To gauge the relative performance of the polls, we computed
the A value for each (Martin, Traugott and Kennedy 2005). The measure A
summarizes the degree to which the poll estimate of the relative vote share of
the top two candidates deviated from the election result. A values closer to zero
reflect greater accuracy than those farther from zero in either direction. Values
with a positive sign reflect overestimation of support for Obama (and under-
estimation of support for McCain) relative to the outcome. Negative values
reflect overestimation of support for McCain (and underestimation of support
for Obama). The results are presented in table 1. Several of the polls allocated
the percentage of undecided voters to the candidates in their final estimates.
A is not altered if the undecided voters are dropped, allocated proportionally,
or kept as undecided. Of course, there are other differences among the polls
that could affect their accuracy, such as how the data are weighted or how they
screen for likely voters. We cannot account for these differences.
   As seen in table 1, landline polls and dual-frame polls differed relatively little
in accuracy by this measure. The mean of the absolute values of A was 0.026 for
Cell-Phone-Only Voters in the 2008 Exit Poll                                               849


the landline RDD polls versus 0.036 for the dual-frame polls. At the same time,
the observed pattern taking account of the sign of A is consistent with the finding
in pre-election polls that dual-frame surveys have a higher vote share for Obama.
Of the nine landline-only polls, four had negative values for A (signifying an
underestimate of Obama’s margin). By contrast only one of the dual-frame
surveys had a negative value for A. Nonetheless, judged by predictive accuracy
the dual-frame surveys did not outperform single-frame surveys.


Design of the 2008 National Exit Poll
Although dual-frame pre-election surveys provide a way to gain insights into
the thinking and demographics of cell-only voters, the national exit poll is
a unique source of data because it doesn’t rely on telephones to reach re-
spondents and it only interviews those who actually have voted. The exit poll
also asks a wider range of questions about political attitudes and behavior
than in-person government surveys that provide benchmarks of the size of the
cell-only population. In the U.S. general elections of 2004 and 2008, the Na-
tional Election Pool (NEP)—ABC, The Associated Press, CBS, CNN, FOX
and NBC—commissioned surveys of voters in all fifty states and the District
of Columbia from Edison Media Research and Mitofsky International, as well
as a national sample. Interviews predominantly were conducted in person on
Election Day in a probability sample stratified by geography and past vote.
(See Appendix for methodological details.)
   In recent years, a growing number of voters in many states have cast ballots
early or absentee before Election Day. By November 2008 nearly a third of all
votes (32.7 percent) were cast before Election Day, up from less than a quarter
(22.5 percent) in 2004 (Associated Press 2009). To cover early voters, the in-
precinct “exit” polls—which interview voters as they exit the voting booth—
have been supplemented in numerous states and nationally by telephone surveys
conducted the week before the election. Note that all analysis herein is based
solely on voters who cast ballots on Election Day in the national sample of
300 precincts, as the 2008 NEP telephone surveys—in eighteen states plus a
national phone sample—only covered households with landline phones. An
analysis of possible effects of excluding cell-only voters from the NEP early
voter telephone samples is beyond the scope of this paper; that noncoverage
has no bearing on the population being studied here, Election Day voters.
   Election Day respondents complete the exit poll by filling out a paper ques-
tionnaire. There were four versions of the national exit poll instrument, two of
which included the question shown in figure 1.3


3. There were three minor differences in the presentation of the question in 2004: The words “in
your home” were underlined; the parenthetical “Check only one” instruction was not capitalized;
and in the second response code there was a comma after the word “regular.”
850                                                     Mokrzycki, Keeter, and Kennedy




Figure 1. Telephone-Status Question as It Appeared on the 2008 National Exit
Poll Questionnaire.
Source.—Election Day national exit poll conducted by National Election Pool,
November 4, 2008.

   Although this survey offers a unique opportunity to examine the cell-only
phenomenon, it is far from perfect. For one thing, the wording of the tele-
phone status question does not permit identification of “cell-mostly” voters—
individuals who have landline phones but mostly rely on their cell phones and
thus may be difficult or impossible to reach in landline samples.4 Also, data
are not available on many substantive and demographic variables in the exit
poll, such as marital status or religious affiliation, which were not included
on the same forms as the telephone status question. And with limited ques-
tionnaire space, the exit poll did not include measures of some demographic
characteristics that government studies consistently find to be strong predictors
of cell-only status, such as living in a household with only unrelated adults or
renting rather than owning one’s home.


Results

INCIDENCE OF CELL - PHONE - ONLY VOTERS OVERALL

We first look at the basic incidence of cell-phone-only voters in the exit poll
sample in 2008 compared to 2004. The proportion of Election Day voters who
live in cell-only households nearly tripled over four years, to 19.9 percent
in 2008 (table 2). This is similar to the finding of the general population
National Health Interview Survey, which found 20.2 percent of households
had no landline but at least one wireless telephone in the second half of 2008.
Another 4.1 percent of Election Day voters reported in the exit poll that their
household had no telephone service at all, indicating that pre-election polls

4. “Cell-mostly” or “cell-mainly” voters are of increasing interest to survey researchers. The
evidence presented thus far is that the potential for bias from omitting “cell-mostly” respondents
from samples is more modest than from noncoverage of those who are “cell-only.” See, for example,
Keeter et al. 2009, one of at least six papers presented at the 2009 AAPOR annual conference that
addressed this topic. NEP considered modifying the wording of the telephone status question on
the exit poll to capture “cell-mostly” data but opted instead to keep the 2004 wording in 2008 for
trend analysis with the largest possible sample size.
Cell-Phone-Only Voters in the 2008 Exit Poll                                                 851


Table 2. Household Telephone Status in Election Day National Exit Polls
                                             2008                               2004
                                 Weighted        Unweighted         Weighted        Unweighted
                                   (%)              N                 (%)              N
Both regular landline               62.3             4,576             70.1            3,962
  and cell phone service
Only regular landline               13.7               952             20.7            1,087
  phone service
Only cell phone service             19.9             1,496               7.1              446
No telephone service at              4.1               317               2.1              124
  home
Total                              100.0             7,341            100.0            5,619

  SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,
2008, and November 2, 2004.

using only landline samples failed to cover about 24 percent of the Election
Day electorate.5

VOTE FOR PRESIDENT BY TELEPHONE STATUS AMONG ELECTION
DAY VOTERS

We now turn to analyzing how the growing population of cell-only voters
differs from those reachable by landline. Table 3 shows how Election Day
voters cast their ballots for president in 2004 and 2008 by telephone status.
The 2008 results reinforce the findings from four years earlier: cell-only voters
remained a source of outsized support for the Democratic candidate, while the
Republican continued to fare best among Election Day voters with both regular
landline service and cell phones in their household. This is not surprising, given
that vote for Republicans tends to grow with higher income, and those with
higher income can better afford to have both landline and cell phones.

THE GROWING CELL - ONLY VOTER POPULATION

While much of the focus on cell-only voters remains on the youngest age
groups6 , where the incidence rates are very high, the majority of cell-only
5. These estimates are subject to debate. Recent research on list-assisted RDD designs suggest
that actual noncoverage rates for both landline RDD and dual-frame RDD are even higher (Fahimi,
Kulp, Brick 2008). Meanwhile, the estimated 4.1 percent incidence of households with no telephone
service is higher than the NHIS estimate of 1.9 percent in the second half of 2008. Reasons for
this difference may include sampling error, the fact that the sample of Election Day voters covers
a smaller portion of the U.S. public than does the NHIS sample of all U.S. households, and
differences in NEP and NHIS question wording.
6. The 2008 NHIS study finds a smaller proportion of all adults age 18–24 (33.1 percent) are in
wireless-only households compared to 18–24 voters in the Election Day exit poll (42.2 percent)
852                                                      Mokrzycki, Keeter, and Kennedy


Table 3. Presidential Vote by Household Telephone Status in Election Day
National Exit Polls
                                                      2008                             2004
                                           Obama             McCain            Kerry           Bush
                                            (%)               (%)               (%)            (%)
ACTUAL POPULAR VOTE                          52.9              45.7             48.3           50.7
  (Election Day + early)
Estimate among all Election                  52.4              46.2             48.4           50.8
  Day voters
Both regular landline and cell               49.2              49.2             46.5           52.6
  phone service
Only regular landline phone                  52.1              46.6             51.6           47.5
  service
Only cell phone service                      60.5              37.8             53.5           44.4
No telephone service at home                 61.1              36.2             58.8           39.5

  SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,
2008, November 2, 2004; Federal Elections Commission.


voters are thirty and older. This was true even in 2004, when 51.7 percent of
cell-only voters in the NEP exit poll were at least thirty years old. Of course,
part of this is simply the age composition of the voting public: the vast majority
of voters are at least thirty years old—more than 80 percent in both 2004 and
2008. The 2008 NHIS study indicates that 21.6 percent of the 30–44 age group
was cell-only in the last half of 2008 (up from 19.1 percent in the first half
of the year), very close to NEP’s estimate for this age group among voters
(20.2 percent cell-only). Table 4 shows these differences in telephone service
from 2004 to 2008 by age groups in the Election Day national exit poll. Note
that in this and following tables, the results for “Landline” voters collapse two
categories in the exit poll question: “Both regular land-line and cell phone
service” and “Only regular land-line phone service.”
   The percentage of voters ages 30–39 who were cell-only more than tripled
between 2004 and 2008, from 7.3 to 23.2 percent. Interestingly, as a percentage
increase over the 2004 baseline this growth rate was even greater than for voters

Blumberg and Luke 2009. While these datasets lack all the necessary variables to draw firm
conclusions, one possible explanation for this finding would be that the youngest cell-phone-only
adults are more likely to vote than their cohorts with landlines. This may be related to a difference
between young adults away at college and thus more likely to be wireless-only compared to young
cohorts who may still be living at home with parents and thus more likely to have landlines in
the household; in the exit poll, cell-only 18- to 24-year-olds were more likely than their landline-
reachable cohorts to be college graduates or have some college education. Another factor in the
difference between the NHIS and NEP Election Day estimates may be differential distribution
of cell-only 18- to 24-year-olds between the early voting and Election Day electorates. Question
wording also may contribute to the difference.
Cell-Phone-Only Voters in the 2008 Exit Poll                                          853


Table 4. Telephone Service by Age in Election Day National Exit Polls
                         18–24     25–29     30–39     40–49    50–64      65+      Total
2008
  Landline                51.7      51.8      72.2      81.4     87.4     91.1      76.0
  Cell only               42.2      40.8      23.2      14.8      9.7      7.4      19.9
  No phone                 6.0       7.4       4.7       3.7      2.9      1.5       4.1
                         100.0     100.0     100.0     100.0    100.0    100.0     100.0
2004
  Landline                77.6      74.5      89.8      94.6     95.6     96.7      91.0
  Cell only               18.5      20.0       7.3       4.2      3.7      2.3       7.1
  No phone                 3.8       5.5       2.9       1.2      0.7      1.1       2.0
                         100.0     100.0     100.0     100.0    100.0    100.0     100.0

Cell-only 2008–2004 +23.7∗ +20.8∗ +15.9∗ +10.6∗ +6.0∗ +5.1∗ +12.8∗
Pct increase on base 128% 104% 218% 253% 165% 226% 182%

  SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,
2008 November 2, 2004.
  ∗ p < .05.




under age 30. This can be explained in part by aging over four years of the
cohort that in 2004 was nominally most likely to be cell-only—those who then
were twenty to twenty-nine years old. In 2008 the cell-only growth rate in
the survey among the 40–49 age group was even greater than for those aged
30–39, even though the absolute change was smaller. Statistically significant
increases also occurred among those 50–64 and 65 and older. With wireless
substitution growing rapidly across all age groups, differences in the growth
rates by age can have consequences for poststratification adjustments by age in
landline-only RDD surveys.



TELEPHONE STATUS BY OTHER DEMOGRAPHIC AND ATTITUDINAL
VARIABLES

Cell-phone-only status also is related to other key demographic correlates of
electoral behavior. These relationships are important because of the implica-
tions for demographic weighting used to adjust for noncoverage. Keeter (2006)
reported how the incidence of cell-only status varied by race, education, income
and other factors in the 2004 national exit poll. We compared those results to
the 2008 figures to determine how the rate of transitioning to cell-only status
changed along these dimensions. The results are presented in figure 2. The
most sizable increases in cell-phone-only status occurred for the lowest income
and least educated voters, and for Latinos. In 2004 some 13 percent of Election
Day voters with an annual income under $15,000 were cell-only; by 2008 that
854                                            Mokrzycki, Keeter, and Kennedy




Figure 2. Proportion of Election Day Voters Who Are Cell-Only, by Demo-
graphics.
Source.—Election Day national exit poll conducted by National Election Pool,
November 4, 2008 and November 2, 2004. The 2008 standard errors range
from 2.0 for the estimate for whites to 8.2 for the estimate for non-high school
graduates.




figure nearly tripled to 37 percent. Similarly, 9 percent of Latino or Hispanic
voters in 2004 were cell-only, but this rose to 28 percent in 2008.
   Among most Election Day voters, the likelihood of being cell-only declined
as income rose, but in both 2004 and 2008 the trend was not monotonic. Curi-
ously, the trend reversed slightly for those with the highest incomes. In 2008 this
curvilinear effect of income is highly significant in logistic regression models
predicting cell-only status, with and without controls for other demographics.
The mechanism behind this pattern is unclear because the ability to afford land-
line service is clearly not a factor for those at the highest income levels. One
potential consequence of this phenomenon may be to diminish the reliability
of landline-only surveys that target high-income population subgroups.
Cell-Phone-Only Voters in the 2008 Exit Poll                                          855


Table 5. Presidential Vote by Telephone Service in Election Day National Exit
Polls
         2008     Landline Cell only     Diff.    2004 Landline Cell only Difference
18–24 Obama         61.2        68.6    −7.4     Kerry      59.5       62.5       −3.0
      McCain        36.8        29.3     7.5     Bush       39.3       36.5        2.8
       Other         2.1         2.1             Other       1.2        1.0
                   100.0       100.0                       100.0      100.0

25–29 Obama         63.3        65.4    −2.1     Kerry      51.5       53.4       −1.9
      McCain        35.6        33.6     1.9     Bush       46.6       45.5        1.2
       Other         1.1         0.9             Other       1.8        1.1
                   100.0       100.0                       100.0      100.0

30–39 Obama         51.3        62.6 −11.3∗      Kerry      46.9       49.3       −2.5
      McCain        47.1        36.2  10.9∗      Bush       52.3       48.0        4.3
       Other         1.6         1.2             Other       0.9        2.7
                   100.0       100.0                       100.0      100.0

40–49 Obama         46.6        54.5    −7.9
      McCain        51.2        44.3     6.9
       Other         2.2         1.1
                   100.0       100.0

50+     Obama       46.9        46.8      0.1
        McCain      52.2        52.1      0.1
         Other       0.9         1.1
                   100.0       100.0

  SOURCE.—Election Day national exit polls conducted by National Election Pool, November 4,
2008 and November 2, 2004.
  ∗ p <.05 in difference of proportions test.




PRESIDENTIAL VOTING BY AGE AND PHONE STATUS

Perhaps the most striking finding of the 2008 exit poll question on telephone
status was how differences in voting patterns between cell-phone-only and
landline-reachable respondents played out among age groups. Differences in
presidential vote between the landline and cell-only groups in 2008 were largest
among voters ages 30–39 (table 5). Obama was the choice of 62.6 percent of
cell-only voters ages 30–39 but only 51.3 percent among those reachable by
landline, a difference of 11.3 percentage points that was significant at the .05
level. The difference was smaller and not significant among all other age groups
in the survey: 7.9 points among those 40–49, 7.4 points among those 18–24, and
less among those 25–29 and 50+. In 2004 older cell-only voters were such a
small part of the voting population that their voting patterns, however different,
856                                            Mokrzycki, Keeter, and Kennedy


posed little risk of biasing election surveys based on landline samples. This had
clearly changed by 2008.
   Among all Election Day voters thirty and older, Obama was the choice of
55.5 percent of those who were cell-only and 47.8 percent of the landline-
accessible, a significant difference of 7.7 percentage points. Among Election
Day voters 18–29, the difference was 4.9 points, though not statistically
significant. Four years earlier, the difference in vote for Democratic presidential
nominee John Kerry among the 30+ age group was 2.8 points; among those
18–29 it was 2.2 points. Although the sample sizes in the poll cannot support
a definitive conclusion, the disjuncture between landline and cell-only voters
appears to have grown more among older voters than among younger ones.
Because age is typically a poststratification weighting variable, this result
suggests adjustments by age may be insufficient to overcome noncoverage in
landline-only RDD surveys.
   Generally, voter attitudes other than presidential vote choice differed some-
what less by telephone status. Table 6 shows cell phone status by party identifi-
cation and ideology, within age groups. The differences across telephone status
within age groups indicate how well cell-phone-only voters are represented by
their landline-accessible peers. Several differences are apparent. Among older
Election Day voters (age 30–39 and 40+), those living in landline households
are significantly more likely to consider themselves conservative than their
cell-only peers. Among younger voters, by contrast, telephone status appears
to be more closely related to party identification, though these differences were
not statistically significant.
   Although not shown, we analyzed a number of other measures asked on the
exit poll questionnaire. Significant differences by telephone status appeared in
variables related to partisanship and ideology; for example, cell-only respon-
dents were 10 points more likely than landline-reachable voters to strongly
disapprove of George W. Bush’s performance as president. On the other hand,
there were no significant differences by telephone status on whether the can-
didates’ race or age were factors in voting for president. And measures of the
economy produced mixed results: there was no significant difference in like-
lihood of calling the economy the nation’s most important issue nor in worry
about the economy’s direction over the next year, but cell-only Election Day
voters were a little more likely than the landline-reachable to say the economy
currently was poor—yet slightly less likely to call it “not so good.” These
results suggest it may not be easy to predict how cell phone noncoverage bias
may affect estimates on individual attitudinal items within a survey.
   There also were interesting differences by phone status in voter mobiliza-
tion, as shown in table 7. Cell-only Election Day voters were far more likely
than those with landlines to have voted for the first time in 2008, though nat-
urally this correlates strongly with age (the youngest voters are most likely to
be cell-only and also most likely to be new voters). But this same group also
was far less likely to have been contacted by either campaign. Even Obama’s
Cell-Phone-Only Voters in the 2008 Exit Poll                                          857


Table 6. Ideology and Party Identification by Telephone Status and Age in
Election Day National Exit Poll
                              Landline phone          Cell phone only          Difference
Age 18–29
  Party identification
    Democrat                        48.4                    43.0                   5.4
    Republican                      27.9                    26.5                   1.5
    Independent                     18.6                    23.8                  −5.2
    Other                            5.1                     6.7                  −1.7
  Ideology
    Liberal                         32.7                    32.2                   0.6
    Moderate                        43.6                    43.6                  −0.1
    Conservative                    23.7                    24.2                  −0.5
Age 30–39
  Party identification
    Democrat                        39.2                    45.0                  −5.8
    Republican                      36.0                    28.3                   7.7
    Independent                     20.6                    21.9                  −1.4
    Other                            4.2                     4.8                  −0.6
  Ideology
    Liberal                         21.9                    32.3                 −10.4∗
    Moderate                        44.3                    43.5                   0.7
    Conservative                    33.8                    24.2                   9.6∗
Age 40+
  Party identification
    Democrat                        39.9                    45.5                  −5.6
    Republican                      36.4                    32.0                   4.4
    Independent                     21.3                    17.9                   3.4
    Other                            2.4                     4.6                  −2.2
  Ideology
    Liberal                         18.6                    17.3                   1.3
    Moderate                        45.2                    53.6                  −8.3∗
    Conservative                    36.1                    29.1                   7.0∗

  SOURCE.—Election Day exit poll conducted by National Election Pool, November 4, 2008.
  ∗ p < .05 in difference of proportions test.




campaign—renowned for its technological innovation, including its plan to
use text messaging to notify 3 million supporters of Obama’s vice-presidential
pick—was more likely to contact landline-reachable than cell-only voters, ac-
cording to the Election Day exit poll. (So it’s not only pollsters who have trouble
reaching those who are wireless-only.) These differences remain statistically
significant even within age groups.
   Given extensive research showing that mobilization efforts are effective at
stimulating voter turnout, these findings have important normative implications.
858                                                Mokrzycki, Keeter, and Kennedy


Table 7. Voter Mobilization by Telephone Status in Election Day National Exit
Poll
                                      Landline phone      Cell phone only      Difference
                                           (%)                  (%)               (%)
Is this the first year you have ever
   voted?
   Yes                                       9.8                22.3             −12.5∗
Did anyone call you or talk to
   you in person on behalf of
   either major presidential
   campaign about coming out to
   vote?
   Contacted by both                        18.0                 7.1             +10.9∗
   Contacted by Obama only                  16.4                15.8              +0.7
   Contacted by McCain only                  8.4                 2.7              +5.7∗
   Contacted by neither                     57.2                74.4             −17.2∗

  SOURCE.—Election Day exit poll conducted by National Election Pool, November 4, 2008.
  ∗ p < .05 in difference of proportions test.




Young people, not surprisingly, are much less likely than older people to have
developed habits of political engagement and are in greater need of mobilization
to vote in elections. Yet, because of their telephone status, they are far less likely
to be reached by campaigns, even if the campaigns intend to reach them. A
similar logic may apply to lower income and minority individuals. This pattern
almost certainly will exacerbate political inequality.



POSSIBLE REASONS FOR POLITICAL DIFFERENCES BETWEEN CELL - ONLY
AND LANDLINE - ACCESSIBLE VOTERS

The observed political differences between the cell-only and landline-
accessible voters likely stem from the outlined life-cycle and socio-economic
differences between these two groups, which correlate with political views.
Pre-election polls typically collect some indicators of these mechanisms. Age,
marital status, and presence of children in the household are commonly used
indicators of life-cycle stage. Income, education, and homeownership are com-
monly used indicators of socio-economic status. Many but not all of these
variables were measured alongside telephone service in the 2008 national exit
poll.
   We used logistic regression to better understand how these factors relate
in a multivariate setting. The outcome we modeled is vote for Obama among
those voting for either of the major party candidates. The model was estimated
separately for those aged 18–29, 30–39, and 40 and older, because we expect
Cell-Phone-Only Voters in the 2008 Exit Poll                                              859


Table 8. Logistic Regression Models Predicting Vote for Obama, by Age
Groupˆ
                                Age 18–29              Age 30–39               Age 40+
Parameter                       B         s.e.         B          s.e.        B           s.e.
Telephone service
   No telephone              0.10      0.26          0.18      0.29         0.30       0.22
   Cell-only                 0.36∗     0.13          0.58∗     0.15         0.14       0.12
Male                        −0.26∗     0.13         −0.25∗     0.12        −0.32∗      0.07
African-American             3.44∗     0.46          2.89∗     0.32         3.84∗      0.32
Hispanic                     0.83∗     0.20          0.54∗     0.21         0.64∗      0.15
Education                    0.11      0.06          0.26∗     0.06         0.25∗      0.04
Income                      −0.26      0.14         −0.29      0.16        −0.38∗      0.09
Incomeˆ2                     0.02      0.02          0.02      0.02         0.03∗      0.01
Intercept                    0.92∗     0.32          0.03      0.42         0.34       0.22
Model sample size               (1,514)                 (1,428)                (3,727)
% Predicted correctly           73.5%                   64.0%                   58.3%

   NOTE.—The results do not change substantively in any of the three models when weighting is
used.
   SOURCE.—Election Day exit poll conducted by National Election Pool, November 4, 2008.
   ∗ p < .05.

   ˆReference groups are “has landline,” female, not African-American, and not Hispanic. Models
are based on Election Day voters voting for either Barack Obama or John McCain. Model estimates
are unweighted so as not to distort the effect of demographic predictors.



the predictors to interrelate differently depending on the voter’s age. We also
want to maintain our focus on how noncoverage error might arise in landline
telephone polls if cell-only voters differ from landline-accessible voters within
their age group, since age is a common variable used in weighting. Presidential
vote was regressed on telephone status (cell-only, no phone, with landline as
the reference group), gender, race, Hispanic ethnicity, education, income, and a
curvilinear effect for income. In a preliminary model, we tested for interaction
effects between telephone service and the demographic variables, but none was
significant. The final model estimates are reported in table 8.
   Cell-only status is a highly significant predictor of voting for Obama among
those 30–39 years old, even when controlling for common weighting variables.
It also remains significant for those 18–29, though to a slightly lesser degree.
Telephone service is not, however, predictive of an Obama vote among those
aged 40 and older.
   The regression analysis predicting the vote finds that including the avail-
able demographic variables—age, sex, education, income and race—reduces
the impact of telephone status somewhat (compared with a model in which
the demographic factors are not included) but leaves much of the variability
unexplained. The gap in predicted probability of a vote for Obama between
860                                            Mokrzycki, Keeter, and Kennedy


cell-only and landline-accessible voters, without controlling for demographic
characteristics, is 10.7 percentage points. Including the demographics in the
model reduces this difference to 6.7 points.
   These results suggest that noncoverage bias can persist in landline survey
estimates even when standard demographic weighting is employed. Unfortu-
nately, other important variables related to phone status and the vote were not
asked on the same forms of the questionnaire that included phone status, in-
cluding marital status and presence of children in the home. Religion, which
is associated with social integration and is correlated with phone status, also
was not asked on the same forms. Homeownership—with renting known to
be a strong predictor of cell-only status—was not asked in the exit poll at all.
A similar multivariate analysis using a September 2008 Pew Research Center
poll found that the independent impact of cell-only status on vote choice dimin-
ished to insignificance when marital status and homeownership were included
along with standard weighting demographics, suggesting that more elaborate
weighting protocols might help to reduce noncoverage bias (Keeter et al. 2009).




Conclusions
Considering the trend in telephone usage and the evidence for the potential
growth of coverage error, researchers should ensure that the cell-only popula-
tion is covered in future pre-election surveys, including the National Election
Pool’s own surveys of early and absentee voters. It may seem odd to make such
a recommendation in the absence of strong evidence of bias in vote preference
estimates in pre-election polls to date. But the risk of error posed by noncover-
age is no longer trivial. And there are other reasons beyond forecast accuracy
to improve the coverage of polls.
   The percentage of Election Day voters who were cell-only nearly tripled
between 2004 and 2008, mirroring trends in government surveys of the gen-
eral population. There is no evidence that this rapid rate of growth is likely
to diminish in the near future. Because of the sizeable differences between
landline-accessible and cell-only voters, the potential for noncoverage bias in
future landline-only surveys may be substantial.
   The good news is that it appears that noncoverage bias did not present a
large problem for national pre-election polls in 2008. Surveys that included
cell phones were no more accurate in forecasting the presidential race than
were those that relied only on landline samples. But warning signs appeared.
Several dual-frame pre-election polls in 2008 found that estimates based on
combined landline and cell phone samples tended to increase the estimates
of preference for Barack Obama by 2–3 percentage points, compared with the
landline sample alone. In a race tighter than the 2008 presidential election, even
a bias this small could cause some surveys to estimate—at least nominally, if
Cell-Phone-Only Voters in the 2008 Exit Poll                                  861


not necessarily with high statistical confidence—that the “wrong” candidate is
ahead.
   Additionally, to the surprise of many analysts, the potential noncoverage
bias is not concentrated solely among the younger voters. Among Election Day
voters in the national exit poll, the majority of the cell-only population was at
least 30 years old, and larger differences in voter preference by telephone status
were observed for those 30–39 than for those 18–29—a pattern also seen in
Pew Research Center pre-election surveys during the fall campaign. The vote
preference differences by phone status in the exit poll remained significant in
a multivariate analysis that controlled for most of the factors normally used
in weighting pre-election polls. Although this bias was modest in 2008, the
potential for bias may continue to grow as cell-only status becomes more
common among older age groups.
   Another issue is the declining ability of landline-based polls to describe
young voters with an acceptable level of reliability. As the cell-only population
has grown, landline samples are obtaining fewer younger adults, leaving many
polls with insufficient samples for such analyses. For example, only 9 percent
of respondents in Pew Research’s landline samples were ages 18–29 in 2008,
compared with a population parameter of 22 percent (Keeter 2009). Young
voters were the subject of intense interest and study in the 2008 cycle, given
their comparatively high levels of engagement in the campaign and the fact
that a higher percentage of them supported Obama than in any other age group.
Pre-election polls needed to be able to describe these young voters and explain
their opinions and motivation, but many were limited in their ability to do
this because of the coverage issue. The National Election Pool’s landline-
only pre-election polls were similarly constrained from describing how the
rapidly growing number of young—and older—people who voted early or
absentee compared with those interviewed in the Election Day national exit
poll. Including cell phone samples is an effective remedy for these problems,
although admittedly more costly than landline-only samples.
   Although not addressed in the current analysis, future survey research must
also contend with the “cell-mostly” population. There is, as yet, no evidence
of serious coverage bias related to voters who have a landline but rely mostly
on their cell phones. But a better understanding of this population is needed,
and the size and characteristics of this group should be monitored closely. If
necessary, steps should be taken to ensure such individuals are covered and
measured appropriately in future pre-election polling and surveys of actual
voters. The inclusion of cell phone samples can address this problem as well.
   Correctly forecasting the margin in an election is important for polling, but
it is not the only—or even the most important—contribution polls make to
the electoral process. Polls help citizens, the news media and policy makers
understand how voters are making their judgments and what considerations
animate them. Polls are critical for informing an understanding of why elections
turn out as they do, and thus have implications for how candidates, once elected,
862                                             Mokrzycki, Keeter, and Kennedy


may govern. For these reasons it is critical that polling fully represent the voting
public. Ultimately, polls that omit one-fifth or more of voters—voters who are
demonstrably different from those included in standard polling—are not likely
to have the credibility they need, even if they correctly predict the final election
tally.



Appendix: Methodology and Question Wording
The sample for the Election Day national exit poll used in this analysis was
drawn from the population of all people who voted in person on November 4,
2008, in the U.S. general elections. A probability sample of precincts was
drawn in each of the fifty states and District of Columbia from among all
voting locations in the state; a subsample of the state precincts was drawn
to create the national sample. Within each precinct, voters were selected at a
systematic interval throughout Election Day so all voters in that precinct had
an equal chance of being interviewed.
   Each state sample was stratified by geography and partisan voting history.
Precincts were selected with a probability proportionate to the number of
voters in each precinct, except that in some states, precincts with large minority
populations were sampled at a higher rate, and in New Hampshire several very
large precincts were sampled with certainty. In those cases, sample weighting
adjusted representation of precincts to their correct share of the total vote.
   Other post hoc weighting of the national exit poll included: weighting to
take into account the probability of selecting a precinct; adjustments based on
the number of respondents and interviewers’ tallies of the gender, estimated
age (into three bands) and race (black or nonblack) of voters selected in the
random sampling interval but who were missed or refused to cooperate; forcing
of survey estimates of each candidate’s vote share to actual vote received;
weighting to targets for age, race and gender; and adjustments based on size
of vote by region. (For data about the entire electorate, the national Election
Day exit poll was combined with National Election Pool telephone polling
conducted the week before the election to sample early and absentee voters;
this analysis focused only on those who voted in person on Election Day.)
   See National Election Pool (2005) for additional methodological detail. The
sampling and interviewing procedures did not substantively change in 2008.
Documentation for the 2008 study will be available at the Interuniversity Con-
sortium for Political and Social Research (ICPSR) and Roper Center for Public
Opinion Research.
   The completion rate for the 2008 Election Day national exit poll was 43.1
percent. This rate is the number of voters who completed the survey out of the
total number selected to take the survey, as tallied by the interviewers. AAPOR
Standard Definitions do not define how to calculate response rates for in-person
exit polling of individuals, only for in-person surveys of households.
Cell-Phone-Only Voters in the 2008 Exit Poll                             863


   Following is the exact wording of the main questions used in this analy-
sis. The complete 2008 national exit poll questionnaires are available in the
supplementary data online at http://poq.oxfordjournals.org/.
What type of telephone service is there in your home that you could use or
be reached on? (CHECK ONLY ONE)
  1. Both regular land-line and cell phone service
  2. Only regular land-line phone service
  3. Only cell phone service
  4. No telephone service at home
In today’s election for president, did you just vote for:
  1.   Barack Obama (Dem)
  2.   John McCain (Rep)
  9.   Did not vote
  0.   Other: Who?
To which age group do you belong?
  1.   18–24   4.   40–44    7.   60–64
  2.   25–29   5.   45–49    8.   65–74
  3.   30–39   6.   50–59    9.   75 or over
Are you:
  1.   White             4. Asian
  2.   Black             5. American Indian
  3.   Hispanic/Latino   6. Other
Are you of Hispanic or Latino descent?
  1.   Yes
  2.   No
   (Note: Those who answer “Yes” to the Hispanic/Latino ethnicity question
are counted as Hispanic/Latino in the race demographic even if they selected
something else there.)
Are you:
  1.   Male    2.   Female
2007 total family income:
  1.   Under $15,000         5. $75,000–$99,999
  2.   $15,000–$29,999       6. $100,000–$149,999
  3.   $30,000–$49,999       7. $150,000–$199,999
  4.   $50,000–$74,999       8. $200,000 or more
864                                            Mokrzycki, Keeter, and Kennedy


What was the last grade of school you completed?
  1.   Did not complete high school
  2.   High school graduate
  3.   Some college or associate degree
  4.   College graduate
  5.   Postgraduate study
No matter how you voted today, do you usually think of yourself as a:
  1.   Democrat
  2.   Republican
  3.   Independent
  4.   Something else
On most political matters, do you consider yourself:
  1.   Liberal
  2.   Moderate
  3.   Conservative
Is this the first year you have ever voted?
  1.   Yes
  2.   No
Did anyone call you or talk to you in person on behalf of either major
presidential campaign about coming out to vote?
  1.   Yes, for Barack Obama
  2.   Yes, for John McCain
  3.   Yes, for both Obama and McCain
  4.   No, I was not contacted



Acknowledgements
The 2004 exit poll data are available from a dataset that is archived at the Roper
Center for Public Opinion Research, University of Connecticut. The 2008 data
are from an NEP dataset that at this date remains unpublished; we obtained the
dataset as all three authors worked or consulted for NEP members during the
2008 elections.



Supplementary Data
Supplementary data are available online at http://poq.oxfordjournals.org/.
Cell-Phone-Only Voters in the 2008 Exit Poll                                                 865


References
American Association for Public Opinion Research. 2008. “Guidelines and Considerations for
  Survey Researchers When Planning and Conducting RDD and Other Telephone Surveys in the
  U.S. with Respondents Reached via Cell Phone Numbers.” Available at http://www.aapor.org/
  uploads/Final_AAPOR_Cell_Phone_TF_report_041208.pdf. Accessed October 7, 2009.
Associated Press Election Research. 2009. “2008 Turnout and Advance Voting.” Unpublished
  report dated April 21, 2009. Cited by permission.
Benford, Robert, Trevor Tompson, Barry Feinberg, Geoff Feinberg, Annie Weber, Nicole Speulda
  and Christopher Fleury. 2009. “Wireless and Wireline: Dual-frame Implications for Sample
  Design Decisions on Estimates, Weighting and Costs.” Paper presented at the 64th Annual
  Conference of the American Association for Public Opinion Research, Hollywood, FL, USA,
  May 2009.
Blumberg, Stephen J., and Julian V. Luke. 2009. “Wireless Substitution: Early Release of Estimates
  from the National Health Interview Survey, July–December 2008.” National Center for Health
  Statistics. Available at http://www.cdc.gov/nchs/nhis.htm. Accessed October 7, 2009.
Fahimi, Mansour, Dale Kulp and J. Michael Brick. 2008. “Bias in List-Assisted 100-Series RDD
  Sampling.” Survey Practice September 28, 2008.
Keeter, Scott. 2006. “The Impact of Cell Phone Noncoverage on Polling in the 2004 Presidential
  Election.” Public Opinion Quarterly 70:88–98.
———. 2009. “New Tricks for Old—and New—Dogs: Challenges and Opportunities Facing
  Communications Research.” March 3, 2009. Available at http://pewresearch.org/pubs/1138/
  survey-problems-new-techniques-communications-research. Accessed October 7, 2009.
Keeter, Scott, Michael Dimock, and Leah Christian. 2008a. “Cell Phones and the 2008 Vote:
  An Update.” Pew Research Center for the People & the Press. Sept. 23, 2008. Available at
  http://pewresearch.org/pubs/964/. Accessed October 7, 2009.
———. 2008b. “Calling Cell Phones In ‘08 Pre-Election Polls.” Pew Research Center for the
  People & the Press. Dec. 18, 2008. Available at http://people-press.org/reports/pdf/cell-phone-
  commentary.pdf. Accessed October 7, 2009.
Keeter, Scott, Courtney Kennedy, April Clark, Trevor Tompson, and Mike Mokrzycki. 2007.
  “What’s Missing from National Landline RDD Surveys? The Impact of the Growing Cell-Only
  Population,” Public Opinion Quarterly 71:772–92.
Keeter, Scott, Jocelyn Kiley, Leah Christian, and Michael Dimock. 2009. “Perils of Polling in
  Election ‘08.” Paper presented at the 64th Annual Conference of the American Association for
  Public Opinion Research, Hollywood, FL, USA, May 2009. Available at http://pewresearch.org/
  pubs/1266/polling-challenges-election-08-success-in-dealing-with. Accessed October 7, 2009.
Langer, Gary, Peyton Craighill, Patrick Moynihan, Jon Cohen, Jennifer Agiesta, and Dave Lambert.
  2009. “‘These Nutty Pollsters’: Methodological Issues in ABC News/Washington Post 2008 Pre-
  Election Polling.” Paper presented at the 64th Annual Conference of the American Association
  for Public Opinion Research, Hollywood, FL, USA, May 2009.
Martin, Elizabeth A., Michael W. Traugott, and Courtney Kennedy. 2005. “A Review and Proposal
  for a New Measure of Poll Accuracy.” Public Opinion Quarterly 69:342–67.
National Election Pool, Edison Media Research, and Mitosky International. 2005. National Election
  Pool Poll # 2004-NATELEC:National Election Day Exit Poll. Roper Center version. Somerville,
  NJ: Edison Media Research/New York, NY: Mitofsky International [producers], 2004. Storrs,
  CT: Roper Center for Public Opinion Research, University of Connecticut [distributor]. Available
  at http://www.ropercenter.uconn.edu/ipoll.html.

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:5
posted:8/1/2010
language:English
pages:21