The Effects of Semantics and Social Desirability in Correcting the Obama Muslim Myth
Brendan Nyhan*, Jason Reifler§, Christopher Edelman, William Passo, Ashley Banks, Emma Boston, Andrew Brown, Robert Carlson, KayAnne Gummersall, Elizabeth Hawkins, Lucy McKinstry, Jonathan Mikkelson, Emily Roesing, Vikram Srinivasan, Sarah Wakeman, Lindsey Wallace, and Rose Yan
May 15, 2009
*Brendan Nyhan, Duke University (bjn3@duke.edu) Jason Reifler, Georgia State University (poljar@langate.gsu.edu) Other co-authors were undergraduate students in Nyhan’s fall 2008 class at Duke.
§
Abstract In this paper, we address the question of how to counter political misperceptions, which are often difficult or impossible to eradicate. One explanation for this difficulty is that corrections frequently take the form of a negation (i.e. “Tom is not sick”), a construction that may fail to reduce the association between the subject and the concept being negated (Mayo et al. 2004). We apply this approach to the persistent rumor from the 2008 presidential campaign that Barack Obama is a Muslim, comparing the effectiveness of what we call a misperception negation (“I am not and never have been of the Muslim faith”) with what we call a corrective affirmation (“I am a Christian”), which should be more effective. As expected, we find that the misperception negation was ineffective. However, our hypothesis that the corrective affirmation would successfully reduce misperceptions was only supported when a non-white experimental administrator was present, suggesting a strong social desirability effect on the acceptance of corrective information. In addition, three-way interactions between the corrective affirmation, race of administrator, and party identification suggest that social desirability effects were more prevalent among Republicans. When nonwhite administrators were absent, the corrective affirmation not only failed to reduce Republican misperceptions but caused a backfire effect in which GOP identifiers became more likely to believe Obama is Muslim and less likely to believe he was being honest about his religion. We interpret this reaction as being driven by Obama’s embrace of Christianity, which may provoke cognitive dissonance among Republicans.
The proliferation of opinionated commentary on cable news channels, talk radio, and the Internet has renewed interest in the topic of political misinformation. However, political scientists have largely failed to distinguish between uninformed people who have no factual knowledge of an issue or controversy and misinformed people who confidently hold false or unsupported beliefs (Kuklinski et al. 2000: 792). As a result, while a vast literature exists documenting the existence of factual ignorance among voters, less evidence has been gathered on political misperceptions. In addition, little is known about how to counter misleading claims and correct political misperceptions, which are often difficult or impossible to eradicate (Kuklinski et al. 2000, Nyhan and Reifler 2008). One possible explanation for this difficulty is that corrections frequently take the form of a negation (i.e. “Tom is not a criminal”), a linguistic construction that can actually strengthen the association in a listener’s mind between the subject and the concept being negated (Mayo et al. 2004). In this case, a listener might subsequently associate Tom and criminality despite the presence of the negation in the statement (“not”). As a result, a more effective correction might be “Tom is an upstanding citizen,” which contradicts the misperception without reinforcing it. In this paper, we apply this approach to test different corrections of the false rumor that Barack Obama is a Muslim, which dogged him during his 2008 presidential campaign. Specifically, we report the results of an experiment conducted during the campaign testing the effectiveness of Obama’s use of what we call a misperception negation (“I am not and never have been of the Muslim faith”) and what we call a corrective affirmation (“I am a Christian who has belonged to the same church for almost
1
twenty years now”) against a control condition. The research above suggests that the corrective affirmation treatment should be more effective. As expected, we find that the misperception negation treatment failed to reduce misperceptions about Obama’s religion. However, two surprising findings emerged. First, despite administering the experiment on computers, the corrective affirmation treatment was only effective at reducing misperceptions in the presence of non-white experimental administrators, suggesting a strong social desirability effect. In addition, three-way interactions between the treatment, race of administrator, and party identification also indicate that these social desirability effects were more prevalent among Republicans – a finding that we validate using a novel measure of implicit associations between the presidential candidates and religion. By contrast, when only white administrators were present, we find that the corrective affirmation caused a backfire effect in which GOP identifiers were more likely to believe that Obama is a Muslim and less likely to believe that he was being honest about his religion. In the discussion section, we interpret these divergent reactions as being driven by Obama’s embrace of Christianity in the corrective affirmation condition, which seems to have provoked counterarguing among opposing partisans when nonwhite administrators were not present.
Theoretical approach Previous research Political science still lacks significant knowledge about how to correct political misperceptions. Only a handful of studies have been conducted that test the effects of providing citizens with correct information. Kuklinski et al. (2000; study 1), Gilens 2
(2001), and Sides and Citrin (2007) all provided participants in survey experiments with correct factual information about an issue and then asked about their policy preferences on the issue. Importantly, however, none of these manipulations made it clear that subjects might have false beliefs or that a misperception existed among some members of the public. Of the three studies, only Gilens found that correct factual information changed participants’ reported issue preferences (for crime and foreign aid). In particular, only two experiments in political science have been attempted to directly correct political misperceptions among the public.1 Study two of Kuklinski et al. (2000) asked participants to guess the portion of the federal budget that is spent on welfare (most participants overestimated the true value). Some were then told the correct answer, which caused them to increase their support for social welfare spending. However, the practical application of this finding is limited – voters are rarely asked for their personal beliefs about a factual question and then authoritatively refuted. Nyhan and Reifler (2008) extended this line of research by assessing the effectiveness of corrective information embedded in a mock news article containing a potentially misleading statement by a political figure. Disappointingly, they found that corrections typically failed to reduce misperceptions among the ideological group most likely to hold them. Even worse, Nyhan and Reifler actually found two cases of a “backfire” effect in which corrections actually increased misperceptions among the targeted ideological group.
There are, however, related literatures on rebutting rumors (e.g., Bordia, DiFonzo, and Schulz 2000, Bordia, DiFonzo, Haines, and Chaseling 2005; DiFonzo and Bordia 2006), inoculating people against persuasion (e.g., Wood 2007, Cialdini et al. n.d.), and refuting misleading advertising (e.g., Johar 1996, Petrova et al. n.d.).
1
3
The (in)effectiveness of negations? Building on these studies, we seek to investigate whether the way in which a misperception is corrected affects a citizen’s decision to accept or reject the correction. This outcome may hinge on characteristics of the correction itself – a notion that is supported by recent research on the processing of linguistic negations. Our study draws on these findings to introduce and test two types of corrections. Specifically, negations have been found to be ineffective for unipolar descriptions (Mayo et al. 2004). These are words that do not have clear opposites, such as the adjectives “romantic” and “talented.” (Bipolar descriptions, on the other hand, have obvious opposites, like “strong/weak” and “rich/poor.”) Mayo et al. proposed that when people hear a negation, people process the “core supposition” first and then negate it by applying a discounting tag such as “not.”2 Thus, when an individual hears the negation “Tom is not a criminal,” he or she would first process the core supposition “Tom is a criminal” and then negate it.3 As a result, the statement might unintentionally activate associations between Tom and criminality, undermining the effect of the negation. Mayo et al. find experimental support for this claim when testing negations of unipolar descriptions for which a well-defined opposing schema does not exist (such as being a criminal).4 By contrast, an obvious opposite schema exists for bipolar descriptions (“Tom
It should be noted that Mayo et al.’s study was conducted in Hebrew, which exclusively uses the word “no” for negations rather than semantic prefixes or suffixes. They note that this should make processing negations easier. See Just and Carpenter (1976) and Grant, Malaviya, and Sternthal (2004) for additional evidence that the core supposition of a negation is processed first. By contrast, the opposing “fusion” model assumes that negations result in the direct activation of “a negationcongruent schema” (Mayo et al. 2004: 435). Thus, “Tom is not sick” would be processed and encoded as “Tom is healthy” without creating any association between Tom and sickness.
4 3
!
4
is sick”), which Mayo et al. find can be negated more effectively (“Tom is healthy”).5 We apply this theoretical approach to examine one of the most troubling misperceptions from the 2008 Presidential election—the persistent and widespread belief that Barack Obama is or had been a Muslim. For instance, aA Pew Research Center poll conducted in September 2008 found that approximately 13% of Americans believed Obama was a Muslim (including 17% of Republicans) and 16% said they “had heard different things” about his religion (2008).6 These misperceptions persisted into Obama’s term – a March 2009 Pew poll found virtually identical results (2009). Based on the research described above, we argue that one reason that misperceptions about Obama’s religion may be difficult to correct is the fact that Muslim is a unipolar descriptor. While there are several conceivable opposites for the term, such as Christian or atheist, none of them is a clear conceptual opposite. Group identifiers such as Muslim do not have the same clearly defined conceptual opposites as the bipolar adjectives used by Mayo et al. (tidy/messy, rich/poor, warm/cold).7 As such, negations are likely to be ineffective in combating the Obama/Muslim misperception. To test this proposition, we present an experiment below contrasting what we call a misperception negation (Obama “[is] not and [has] never been of the Muslim faith”) and a corrective affirmation that contains the truth (Obama “[is] a Christian”) with a control condition. Our goal is to determine whether corrective affirmations can dispel the
Similarly, Gawronski et al. (2008) found that training subjects to negate stereotypes actually enhanced their stereotypic associations. By contrast, repeatedly affirming counter-stereotypic associations was found to successfully reduce the activation of negative stereotypes. Similarly, a Newsweek poll released on July 11, 2008 found that 12% of respondents believe Obama used a Koran for swearing into the U.S. Senate. 26% believe he was raised as a Muslim, and an astounding 39% believe he attended Islamic school as a child growing up in Indonesia (Newsweek 2008). Mayo et al. used a pre-test to determine which descriptors were unipolar and which were bipolar. Subjects were given a list of single-word descriptors and asked to write down the opposite of each descriptor. Each descriptor was categorized as unipolar if more than 80% of respondents could not think of an opposite or used a negation of the original descriptor. We did not perform such a test for the Muslim descriptor.
# " !
5
misperception about Obama’s religion and if they are more effective than misperception negations in doing so. As described below, Obama responded to this rumor using both the corrective affirmation and misperception negation constructions, which allowed us to use actual video of him speaking as manipulations. (See below for more detail on the experimental design, including a description of the video that was used as a control.) However, the application of this theoretical framework may be complicated by the realities of contemporary politics, particularly in the context of a presidential campaign. The Mayo et al. experiments concerned descriptions of a hypothetical person, whereas our research concerns political figures and issues on which participants have pre-existing attitudes and beliefs. Numerous studies have found that people’s reactions to new information are frequently influenced by their prior beliefs (e.g. Edwards and Smith 1996, Taber and Lodge 2006, Taber, Cann and Kuscova forthcoming). In some cases, receiving new information that contradicts participants’ political views or factual beliefs can backfire and strengthen participants’ previous beliefs (e.g. Redlawsk 2002, Nyhan and Reifler 2008). Given that the presidential nominee of a major party is making the correction during an election campaign, we expect subjects’ reactions to our correction manipulations to be moderated by their partisanship.8 We thus have two main hypotheses about the effect of corrections on misperceptions:
8 This is a similar approach to Nyhan and Reifler (2008), who find that participants’ reactions to their correction manipulations were moderated by ideology.
6
Hypothesis 1: Corrective affirmation Corrective affirmations will be more effective than negations in correcting the unipolar misperception that Obama is or was a Muslim.
Hypothesis 2: Party affiliation (interaction) Party affiliation will moderate the effectiveness of efforts to correct the Obama Muslim misperception
To operationalize Hypothesis 2, we included a dummy variable for self-identification as a Republican and an interaction between GOP affiliation and the experimental treatments. We also included variables controlling for political knowledge and self-identification as black or African American (hereafter black) to increase the precision of our estimates.
Race of administrator and social desirability An extensive literature examines how social desirability concerns can influence the way survey or experimental participants answer questions, particularly when the questions relate to racial issues or the interviewer and respondent are of different races. In particular, most studies have found that white respondents tend to “give more liberal or pro-black opinions when the interviewer is black” (Hatchett and Schuman 1975/1976: 525) – an effect that is frequently interpreted as an attempt to express opinions that conform to the interviewer’s perceived expectations or to societal norms. As a result, great pains were taken to minimize social desirability effects in the present research. Among other things, the experiment was conducted on computers, which have been 7
found to increase self-reporting of socially sensitive behaviors (e.g. Erdman, Klein and Greist 1983; Baker, Bradburn, and Johnson 1994; O’Reilly et al. 1994).9 Despite these efforts, however, we discovered an unanticipated race of administrator effect on subjects’ responses that is described in the results section below. (Since the administrators did not directly interview participants, we prefer the term “race of administrator” to “race of interviewer.”) Specifically, participant responses to the correction treatments varied depending on whether one or more non-white experimental administrators were present. This variation does not appear to be the result of differences in participant characteristics – it seems quite unlikely that the moderate differences between groups by race of administrator can account for the substantial variation in treatment effects observed in the data.10 The finding that the presence of non-white administrators11 affected the correction of misperceptions is consistent with previous studies of the effects of race on the survey response, which have found that even the “mere presence” of a person of a different race can induce more socially desirable answers (Krysan 2003). In particular, one could imagine that a question about the religion of a minority presidential candidate would be particularly sensitive – participants may have been unwilling to report their beliefs accurately if they felt that minority administrators would view them as intolerant.
We also developed a new measure of implicit associations between the presidential candidates and religion (described further below) in an attempt to circumvent social desirability problems.
10 Specifically, individuals who participated in the experiment when nonwhite administrators were present were somewhat less likely to describe themselves as black or African American (43% for white administrators versus 55% for nonwhite; !2(1) = 3.44, p < .10) and had higher political knowledge scores (white administrators: M = 0.292, SE = 0.026; nonwhite: M = 0.385, SE = 0.037; p < .05). No statistically significant differences were observed between groups for partisanship or GOP self-identification.
!
In addition to being physically present, administrators gave participants instructions on how to complete the experiment and answered participants’ questions while they were taking it. Sometimes the participants’ computer screens were partially visible to administrators due to the constraints of the public locations at which the experiment was conducted (see below for details on the administration of the experiment).
""
8
While experimental administration duties were not randomly assigned, it is still important to account for these effects in our data analysis. As such, we have three related hypotheses about how the race of administrator effect will modify the hypotheses above:
Hypothesis 3: Race of administrator (direct effect) Race of administrator will have a direct effect on participants’ reported perceptions of Obama’s religion.
Hypothesis 4: Race of administrator (treatment interaction) Race of administrator will moderate the effects of the experimental manipulations that are intended to correct misperceptions about Obama’s religion.
Hypothesis 5: Partisan social desirability (three-way interaction) The effect of partisanship on the correction treatment effects will be moderated by the presence or absence of a non-white administrator.
After splitting our data and estimating separate models by race of administrator, we operationalize hypotheses 3-5 in the results section below by including a dummy variable for the presence of at least one non-white experimental administrator as well as two- and three-way interactions with the treatments and the GOP variable.12
One might speculate that the presence of nonwhite interviewers would have different effects depending on the racial background of the participant. However, additional analyses (which are omitted but available upon request) show that results are qualitatively similar when we exclude black respondents from the sample (which dramatically reduces sample sizes). They are therefore retained in all subsequent analyses.
!"
9
Experimental design and sample The present experiment contrasts the effects of two different strategies for correcting misperceptions. Specifically, we used a between-subjects design in which participants were randomly assigned to one of three experimental conditions – two corrections and a control group.13 The use of an experimental design mitigates possible concerns about studying real-world political issues; despite participants’ differing preexisting beliefs and opinions, the randomization process allows us to be confident that the resulting differences in our dependent variables (measures of participants’ beliefs about Obama’s religion) were caused by the experimental treatments. Participants in the experiment were recruited by experimental administrators who set up tables in public areas of a university hospital in the southeast.14 For our experiment, there were fifteen administrators, twelve of whom are white and three of whom are non-white (one is African American, one is Indian-American, and one is Asian American). The administrators typically worked in teams of two or three. After consenting to participate, participants were provided with instructions by the administrator and given a laptop computer and headphones to take the experiment. No further interaction with the administrators occurred during the experiment unless a participant approached an administrator to ask a question.
13 The experiment also included an alternate correction treatment unrelated to the semantic processing argument we focus on in this paper. This alternate treatment did not have significant effects. Because the effects were non-significant and because the alternate treatment was unrelated to the arguments that are discussed here, we exclude data from subjects in the alternate treatment condition. The primary experimental manipulation discussed in this paper thus includes two experimental conditions and one control condition. However, it is technically a 3x2 experiment because we also independently randomized the order of the third and fifth blocks of the IAT per Greenwald, McGhee, and Schwartz (1998). Full details of this manipulation, which do not affect the results, are described in Appendix B. 14 These tables were set up in lobby and hallway areas. The IRB protocol prohibited recruiting from clinical areas of the hospital or patient waiting rooms.
10
The experiment was programmed in the experimental psychology software programs MediaLab and DirecRT, which made it possible to show actual video clips of the 2008 presidential candidates to participants. After answering preliminary questions, all participants viewed a series of video clips of both Senators McCain and Obama describing various personal matters that were intended to disguise the purpose of the experiment. Participants were then randomly assigned to one of three experimental conditions (misperception negation, corrective affirmation, or the control group). Participants in the misperception negation condition viewed a video in which Obama negated the perception that he is a Muslim by saying, “I'm unequivocal about this -- I am not and never have been of the Muslim faith.” Participants in the corrective affirmation condition viewed a video of Obama instead affirming his Christian beliefs by saying “I am a Christian who has belonged to the same church for almost twenty years now.” Finally, control group members viewed a clip in which Obama discussed his decision to run for president. After each video, a textual quotation from the clip was presented on screen. (See Appendix A for transcripts and sources.) Each clip was 10-11 seconds long. After watching these videos, participants completed a series of questions about their perceptions of each candidate’s religious preference as well as an implicit association test measuring the extent to which the participant tended to associate Obama with Islam (discussed further below). Participants received five dollars in compensation. The participants were a mix of patients, patients’ family members, general visitors and hospital employees.15 A total of 230 participants were recruited. African-American participants comprised 51% of our sample, while 34% identified themselves as white.
15 We initially allowed university hospital employees to take part in the study but excluded them after three days due to the concern that some might participate repeatedly to receive multiple incentive payments.
11
45% of participants considered themselves to be evangelical or born-again Christians while 3% identified themselves as Muslim. Politically, 15% considered themselves to be Republicans, 21% identified themselves as independents, and 64% considered themselves to be Democrats.16 The highest level of educational attainment achieved by participants was unusually diverse: 30% had a high school degree or less, 25% attended some college, 25% graduated from college, and 20% held a post-graduate degree.
Measures Dependent variables We measured participants’ perceptions of Obama’s religion using both explicit and implicit measures. Our three measures of misperceptions were ReligionChoice, ExMuslim, and Muslim, each of which was coded such that higher values indicate stronger beliefs that Obama is or was a Muslim. ReligionChoice was created from a question asking participants “Do you happen to know what Barack Obama’s religion is?” Participants who chose “Christian” or “Muslim” were then asked a branching followup question: “How devout of a Christian/Muslim do you think Obama is?” Responses were recoded into a five-point scale.17 Muslim and Ex-Muslim are seven-point Likert agreedisagree scales asking whether participants believed Obama “is a Muslim” and “used to be a Muslim,” respectively.18 To measure participants’ subjective reactions to the
16 Not surprisingly, African American participants largely self-identified as Democrats (68%). The partisan distribution of the non-black sample was somewhat more balanced: 53% Democrats, 23% independent, and 20% Republican. 17 “Extremely/very devout Muslim” (5), “Somewhat/not too/not at all devout Muslim” (4), “Don’t know” (3), “Somewhat/not too/not at all devout Christian” (2), “Extremely/very devout Christian” (1). Three responses were dropped due to response times of less than one second (indicating that subjects did not read the question) and thirteen were dropped because they selected religions other than Christian and Muslim. !"
Six observations for Muslim and five observations for Ex-Muslim were dropped due to response times of less than one second, leaving 224 and 225 valid observations, respectively. Don’t know responses were recoded as 4, the middle response option (see Appendix A for exact wording).
12
corrections, we also asked about participants’ perceptions of Obama’s honesty about his religion (Honest) and feelings toward him (Feelings). The variables were measured on a 1-7 scale and a 0-10 scale, respectively, where higher values represented more positive perceptions. (See Appendix A for details.)
The candidate religion IAT In order to obtain a measure of respondent associations toward Obama’s religion that is less likely to suffer from social desirability bias, we created ReligionIAT, which measures participant’s implicit attitudes about the relationship between Obama and the Muslim religion using a version of the Implicit Association Test (IAT). The IAT was devised to measure implicit attitudes that “are under the control of automatically activated evaluation, without the performer’s awareness of that causation” (Greenwald et al. 1998: 1464). It measures implicit associations based on performance speed in the joint classification of target concepts and attributes: the faster the participant completes the classification, the easier the pairing is, and thus the stronger the relation is in the participant’s mind. Conversely, the longer it takes a participant to complete the classification, the weaker the relation is (Greenwald et al. 1998: 1466).19 Though the most well-known IAT studies measure the association between race and positive or negative attributes, the test can also be used to measure stereotypic associations between individuals or groups and categories – for instance, between the male gender and the disciplines of math and science (Nosek, Banaji, and Greenwald 2004).
For instance, in Experiment 1 of Greenwald et al.’s study, the target concepts were “flowers” and “insects” and the attributes used were “pleasant” and “unpleasant.” After first classifying target words (e.g. daisy, fly, tulip) as flowers or insects and attribute words (e.g. lucky, poison, grief) as either “pleasant” or “unpleasant,” subjects were asked to alternately classify stimuli from both groups. The pairing of the target and attribute was then reversed and the process repeated. The IAT compares response times based on the pairing of target and attribute. In this case, the pairings of flower/unpleasant and insect/pleasant might slow reaction times relative to flower/pleasant and insect/unpleasant.
!"
13
In this case, we measured participant’s implicit associations about Obama’s religion because their responses to explicit measures may be biased by self-presentational motivations (Kim and Greenwald 1998). Since the IAT is an inherently relative measure (i.e. it compares the strength of implicit associations), we constructed a candidate religion IAT comparing Christian and Muslim associations for both Barack Obama and his presidential rival John McCain. The IAT portion of the experiment was constructed in accordance with the guidelines of Greenwald, Nosek, and Banaji (2003, 2005). (See Appendix B for full details.) The stimuli we used in this experiment included similarlooking headshots of John McCain and Barack Obama as well as words that were stereotypically either Christian (Jesus, Bible, Easter, baptism, crucifix, and church) or Muslim (Allah, Ramadan, Koran, Islam, Mecca, and mosque).20 We followed the recommendations of Nosek et al. (2003) in excluding unreliable data from our analysis and then computed participants’ IAT scores according to the recommendations of Greenwald et al. (2003).21 While there is some evidence that contextual factors can influence IAT scores (e.g. Deutsch and Gawronski 2009), responses to the IAT are far less susceptible to social desirability bias than explicit measures. As such, ReligionIAT provide a baseline measure of associations toward Obama’s religion that we will later contrast with subjects’ self-reported beliefs. As with the explicit dependent variables
!"
We tried to select words that we felt best represented the categories in question, could not be easily misconstrued,
and were approximately equal in length and number of syllables (Nosek et al. 2005). 21 First, trials in which participants took longer than 10,000 milliseconds to respond to an item were eliminated. We made this correction because it is likely that something unrelated to the experiment distracted the participant. Second, all IAT results were dropped for participants who had latency times of less than 300 milliseconds on more than 10% of the trials. This was done to exclude participants who simply pressed one or both buttons quickly rather than reacting to stimuli. In addition, the IAT data of 31 participants was lost due to computer malfunctions and data recording errors.
14
above, ReligionIAT was constructed so that higher scores represent stronger implicit associations between Obama and Islam.22
Results Descriptive statistics For the results presented below to be useful, it is important that misperceptions about Obama’s religion exist in the experimental sample. In our case, we can contrast the ReligionChoice dependent variable described above with the Sept. 14, 2008 Pew poll question on which it was based. Levels of misperceptions were consistent in the two samples – eleven percent of control respondents said Obama is a Muslim, compared with twelve percent in the Pew poll – suggesting that our sample had broadly comparable levels of misperceptions as the general public. Among subgroups of interest, identification of Obama as a Muslim was greater among Republicans (Pew: 17 percent; experimental controls: 25 percent) and lower among black respondents (Pew: four percent; experimental controls: eight percent) in both samples. Before turning to statistical models, Figure 1 introduces the data by plotting the distributions of the four variables measuring misperceptions about Obama’s religion across all experimental conditions – ReligionChoice, Muslim, Ex-Muslim, and ReligionIAT. (Each is coded so that higher values represent stronger misperceptions.)
[Figure 1]
!!
It is important to note that, unlike the explicit dependent variables, ReligionIAT is not a linear measure of misperceptions. There is no definitive answer to the question of which presidential candidate is “more” Christian (and thus a negative score is not necessarily the “correct” answer). However, positive scores can be interpreted as reflecting implicit misperceptions. "
15
We observe that approximately half of participants strongly disagree with the misperception on the three explicit dependent variables, responding that Obama is a practicing Christian (44%) and strongly disagreeing with the claims that Obama is or was a Muslim (53% and 46%, respectively). Also, approximately one quarter of respondents respond on each question that they either don’t know Obama’s religion or “neither agree nor disagree” with the claims about him being Muslim, which places them in the middle category for ReligionChoice (19%), Muslim (26%), and Ex-Muslim (29%). Finally, a smaller group endorsed the misperception that Obama is or was a Muslim, which places them above the midpoint on these dependent variables. The size of this group ranged from 6% for ReligionChoice to 20% for Ex-Muslim (it was 15% for Muslim). In the case of the ReligionIAT variable, a positive score indicates that the participant more strongly associated Obama with Muslim and McCain with Christian than Obama with Christian and McCain with Muslim. Conversely, a negative score indicates that the participant more strongly associated Obama with Christian and McCain with Muslim. The average ReligionIAT score for all participants combined was slightly positive,23 meaning that participants were slightly more likely to associate Barack Obama with Muslim (and John McCain with Christian) than Barack Obama with Christian (and John McCain with Muslim).24 Scores were approximately normally distributed (M = 0.068, SD = 0.578). There were, however, clear differences between subgroups that correspond to our expectations and help to validate the measure. For instance, white
The actual numerical value is meaningless except as a basis of comparison because it is simply the result of dividing the average latency by the standard deviation. The slightly positive average ReligionIAT score means participants took slightly longer to associate Obama with Christian than with Muslim.
24 Hereafter, this tendency will just be described as associating Obama with Muslim, and the opposite tendency will be described as associating Obama with Christian. 23
16
participants were more likely to associate Obama with Muslim (M = 0.246, SE = 0.067) than were black participants (M = -0.056, SE = 0.056) in a t-test (p < .01).25 Similarly, Republicans were more likely to associate Obama with Muslim (M = 0.467, SE = 0.104) than were non-Republicans (M = 0.002, SE = 0.043; p < .01). Participants’ ReligionIAT scores were also positively correlated with their explicit beliefs about Obama’s religion. For instance, the ReligionChoice dependent variable asked participants what they believed was Obama’s religion. Those who responded “Muslim” were somewhat more likely to associate Obama with Muslim in the IAT (M = 0.457, SE = 0.216) than were those who responded “Christian” (M = -0.025, SE = 0.045; p < .10). Similarly, subjects who indicated greater agreement with the statements “Barack Obama is a Muslim” (Muslim) and “Barack Obama used to be a Muslim” (Ex-Muslim) associated Obama more with Muslim in the IAT (r = 0.16, p < 0.05; r = 0.12, p < .10, respectively).
Misperceptions by race of administrator Having established the validity of our measures, we now use OLS statistical models to estimate the effects of the correction treatments on participant (mis)perceptions about Barack Obama’s religion. The primary explanatory variables of interest are the two correction treatments: the misperception negation (“I am not and never have been of the Muslim faith”) and the corrective affirmation (“I am a Christian”). (The control condition is the baseline used for comparison.) Following Hypothesis 2, we include an indicator variable for GOP self-identification and interactions between each treatment and the GOP indicator. Finally, we include controls for political knowledge and for black participants.
25 p-values for all t-tests are two-tailed and reflect an assumption of unequal variance between groups (approximate degrees of freedom were calculated using Satterthwaite’s approximation).
17
(Due to the number of models reported and the difficulty of interpreting control variables in a convenience sample, we focus on treatment variables in the discussion below.) To illustrate the apparent social desirability effect, Table 1 compares model results for the three explicit dependent variables broken out by race of administrator.26
[Table 1]
Given the presence of interaction terms, the treatment variable coefficients can be interpreted as the effect of the treatments when the other term (GOP) is equal to 0 (Brambor, Clark, and Golder 2006). In other words, the coefficients in the first two rows of the table represent the effect of the treatments on non-Republicans. They indicate that the corrective affirmation treatment was effective in reducing misperceptions for nonRepublicans across all three explicit dependent variables, but only when non-white experimental administrators were present – the effect seems to disappear when administrators were exclusively white. No such pattern is found for the misperception negation treatment, which is only significant once at the p < .10 level. In addition, we observe significant differences in Republican responses depending on the presence of non-white administrators. The coefficient for GOP self-identification is positive and statistically significant for all three dependent variables when (and only when) non-white administrators were present. Finally, the interaction between the corrective affirmation treatment and GOP reverses sign depending on the race of administrator, suggesting possible three-way interactions. In particular, the interaction
26
Results are nearly identical for ordered probit (available upon request).
18
coefficient for Muslim is 2.58 on a seven-point scale (95% CI: 0.27, 4.89), suggesting that a massive backfire effect took place among Republicans in response to the corrective affirmation treatment. We examine this finding more extensively in the next section.
Interactive models of misperceptions To assess whether the presence of non-white administrators affected the treatment effects, we ran the full set of two- and three-way interactions between the correction treatments, party identification, and the presence of a non-white administrator (presented in Table 2).
[Table 2]
Several two-way interaction terms involving the corrective affirmation treatment are found to be statistically significant across the three explicit dependent variables (corrective affirmation x GOP for Muslim [p < .05] and non-white x corrective affirmation for Muslim and Ex-Muslim [p < .10 and p < .05, respectively]). In addition, we find a statistically significant three-way interaction for two of the three explicit dependent variables, which indicates that the two-way interaction between corrective affirmation and GOP affiliation was itself moderated by the presence or absence of a nonwhite administrator (p < .05 for ReligionChoice and p < .10 for Muslim). Since it is difficult to interpret three-way interaction models, we present a series of predicted effect graphs to illuminate these findings below. (By contrast, none of the terms involving misperception negation achieve statistical significance, nor do any of the predicted effects
19
of the manipulation. They are thus omitted but are available upon request. We consider explanations for this discrepancy in the discussion section below.) Figures 2-4 illustrate the estimated effects of the corrective affirmation treatment by subgroup for ReligionChoice, Muslim, and Ex-Muslim, respectively.27
[Figures 2-4]
Given the two- and three-way interaction models estimated above, we present predicted effects for the corrective affirmation treatment for both Republicans and non-Republicans with both white and non-white experimental administrators. Predicted effects for Republicans appear in the left panel and those for non-Republicans appear in the right panel. Predicted effects for subjects with white administrators are represented as a solid line and those with non-white administrators are represented as a dashed line. Our first finding is that the corrective affirmation made participants less likely to believe that Obama is or was Muslim for each explicit dependent variable when a nonwhite administrator was present. This result is illustrated by the negative slopes on the dashed lines in each panel of each figure for Figures 2-4, which indicate decreasing levels of belief that Obama is or was a Muslim. The changes in predicted values are substantively large and frequently statistically significant across all the explicit dependent variables for both non-Republicans (ReligionChoice: !Y = -0.85, p < .01; Muslim: !Y = -
Predicted values (and p-values for predicted changes) were calculated using Clarify (King, Tomz, and Wittenberg 2000; Tomz, Wittenberg, and King 2003). Knowledge was set to the sample mean and the indicator variable for black participants was set to 0. Marginal effects were also calculated following Brambor, Clark, and Golder (2006) – results were substantively identical (available upon request).
27
20
0.95, p < .10; Ex-Muslim: !Y = -1.10, p < .05) and Republicans (ReligionChoice: !Y = 2.00, p < .01; Muslim: !Y = -1.50, n.s.; Ex-Muslim: !Y = -1.57, n.s.). In addition, we also found that the corrective affirmation treatment had a dramatically different effect among Republican participants depending on whether a nonwhite administrator was present. When only white administrators were present (the solid line in the left panels of Figures 2-4), the predicted effect of the corrective affirmation was consistently positive, suggesting that it actually increased misperceptions among Republicans (ReligionChoice: !Y = 0.61, n.s.; Muslim: !Y = 2.81, p < .01; Ex-Muslim: !Y = 1.11, n.s.). In particular, perceptions that Obama is a Muslim increased dramatically. By contrast, when a non-white administrator was present (the dashed line in the left panels of Figures 2-4), the corrective affirmation caused Republican misperceptions to decrease as noted above (ReligionChoice: !Y = -2.00, p < .01; Muslim: !Y = -1.50, n.s.; Ex-Muslim: !Y = -1.57, n.s.). This difference in treatment effects accounts for the significant three-way interaction terms in Table 2. Finally, the corrective affirmation treatment simply had no statistically significant effect for non-Republican participants who took the experiment in the presence of only white administrators (the solid line in the right panels of Figures 2-4).28 Corroborating evidence for the results above is provided when we estimate identical two- and three-way interaction models for perceptions of Obama’s honesty about his religion (Honest) and feelings toward him on a feeling thermometer (Feelings). The results of these models are presented in Table 3.
28
ReligionChoice: !Y = -0.25, n.s.; Muslim: !Y = 0.16, n.s.; Ex-Muslim: !Y = 0.40, n.s.
21
[Table 3]
We again find significant three-way interactions among the corrective affirmation treatment, GOP self-identification, and race of administrator for both Honest and Feelings (p < .05 and p < .10, respectively). To illustrate these effects, we plot predicted values in Figures 5 and 6 using an identical procedure to Figures 2-4.
[Figures 5-6]
The predicted effects for both variables are consistent with the account provided above. When non-white administrators are present, Republicans report substantively large increases in perceptions of Obama’s honesty about his religion and positive feelings toward him, though neither increase is statistically significant (Honest: !Y = 0.88, n.s.; Feelings: !Y = 2.66, n.s.). By contrast, when non-white administrators are absent, Republican perceptions of Obama’s honesty about his religion and feelings toward him decline dramatically (Honest: !Y = -3.13, p < .01; Feelings: !Y = -2.34, n.s.). This discrepancy accounts for the statistically significant three-way interaction terms in Table 3. As before, this pattern is not observed among non-Republicans – treatment effects are not statistically significant for any relevant participant/administrator grouping.29 We can also shed light on the pattern of responses described above using ReligionIAT. As expected, the experimental treatments almost never had statistically
Non-Republican, nonwhite (Honest: !Y = 0.45, n.s.; Feelings: !Y = 0.23, n.s.); non-Republican, white (Honest: !Y = -0.33, n.s.; Feelings: !Y = -0.46, n.s.).
29
22
significant effects on this variable, which measures automatic associations that are largely outside of conscious control (results available upon request). We therefore use ReligionIAT to highlight the divergent relationships between participants’ explicit responses and their implicit associations in the presence of non-white interviewers, particularly when given the corrective affirmation treatment. Specifically, we regress each dependent variable (ReligionChoice, Muslim, and Ex-Muslim) on ReligionIAT, the GOP indicator variable, and an interaction term (along with controls for knowledge and African American participants). This model is run separately for four sub-samples: white administrators, control condition; white administrators, corrective affirmation condition; non-white administrators, control condition; and non-white administrators, corrective affirmation condition. Our results are provided in Tables 4a-4c, which correspond to ReligionChoice, Muslim, and Ex-Muslim, respectively.30
[Table 4]
The tables reveal the expected divergence between explicit and implicit measures for two of the three dependent variables. In the absence of social desirability bias, we expect a positive relationship between ReligionIAT and the explicit dependent variables since all four are coded so that higher values reflect a greater association between Obama and Islam. However, for Muslim and Ex-Muslim (though not ReligionChoice), there is a statistically significant negative interaction between ReligionIAT and the GOP indicator
!"
While it might be desirable to directly test these differences in a single model, doing so would require estimating a model with a four-way interaction and fourteen constituent terms (Brambor, Clark, and Golder 2006). We do not have enough data to estimate such a model (nor is it clear how useful such a model would be given the conceptual complexity of a four-way interaction).
23
variable only for participants who received the corrective affirmation treatment with nonwhite experimental administrators present (p < .05 in both cases).31 The net effect is negative and statistically significant (p < .10 for Muslim and p < .01 for Ex-Muslim), indicating that stronger implicit associations between Obama and Islam were associated with lower explicit responses from GOP identifiers on the relevant dependent variables. In other words, Republicans who strongly associated Obama with Islam gave explicit responses indicating exactly the opposite (and the converse). While we do cannot derive conclusive inferences from the relationship between two observational variables, the differences in the patterns of association between the ReligionIAT measure and the explicit dependent variables support the interpretation of Table 2 that is developed above.
Discussion The estimated treatment effects for the corrective affirmation manipulation are summarized in Table 5.32 (As previously noted, we find that the misperception negation had no significant effects regardless of administrator race. It is therefore excluded.)
[Table 5]
An examination of treatment effects for non-Republicans in the second and third columns reveals that consistent differences by race of administrator. When non-white
Given the small sample sizes, we cannot reject the null hypothesis of no difference in coefficients across the subsamples. However, the patterns of statistical significance are consistent with the account provided above. Under a stricter standard, we can reject the null hypothesis that the treatment effects (including interaction terms where appropriate) are equal for non-Republicans with white administrators for ReligionChoice and Ex-Muslim (p < .10 and p < .05, respectively) and for non-Republican and Republican participants with nonwhite administrators on ReligionChoice (p < .05 and p < .10, respectively). The corrective affirmation reduced misperceptions more than the misperception negation in both cases.
!# !"
24
administrators were present, the corrective affirmation treatment caused a statistically significant decrease in misperceptions on all three dependent variables measuring misperceptions about Obama’s religion. But when non-white administrators were absent, the corrective affirmation had no statistically significant effects. The fourth and fifth columns highlight an even more interesting interaction between the corrective affirmation and race of administrator among Republican identifiers. The corrective affirmation seems to be modestly effective among Republican subjects in the presence of a non-white administrator (all five dependent variables are signed such that the treatment improved perceptions of Obama, though only one reaches conventional levels of statistical significance). However, when only white administrators are present with Republican subjects, the corrective affirmation failed to reduce misperceptions and caused Republicans to be more likely to believe Obama was a Muslim and less likely to believe he was being honest about his religion. Similar misperception-increasing backfire effects were found for Republicans on ReligionChoice and Ex-Muslim, though they did not reach conventional levels of statistical significance. We interpret these findings to mean that the presence of non-white experimental administrators created social desirability conditions that boosted the power of the corrective affirmation, which was otherwise ineffective. This interpretation is consistent with the finding that GOP respondents’ ReligionIAT scores and their responses on Muslim and Ex-Muslim diverged when receiving the corrective affirmation treatment with non-white interviewers present. In the absence of these social desirability conditions, GOP identifiers who received the treatment actually moved in a misperception-increasing direction on Muslim, perceived Obama as less honest about his religion, and felt less 25
warmly toward him. This account corresponds to Nyhan and Reifler’s finding of backfire effects among conservatives who received corrections of misperceptions about the war in Iraq and the revenue-stimulating effects of tax cuts (2008). One important question for future research is why the corrective affirmation provoked a backlash when nonwhite administrators were absent and the misperception negation did not. This discrepancy may be the result of cognitive dissonance among the Republican subsample when Obama invokes his Christian beliefs in the corrective affirmation treatment. GOP identifiers were more likely to self-identify as evangelical Christians (59% vs. 43%; !2(1) = 3.03, p < .10) and to report positive feelings toward Christians than other participants (M = 8.37, SE = 0.44 vs. M = 7.52, SE = 0.20; p < .10). They also reported less positive feelings towards Obama (M = 5.81, SE = 0.57 vs. M = 8.05, SE = 0.18; p < .01). The disjunction between GOP identifiers’ positive feelings toward Christianity and their lukewarm views of Obama may have induced cognitive dissonance, leading them to denigrate his claims about his religious beliefs. This counterarguing may have led to increased belief in the misperceptions that Obama is or was a Muslim as well as the corresponding declines in Honesty and Feelings. The results presented here underscore the extreme difficulty associated with debunking political misperceptions. We find that misperception negations are not effective at reducing misperceptions – a result that is consistent with psychological research on the processing of negations. However, the corrective affirmation – a seemingly intuitive alternate approach – fails to reduce misperceptions without the social desirability pressure of non-white experimental administrators and provokes a backfire effect when provided to the least favorably disposed subjects. 26
Conclusion This study extends the state of the art in research on correcting political misperceptions using a salient rumor from the 2008 presidential campaign. Specifically, we apply a social psychological theory about the ineffectiveness of negations to test corrections of the misperception that Barack Obama is a Muslim, contrasting a misperception negation (“I am not and never have been of the Muslim faith”) with a corrective affirmation (“I am a Christian”). Our experiment has several innovative features, including a focus on the effectiveness of a candidate’s attempts to correct a misperception (rather than a neutral source), the use of video-based corrections as experimental manipulations, and the construction of an implicit association test measuring associations between the presidential candidates and religion. We find that the corrective affirmation was effective at reducing misperceptions only when non-white experimental administrators were present, suggesting a social desirability effect. In their absence, Republicans appear to have been negatively provoked by the treatment, increasing their levels of belief in the misperception and decreasing their perceptions of Obama’s honesty about his religion. Our results suggest that researchers should consider randomizing the assignment of experimental administrators even when conducting computer-based survey or experimental research. These findings strongly suggest that the race of administrators can even affect the results of computer-based experiments. Further research should also be conducted to ascertain the influence of minority administrators from different racial groups. In our case, we cannot disentangle whether the effects we found were due to the presence of a racial minority, the specific presence of someone who is black, or someone who may be falsely perceived as Muslim (the Indian-American administrator). 27
Future research should also seek to improve on several limitations of this study and to pursue several possible extensions. For instance, it would be desirable to test the effectiveness of both correction treatments on a larger sample with more Republican and white respondents. One might also wish to evaluate the relative effectiveness of written corrective treatments relative to video clips. A third extension could contrast the effect of corrections of misperceptions about Obama’s religion based on the source of the information. Our use of Obama as the source had two primary virtues – it allowed us to test how candidates can most effectively correct misperceptions about themselves using real video clips. However, perceptions of Obama’s credibility likely varied widely along partisan lines during the election campaign. It is unknown whether Hypothesis 1 would have received greater support if a neutral source had been used. Overall, the realistic nature of our experiment makes a valuable contribution to the study of misperceptions and attempts to correct them. In particular, we demonstrate the key role played by desirability concerns in reporting one’s belief in politically sensitive misperceptions – a finding that may raise questions about other studies of public opinion about Barack Obama’s religion. In the absence of such concerns, our results highlight the difficulty of correcting misperceptions and the potential for corrections to provoke a backlash from citizens with the strongest commitments to those beliefs.
28
Works cited Bordia, Prashant, Nicholas DiFonzo, and Cassandra A. Schulz. 2000. “Source Characteristics in Denying Rumors of Organizational Closure: Honesty Is the Best Policy.” Journal of Applied Social Psychology 30(11): 2309-2321. Bordia, Prashant, Nicholas DiFonzo, Robin Haines, and Elizabeth Chaseling. 2005. “Rumors Denials as Persuasive Messages: Effects of Personal Relevance, Source, and Message Characteristics. Journal of Applied Social Psychology 35(6): 13011331. Brambor, Thomas, William Roberts Clark, and Matt Golder. 2006. “Understanding Interaction Models: Improving Empirical Analyses.” Political Analysis, 14: 6382. Cialdini, Robert B., Petia K. Petrova, Linda J. Demaine, Daniel W. Barrett, Brad J. Sagarin, Kelton L. Rhoads, and Jon Maner. 2008. “The Poison Parasite Defense: Instilling Persistent Resistance to Persuasion.” Unpublished manuscript. Deutsch, Roland and Bertram Gawronski. 2009. “When the Method Makes a Difference: Antagonistic Effects on ‘Automatic Evaluations’ as a Function of Task Characteristics of the Measure.” Journal of Experimental Social Psychology 45: 101-114. DiFonzo, Nicholas and Prashant Bordia. 2006. Rumor Psychology: Social and Organizational Approaches. American Psychological Association. Edwards, Kari, and Edward E. Smith. 1996. “A Disconfirmation Bias in the Evaluation of Arguments.” Journal of Personality and Social Psychology, 71(1): 5-24. Gawronski, Bertram, Roland Deutsch, Sawsan Mbirkou, Beate Seibt, and Fritz Strack. 2008. “When ‘Just Say No’ is not enough: Affirmation versus negation training and the reduction of automatic stereotype activation.” Journal of Experimental Social Psychology 44: 370-377. Gilens, Martin. 2001. “Political Ignorance and Collective Policy Preferences.” American Political Science Review, 95(2): 379-396. Greenwald, Anthony G., Brian A. Nosek, and Mahzarin R. Banaji. 2003. Understanding and using the implicit association test: I. An improved scoring algorithm. Attitudes and Social Cognitions, 85(2): 197-216. Greenwald, Anthony G., Debbie E. McGhee, and Jordan L. K. Schwartz. 1998. Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, 1464-1480.
29
Hatchett, Shirley and Howard Schuman. 1975/1976. “White Participants and Race-ofInterviewer Effects.” Public Opinion Quarterly, 39: 523-526. Hopkins, Daniel J. 2008. “No More Wilder Effect, Never a Whitman Effect: When and Why Polls Mislead about Black and Female Candidates.” Unpublished manuscript. Johar, G.V. 1996. “Intended and Unintended Effects of Corrective Advertising on Beliefs and Evaluations.” Journal of Consumer Psychology 5(3): 209-230. Johnson, Holly M. and Colleen M. Seifert. 1994. “Sources of the Continued Influence Effect: When Misinformation in Memory Affects Later Inferences.” Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(6): 1420-1436. King, Gary, Michael Tomz, and Jason Wittenberg. 2000. “Making the Most of Statistical Analyses: Improving Interpretation and Presentation.” American Journal of Political Science, 44(2): 341-355. Krysan, Maria and Mick P. Couper. 2003. “Race in the Live and the Virtual Interview: Racial Deference, Social Desirability, and the Activation Effects in Attitude Surveys.” Social Psychology Quarterly. 66(4): 364-383. Kuklinski, James H., Paul J. Quirk, Jennifer Jerit, David Schweider, and Robert F. Rich. 2000. “Misinformation and the Currency of Democratic Citizenship.” The Journal of Politics, 62(3):790-816. Mayo, Ruth, Yaacov Schul and Eugene Burnstein. 2004. “‘I am not guilty’ vs ‘I am innocent’: Successful misperception negation may depend on the schema used for its encoding.” Journal of Experimental Social Psychology, 40(4): 433-449. Nosek, Brian A., Mahzarin R. Banaji, and Anthony G. Greenwald. 2002. “Math=Male, Me=Female, Therefore Math!Me.” Journal of Personality and Social Psychology 83(1): 44-59. Nosek, Brian A. 2004. The relationship between implicit and explicit attitudes. Paper presented at Society for Personality and Social Psychology meeting, Austin, TX. Nosek, Brian A., Anthony G. Greenwald and Mahzarin R. 2005. Understanding and Using the Implicit Association Test: II. Method Variables and Construct Validity. Personality and Social Psychology Bulletin, 31, 166-180. Nyhan, Brendan and Jason Reifler. 2008. “When Corrections Fail: The persistence of political misperceptions.” Viewed on 10 November 2008.
30
Obama, Barack. “Call to Renewal Keynote Address.” June 28, 2006. http://www.barackobama.com/2006/06/28/call_to_renewal_keynote_address.php Oskamp, S., and Schultz, P.W. 2004. Attitudes and opinions, third edition. Philadelphia: Lawrence Erlbaum. Petrova, Petia K., Robert B. Cialdini, Noah J. Goldstein, and Vladas Griskevicius. 2008. “Protecting Consumers from Harmful Advertising: What Constitutes an Effective Counter Argument?” Unpublished manuscript. Princeton Survey Research Associates International. 2008. “Newsweek Poll Obama and God.” Poll conducted July 9-10, 2008 and released July 11, 2008. Results downloaded February 23, 2009 from http://www.psrai.com/_uploads/0807%20ftop%20w%20methodology.pdf Pew Research Center for the People & the Press. 2009. “No Decline in Belief That Obama is a Muslim.” Poll conducted March 9-12, 2009 and released April 1, 2009. Results downloaded April 24, 2009 from http://pewresearch.org/pubs/1176/obama-muslim-opinion-not-changed Pew Research Center for the People & the Press. 2008. “McCain Gains On Issues, But Stalls As Candidate of Change.” Poll conducted September 9-14, 2008 and released September 18, 2008. Results downloaded February 23, 2009 from http://people-press.org/report/450/presidential-race-remains-even Redlawsk, David. “Implications of Motivated Reasoning for Voter Information Processing.” International Society of Political Psychology, 2001. Sides, John and Jack Citrin. 2007. “How Large the Huddled Masses? The Causes and Consequences of Public Misperceptions about Immigrant Populations.” Paper presented at the 2007 annual meeting of the Midwest Political Science Association, Chicago, IL. Skurnik, Ian, Carolyn Yoon, Denise C. Park, and Norbert Schwarz. 2005. “How warnings about false claims become recommendations. Journal of Consumer Research 31: 713-724. Taber, Charles S. and Milton Lodge. 2006. “Motivated Skepticism in the Evaluation of Political Beliefs.” American Journal of Political Science, 50(3): 755-769. Taber, Charles S., Damon Cann, and Simona Kucsova. Forthcoming. “The Motivated Processing of Political Arguments.” Political Behavior. Tomz, Michael, Jason Wittenberg, and Gary King. 2003. “Clarify: Software for Interpreting and Presenting Statistical Results.” Journal of Statistical Software 8(1): 1-30. 31
Wood, Michelle L.M. 2007. “Rethinking the Inoculation Analogy: Effects on Subjects With Differing Preexisting Attitudes.” Human Communication Research 33(3): 357-378.
32
Appendix A Obama manipulations Participants are randomly assigned one of the following four video clips. Each is 10-11 seconds long and contains a close-up shot of Obama talking. The misperception negation and corrective affirmation clips that follow are taken from the same interview and thus have identical camera shots and backdrops. Prompt: “You will now watch another video. Please pay close attention, as you did with the last video.” 1. Control video (“60 Minutes,” Feb. 10, 2008) Transcript (not visible during experiment) OBAMA: "I had to think about this long and hard at the beginning of this process and say are you deluding yourself or do you really think that you can do all those things." Excerpt displayed on screen at end of video: “I had to think about this long and hard at the beginning of this process.” 2. Misperception negation correction video (Christian Broadcasting Network, Jan. 22, 2007) Transcript (not visible during experiment) OBAMA: "I'm unequivocal about this -- I am not and never have been of the Muslim faith. I think that those who are of the Muslim faith are deserving of respect and dignity." Excerpt displayed on screen at end of video: “I'm unequivocal about this -- I am not and never have been of the Muslim faith.” 3. corrective affirmation correction video (Christian Broadcasting Network, Jan. 22, 2007) Transcript (not visible during experiment) OBAMA: "I want to make sure that your viewers understand that I am a Christian who has belonged to the same church for almost twenty years now." Excerpt displayed on screen at end of video: “I am a Christian who has belonged to the same church for almost twenty years now.” Explicit dependent variables “Do you happen to know what Barack Obama's religion is?” -Jewish -Buddhist 33
-Christian -Muslim -Hindu -Atheist -Agnostic -Something else -Don't know Branching on Christian and Muslim responses: “How devout of a Christian/Muslim do you think Obama is?” -Extremely devout Christian/Muslim -Very devout Christian/Muslim -Somewhat devout Christian/Muslim -Not too devout Christian/Muslim -Not at all devout Christian/Muslim ReligionChoice response scale: -“Extremely/very devout Muslim” (5) -“Somewhat/not too/not at all devout Muslim” (4) -“Don’t know” (3) -“Somewhat/not too/not at all devout Christian” (2) -“Extremely/very devout Christian” (1) “Please indicate whether you agree or disagree with the following statements:” “Barack Obama is a Muslim.” (Muslim) -Strongly agree (7) -Somewhat agree (6) -Slightly agree (5) -Neither agree nor disagree (4) -Slightly disagree (3) -Somewhat disagree (2) -Strongly disagree (1) -Don't know (4) “Barack Obama used to be a Muslim.” (Ex-Muslim) -Strongly agree (7) -Somewhat agree (6) -Slightly agree (5) -Neither agree nor disagree (4) -Slightly disagree (3) -Somewhat disagree (2) -Strongly disagree (1) -Don't know (4)
34
“Please state whether you agree or disagree with the following statements:” “Barack Obama is being honest about his religious faith.” (Honest) -Strongly agree (7) -Somewhat agree (6) -Slightly agree (5) -Neither agree nor disagree (4) -Slightly disagree (3) -Somewhat disagree (2) -Strongly disagree (1) “Here are some questions about your feelings toward some people and groups who have been in the news. For each one, please rate them using something called the feeling thermometer. Your responses will be kept strictly confidential so please be open and honest in your answers.” “For each item, you can choose any number between 0 and 10. The higher the number, the warmer or more favorable you feel toward that person or group; the lower the number, the colder or less favorable. You would rate the person or group as 5 if you feel neither warm nor cold toward them.” “If we come to a person or group that you don't recognize, you don't need to rate them -just move on to the next one.” Barack Obama (Feelings) 0 1 2 3 4 5 6 7 8 9 10
35
Appendix B The IAT application began with explicit instructions that directed participants to press the “E” key for images or words that fit under the category on the left-hand side of the screen and the “I” key for images or words that fit under the category on the right-hand side of the screen. They were told that if they responded incorrectly, a red “X” would appear, in which case they should press the opposite button. They were then told to inform the experimenter if they had any questions before the start of the IAT, and if they had none, they should press the spacebar to continue. Participants then completed the IAT as specified in these instructions. The IAT followed the block order described in the “Construction” section above. Between each block of the IAT, the instructions regarding which button to press for each category were repeated. Table B1 summarizes the structure of the IAT, which consists of seven “blocks.” Four of these are used in the final algorithm used to compute each participant’s IAT score. During each of these blocks, the computer recorded the time it took the participant to respond (“latency time”) and whether or not their answer was correct. The first two practice blocks were intended to help participants learn the concept dimensions (of John McCain vs. Barack Obama) and the attribute dimensions (of Christian vs. Islam). The first block of 20 trials required participants to sort a mix of images of John McCain and Barack Obama based on what image appeared on screen by pressing either the “E” key for McCain or the “I” key for Obama. The second block of 20 trials required participants to sort words that were either Christian or Muslim into their appropriate categories by pressing the “E” key for Christian or the “I” key for Muslim.
36
The purpose of the next two practice blocks was to test the first phase of conceptattitude pairing. The third block of 20 practice trials then required participants to press the “E” key if either an image of Barack Obama or a Muslim word appeared on screen and to press the “I” key if either an image of John McCain or a Christian word appeared on screen. The fourth block repeated the procedure of the third block, but this block actually tested participants’ implicit associations. The fifth block served as another practice block for participants and was intended to help participants learn to switch the spatial locations on the screen of the concepts they represent. In 20 trials, participants repeated the exact same task as in the first block of trials except that the categories were assigned to opposite keys. The final two blocks represented the second phase of concept-attitude pairing. The sixth block of 20 practice trials required participants to press the “E” key if either an image of Barack Obama or a Christian word appeared on screen and to press the “I” key if either an image of John McCain or a Muslim word appeared on screen. The seventh block repeated the procedure of the sixth block but consisted of 40 test trials. Research shows that if the intuitive pairing is first (in this case, Obama with Muslim and McCain with Christian), the difference in latency times will be greater than if the non-intuitive pairing is first (Greenwald, McGhee, & Schwartz, 1998). To correct this error, the positions of the pairings in the presentation order were randomly switched for half of the participants. The tables below describe the two orderings of the blocks that were used in the procedure.
37
Intuitive pairing first:
Block 1 2 3 4 5 6 7 # of trials 20 20 20 40 20 20 40 Function Practice Practice Practice Test Practice Practice Test Item(s) assigned to “E” key John McCain images Christian words Obama images + Muslim words Obama images + Muslim words Barack Obama images Obama images + Christian words Obama images + Christian words Item(s) assigned to “I” key Barack Obama images Muslim words McCain images + Christian words McCain images + Christian words John McCain images Obama images + Muslim words Obama images + Muslim words
Non-intuitive pairing first:
Block 1 2 3 4 5 6 7 # of trials 20 20 20 40 20 20 40 Function Practice Practice Practice Test Practice Practice Test Item(s) assigned to “E” key Barack Obama images Christian words Obama images + Christian words Obama images + Christian words John McCain images McCain images + Christian words McCain images + Christian words Item(s) assigned to “I” key John McCain images Muslim words McCain images + Muslim words McCain images + Muslim words Barack Obama images Obama images + Muslim words Obama images + Muslim words
38
Figure 1: Distributions of misperception variables
ReligionChoice
50 40 Percent 20 30 40 50
Muslim
10
Percent 20 30
0
Devout Christian
Don’t know
Devout Muslim
0
Strongly disagree
10
Neither
Strongly agree
Ex−Muslim
50 40 Percent 10 15 20
ReligionIAT
Percent 20 30
10
0
0 −1.5
5
Stongly disagree
Neither
Strongly agree
−1
−.5
0
.5
1
1.5
Higher values represent greater levels of belief that Obama is or was a Muslim (ReligionChoice, Muslim, Ex-Muslim) or stronger associations between Obama and Islam (ReligionIAT)
39
Figure 2: Corrective affirmation treatment effects for ReligionChoice
GOP
3.5 3.5
Non−GOP
3
Obama religion choice 2.5
2
1.5
0 Corrective affirmation
White administrators Non−white
1
1.5 0 Corrective affirmation
White administrators Non−white
2
2.5
3
1
Higher values represent greater levels of belief that Obama is a Muslim.
40
Figure 3: Corrective affirmation treatment effects for Muslim
GOP
6 6
Non−GOP
5
Obama Muslim scale 3 4
2
1
0 Corrective affirmation
White administrators Non−white
1
1 0 Corrective affirmation
White administrators Non−white
2
3
4
5
1
Higher values represent greater levels of belief that Obama “is a Muslim.”
41
Figure 4: Corrective affirmation treatment effects for Ex-Muslim
GOP
6 6
Non−GOP
5
Obama Ex−Muslim scale 4
3
2
0 Corrective affirmation
White administrators Non−white
1
2 0 Corrective affirmation
White administrators Non−white
3
4
5
1
Higher values represent greater levels of belief that Obama “used to be a Muslim.”
42
Figure 5: Corrective affirmation treatment effects for Honest
GOP
7 7
Non−GOP
6
Obama honesty 5
4
3
0 Corrective affirmation
White administrators Non−white
1
3 0 Corrective affirmation
White administrators Non−white
4
5
6
1
Higher values represent greater levels of belief that Obama is “being honest about his religious faith.”
43
Figure 6: Corrective affirmation treatment effects for Feelings
GOP
8 8
Non−GOP
7
Obama feeling thermometer 6
5
4
0 Corrective affirmation
White administrators Non−white
1
4 0 Corrective affirmation
White administrators Non−white
5
6
7
1
Higher values represent warmer feelings toward Obama.
44
Table 1: Treatment effects by race of administrator
White administrators Non-white administrators ReligionChoice Muslim Ex-Muslim ReligionChoice Muslim Ex-Muslim Misperception negation 0.18 -0.28 -0.42 -0.23 -0.75* -0.73 (0.24) (0.44) (0.43) (0.29) (0.45) (0.59) Corrective affirmation -0.24 0.18 0.39 -0.86*** -0.97** -1.09* (0.23) (0.43) (0.42) (0.29) (0.43) (0.57) Black respondent -0.17 -0.38 -0.53 -0.16 -0.03 0.14 (0.20) (0.36) (0.35) (0.26) (0.39) (0.52) Political knowledge -0.32 -2.71*** -2.54*** -0.44 -2.72*** -1.63** (0.31) (0.58) (0.56) (0.39) (0.58) (0.76) GOP -0.45 -0.29 0.64 1.07** 1.74** 2.08** (0.40) (0.72) (0.70) (0.53) (0.71) (0.95) Misperception negation X GOP 0.56 1.66 -0.60 -0.59 -0.36 -0.60 (0.63) (1.17) (1.14) (0.75) (1.06) (1.41) Corrective affirmation X GOP 0.85 2.58** 0.71 -1.09 -0.55 -0.67 (0.67) (1.17) (1.14) (0.70) (0.96) (1.28) Constant 2.08*** 3.77*** 4.04*** 2.55*** 3.68*** 3.57*** (0.25) (0.46) (0.45) (0.32) (0.48) (0.64) 0.05 0.19 0.17 0.26 0.41 0.24 R2 N 137 147 147 77 77 78 *p < .10, **p < 0.05, ***p < .01 (two-sided)
Higher values on ReligionChoice, Muslim, and Ex-Muslim represent greater levels of belief that Obama is or was a Muslim.
45
Table 2: Three-way interaction models of misperceptions
ReligionChoice 0.18 (0.23) Corrective affirmation -0.25 (0.22) Black respondent -0.17 (0.15) Political knowledge -0.37 (0.24) GOP -0.44 (0.39) Non-white administrator 0.43 (0.27) 0.56 Misperception negation x GOP (0.61) Non-white x GOP 1.53** (0.67) Non-white x misp. negation -0.41 (0.38) -1.19 Non-white x misp. negation x GOP (0.98) Corrective affirmation x GOP 0.86 (0.64) Non-white x corr. affirmation -0.60 (0.37) Non-white x corr. affirmation x GOP -1.97** (0.97) IAT counterbalance
Misperception negation Constant R2 N *p < .10, **p < 0.05, ***p < .01 (two-sided) 2.09*** (0.22) 0.14 214
Muslim -0.30 (0.40) 0.16 (0.39) -0.27 (0.27) -2.74*** (0.43) -0.28 (0.66) 0.08 (0.48) 1.68 (1.07) 1.95* (1.11) -0.50 (0.69) -2.04 (1.70) 2.63** (1.06) -1.11* (0.66) -3.12* (1.61)
Ex-Muslim -0.40 (0.43) 0.40 (0.41) -0.33 (0.29) -2.25*** (0.45) 0.62 (0.70) 0.23 (0.51) -0.57 (1.13) 1.26 (1.17) -0.43 (0.73) 0.21 (1.80) 0.74 (1.13) -1.51** (0.70) -1.20 (1.71)
3.72*** (0.38) 0.26 224
3.83*** (0.40) 0.19 225
Higher values on ReligionChoice, Muslim, and Ex-Muslim represent greater levels of belief that Obama is or was a Muslim.
46
Table 3: Three-way interaction models of Obama perceptions
Honest -0.44 (0.37) Corrective affirmation -0.34 (0.36) Black respondent 0.45* (0.25) Political knowledge 0.71* (0.39) GOP 0.72 (0.61) Non-white administrator -0.33 (0.44) Misperception negation x GOP -0.99 (0.98) Non-white x GOP -1.38 (1.02) Non-white x misp. negation 0.23 (0.64) Non-white x misp. negation x GOP 0.93 (1.57) Corrective affirmation x GOP -2.71*** (0.98) Non-white x corr. affirmation 0.8 (0.62) Non-white x corr. affirmation x GOP 3.13** (1.49) Constant 5.29*** (0.35) R2 0.10 N 222 *p < .10, **p < 0.05, ***p < .01 (two-sided)
Misperception negation
Feelings 0.32 (0.59) -0.42 (0.57) 0.76* (0.40) -0.73 (0.64) -1.14 (1.01) -0.04 (0.72) -1.12 (1.59) -1.43 (1.65) -0.08 (1.01) 0.35 (2.50) -1.9 (1.58) 0.65 (0.98) 4.24* (2.42) 7.87*** (0.56) 0.16 219
Higher values on Honest and Feelings represent more positive perceptions of Obama’s honesty about his religious faith and feelings toward him, respectively.
47
Table 4: ReligionIAT correspondence with explicit dependent variables (4a) DV=ReligionChoice
White administrators Control ReligionIAT GOP ReligionIAT x GOP Political knowledge Black respondent Constant R2 N 0.32 (0.34) -0.60 (0.47) 0.06 (0.73) -0.27 (0.56) 0.08 (0.38) 1.83*** (0.40) 0.08 40 Corr. aff. 0.39 (0.25) -0.84 (0.91) 2.40 (1.57) -0.72 (0.51) 0.02 (0.28) 1.79*** (0.26) 0.16 45 Non-white administrators Control 0.71 (0.51) 2.03 (1.92) -1.66 (4.36) -1.04 (1.00) -0.23 (0.72) 2.69*** (0.71) 0.44 18 Corr. aff. -0.15 (0.22) -0.26 (0.28) 0.50 (0.57) 0.73 (0.44) 0.06 (0.27) 1.16*** (0.31) 0.25 25
*p < .10, **p < 0.05, ***p < .01 (two-sided)
(4b) DV=Muslim
White administrators Control ReligionIAT GOP ReligionIAT x GOP Political knowledge Black respondent Constant R2 N 1.07* (0.62) -0.81 (0.84) 0.40 (1.30) -2.69*** (0.97) -0.03 (0.66) 3.51*** (0.69) 0.26 41 Corr. aff. 0.28 (0.51) 3.05* (1.81) -0.80 (3.31) -4.91*** (1.11) -0.63 (0.59) 4.59*** (0.55) 0.38 48 Non-white administrators Control 0.05 (0.81) 1.99 (1.98) -1.03 (2.27) -2.67 (1.53) -0.98 (1.14) 3.75*** (1.10) 0.33 19 Corr. aff. 0.38 (0.50) 1.26* (0.64) -2.77** (1.32) -2.39** (1.03) -0.03 (0.63) 2.63*** (0.72) 0.51 25
*p < .10, **p < 0.05, ***p < .01 (two-sided)
48
(4c) DV=Ex-Muslim
White administrators Control ReligionIAT GOP ReligionIAT x GOP Political knowledge Black respondent Constant R2 N 1.50** (0.65) -0.14 (0.88) 1.02 (1.37) -2.57** (1.01) 0.01 (0.69) 3.69*** (0.73) 0.30 41 Corr. aff. 0.56 (0.41) 1.46 (1.48) -0.73 (2.71) -4.25*** (0.88) -1.48*** (0.48) 5.44*** (0.44) 0.45 48 Non-white administrators Control 0.74 (1.01) 3.60 (2.47) -1.52 (2.84) -2.19 (1.91) 0.05 (1.42) 4.20*** (1.38) 0.30 19 Corr. aff. -0.71 (0.57) 2.25*** (0.73) -3.27** (1.51) -1.29 (1.17) 0.08 (0.73) 2.22** (0.82) 0.53 26
*p < .10, **p < 0.05, ***p < .01 (two-sided)
Higher values on ReligionChoice, Muslim, and Ex-Muslim represent greater levels of belief that Obama is or was a Muslim.
49
Table 5: Predicted effects of corrective affirmation treatment
Party: ReligionChoice Muslim Ex-Muslim Honest Feelings Non-GOP White adm. Non-white -0.25 -0.85*** (0.21) (0.31) 0.16 -0.95* (0.38) (0.54) 0.40 -1.10** (0.42) (0.56) -0.33 0.45 (0.36) (0.51) -0.46 0.23 (0.56) (0.85) GOP White adm. Non-white 0.61 -2.00*** (0.63) (0.66) 2.81*** -1.50 (1.00) (1.09) 1.11 -1.57 (1.06) (1.18) -3.13*** 0.89 (0.90) (1.00) -2.34 2.66 (1.49) (1.69)
*p < .10, **p < 0.05, ***p < .01 (two-sided)
Higher values on ReligionChoice, Muslim, and Ex-Muslim represent greater levels of belief that Obama is or was a Muslim. Higher values on Honest and Feelings represent more positive perceptions of Obama’s honesty about his religious faith and feelings toward him, respectively.
50