Positive Affect Negative Affect Schedule: A Reliability Generalization Study Evie M. Muilenburg-Trevino University of Oklahoma-Tulsa, Center of Applied Research for Non-profit Organizations Abstract Figure 1: Boxplot of Positive Affect (PA) Reliability Scores One- reliability One-way analyses of variance (ANOVA) were computed to examine reliability instructions. coefficients for the PA and NA scales with timeframe instructions. In the first Reliability generalization procedures were used in the present study to examine study timeframe analysis, PA reliability estimates were compared with eight timeframe internal consistency estimates of the Positive Affect Negative Affect Schedule Affect instructions. The assumption for homogeneity of variance was not violated two- (PANAS; Watson, Clark, & Tellegen, 1998), a two-factor measure for self- self- [Levene’ [Levene’s F(7, 100) = 1.915; p > .05]. Results indicated a statistically significant reported mood. In addition, study characteristics that may impact score reliability impact post difference, F(7, 100) = 3.629; p < .01, eta square = .203. A post hoc analysis was were investigated. The mean reliability estimates for PA and NA scales were computed to examine the difference among timeframe instructions. The Tukey acceptable with both scales demonstrating negative skewness. In regard to study year” HSD post hoc indicated that “year” (M = .75; SD = .11) was significantly lower characteristics, number of items correlated with PA and NA scale reliability moment” today” than “moment” (M = .88; SD = .05; p < .01), “today” (M = .88; SD = .02; p <.01), estimates. Finally, ANOVA results indicated that timeframe was significant in the significant week” weeks” “past week” (M = .86; SD = .06; p < .05), “past few weeks” (M = .87; SD = .02; p < level of score reliability for PA with “year” having the lowest score reliability. year” month” .01), and “past month” (M = .86; SD = .03; p < .05) timeframes. No statistically year” days” significant differences existed between “year” and “past few days” (M = .86; SD = Introduction general” .04; p > .05) or “general” (M = .84; SD = .06; p > .05) timeframe instructions. 0 .70 0 .80 0 .90 Watson, Clark & Tellegen (1988) developed the Positive Affect Negative Affect PA Reli abi li ty Coeffi ci ent Another ANOVA was computed to examine NA reliability estimates and and Negative Schedule (PANAS) validating a two factor measure for self-reported mood. Given variance timeframe instructions. The assumption for homogeneity of variance was not self- that the PANAS is a widely used measure in psychological research, evaluating Figure 2: Boxplot of Negative Affect (NA) Reliability Scores [Levene’ difference violated [Levene’s F(7, 87) = 1.329; p > .05]. No statistically significant difference research, the psychometric properties of its scores could increase the level of confidence in was found, F(7, 87) = 0.601; p < .05, partial eta = .046. level current interpreting empirically based conclusions. The purpose of the current study is to present a reliability generalization analysis across studies using the PANAS. using Conclusion While the theoretical and empirical support for the construct of is promising, research surrounding score reliability remains to be established. established. The objectives of the present study were to examine reliability estimates for the PANAS using RG techniques and investigate study characteristics that may METHODS influence score reliability. The mean reliability estimates for PA and NA scale was acceptable with both scales demonstrating a negative skewness in the A literature search was conducted in the PsychARTICLES and PsychINFO characteristics, distribution of reliability estimates. In regard to study characteristics, number of PsychINFO databases using the terms positive and negative affect schedule and positive and this items correlated with PA and NA scale reliability estimates; this is not surprising negative affect scale. Duplicated article were eliminating, resulting in a total of given that score reliability tends to increase as the number of items increases (cf. resulting 316 articles. An examination of these articles identified 59 articles did not use the reliability Cortina, 1993). Mean age also correlated negatively with PA reliability estimates. articles scale and 22 articles were written in non-English or were unavailable or unusable. 0 .70 0 .80 0 .90 level ANOVA results indicated that timeframe was significant in the level of score non- Of the remaining 235 articles, 62 (26.4%) did not report a reliability score and 74 NA Reli abi li ty Coeffi ci ent year” reliability for PA with consideration of “year” having the lowest score reliability. reliability (31.5%) induced a reliability score from another study. Cronbach alpha A correlation matrix including descriptive characteristics for variables in the study variables REFERENCES test- coefficients were reported in 94 studies and test-retest scores were reported in 5 is presented in Table 1. The correlation between the reliability coefficient and test- studies. Because of the limited number of test-retest scores, only cronbach alpha number of items was statistically significant for the PA scale (rxy = .45; p < .01) (rxy examination Cortina, J. M. (1993). What is a coefficient alpha? An examination of theory and estimates were used in the present study. Some articles provided reliability and NA scale (rxy = .19; p < .05). In addition, the correlation between the PA estimates for multiple samples, resulting in 140 coefficient alphas from the 94 alphas 78, 98- application. Journal of Applied Psychology, 78, 98-104. significant reliability coefficient and mean age was also statistically significant (rxy = -.36; p < articles. .01). Henson, R. K., & Thompson, B. (2002). Characterizing measurement error in RESULTS scores across studies: Some recommendations for conducting “reliability Zero- Table 1: Zero-Order Correlation Matrix of PANAS Reliability Scores and Study generalization” studies. Measurement and Evaluation in Counseling and generalization” Variables Development, 35, 113-126. For the positive affect (PA) scale, the mean reliability coefficient for the 133 cases coefficient 35, 113- PA α NA α reporting internal consistency reliability was .86 (SD = .06; SE = .005) with a 95% M SD Correlation Correlation confidence interval ranging from .85 to .87. The median and mode reliability Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of Number of scales. brief measures of positive and negative affect: The PANAS scales. Journal of from coefficient were .88 and .89 respectively, with scores ranging from .62 to .96. The items 9.42 1.59 .45** .19* PA distribution was negatively skewed (-2.05; SE = .210) with a kurtosis of 5.63 54, 1063- Personality and Social Psychology, 54, 1063-1070. (- Sample size 253.67 321.45 -.10 .09 reliability (SE = .417). For the negative affect (NA) scale, the mean reliability coefficient for Mean age 35.05 19.50 -.36** -.11 the 119 cases reporting internal consistency reliability was .84 (SD = .05; SE = Percent male 38.36 26.95 -.01 -.18 .005) with a 95% confidence interval ranging from .83 to .85 The median and Percent mode reliability coefficient were .85 and .87 respectively, with scores ranging from Caucasian 67.04 32.52 -.05 -.07 (- .67 to .95. The NA distribution was negatively skewed (-.98; SE = .222) with a kurtosis of 1.10 (SE = .440). See Figures 1 and 2 for boxplots of PA and NA of Note. * p < .05, ** p < .01 reliability scores.