VIEWS: 52 PAGES: 17 POSTED ON: 12/3/2011
Meade, A. W. (2008, April). Power of AFIs to Detect CFA Model Misfit. Paper presented at the 23th Annual Conference of the Society for Industrial and Organizational Psychology, San Francisco, CA. Power of AFIs to Detect CFA Model Misfit Adam W. Meade North Carolina State University Hu and Bentler (1999) have derived guidelines for approximate fit indices (AFIs) that are indicative of adequate model fit. The current study evaluated these guidelines for data in which an unmodeled factor was present. Results indicated poor power to detect model misspecification for all AFIs examined. Confirmatory factor analysis (CFA) has 2007). Many of these AFIs are derived from the same become a primary tool in scale development and fit function used to calculate the chi-square statistic measure evaluation in psychological and (e.g., CFI, IFI, NFI, RNI, TLI), while others index organizational research (Hinkin, 1998). While there average discrepancy between reproduced and are several methods of evaluating CFA model fit, the observed correlations (e.g., RMSR). Excellent excessive sensitivity of the chi-square statistic has led overviews of the AFIs are available in the extant to the development and application of many literature (e.g., Hu & Bentler, 1998; Marsh et al., approximate fit indices (AFIs; see Marsh, Bella, and 1996; Tanaka, 1993) and will not be discussed in Hau, 1996). Previous simulation studies have detail. indicated that reasonable assurance of adequate There have been many simulation studies model fit may be found when some fit indices are over the years in which various model meet or exceed particular values. A recent such study misspecification has been simulated in order to by Hu and Benter (1999) has been enormously determine the performance of the AFIs. The most influential, cited over 2000 times (as of September influential article on AFI performance has been that 10, 2007), to the extent that Barrett (2007) has by Hu and Bentler (1999). In their study, recently stated that this work has become the “bible” misspecification was based on improperly of the “Golden Rules” of fit. constrained factor covariances or factor cross- In their simulation work, Hu and Bentler loadings. Based on these simulations, Hu and Bentler (1999) simulated several types of model proposed evaluating model fit using SRMR values misspecification in order to derive their suggested less than or equal to .08 as that index was most AFI values that indicate acceptable fit. These sensitive to misspecification among factor misspecifications included improperly specified covariances. Additionally they suggested evaluating covariances among latent factors and improperly model fit using one of the indices more sensitive to constrained factor loadings (i.e., erroneously omitting factor loading misspecification. These include TLI, cross-loadings). Despite the sizable nature of their IFI, CFI, RNI, and Gamma-hat values greater than or simulations, they failed to investigate the effect of equal to .95, while .90 was recommended for Mc, model misspecification due to an unmodeled latent with a recommended RMSEA value less than or factor. In this study, we simulate a number of models equal to .06. with a misspecified factor structure in which an More recently, other authors have continued additional latent variable is omitted, to evaluate to index the performance of AFIs under certain data behavior of both the chi-square statistic and AFIs. conditions. For instance, Marsh, Hau, and Wen (2004) sought to replicate many of Hu and Benter’s Prior Fit Index Simulation Research (1999) conditions. They came to the general The excessive sensitivity of the chi-square conclusion that the Hu and Bentler recommended statistic with large samples has been known for some cutoffs were somewhat conservative for some types time, which rapidly gave rise to the development of of models. Beauducel and Wittmann (2005) sought to several AFIs in order to better index the extent to simulate data more similar to personality data than which models “approximately” fit the data (Steiger, were Hu and Bentler’s (1999) simulations. They used AFIs and CFA Misfit 2 lower correlations and factor loadings than Hu & replications that exceeded the values suggested by Hu Bentler (i.e., lower communality and factor and Bentler (1999). reliability) and simulated data with secondary loadings on modeled factors. They found that misfit Method under typical conditions encountered in personality research may be less likely to be detected with An initial structural model was developed RMSEA and RMSR than other AFIs. for four correlated factors. Factor loadings and factor Fan & Silvo (2005) did not evaluate correlations were taken from a recently published individual AFIs, but rather the strategy put forth by study (Donnellan, Oswald, Baird, & Lucas, 2006) of Hu and Benter (1999) of reporting two indices. Their a short form of a commonly used personality survey, results question Hu and Benter’s finding that RMSR the International Personality Item Pool (IPIP; was more sensitive to factor covariance Goldberg, 1999). While the IPIP assesses the Big misspecification while other AFIs were more Five personality dimensions, we utilized factor sensitive to measurement model misspecifications. loadings and factor correlations from only four of They suggested reporting multiple AFIs. these factors, in order to keep the model of realistic and manageable size. Thus, our initial data contained Unmodeled Factors 16 items with four items loading onto each of the In sum, while many excellent simulation four factors. These factor loadings were treated as studies on AFIs have been conducted recently, none error-free population parameters for our data (see have evaluated the potential impact of an unmodeled Table 1). Item uniqueness terms were created such factor. There are several instances in which such a that item variance was equal to unity. We also factor may be present. First, an unmodeled factor introduced nominal model misfit by simulating cross- affecting several items may be present when several loadings for items onto all three secondary factors. related measures of a broader construct are modeled. These cross-loadings were sampled from a normal For instance, a general mental ability factor may be distribution (µ=0.0, σ=.05) such that the expected present among a group of related but distinct ability range of such cross-loadings was between -.10 and tests (e.g., mechanical, quantitative). Similarly, some +.10. Once population data were simulated, sampling models of assessment center performance have been error was introduced into 500 sample replications. put forth in which a general performance factor that Because we were interested in two types of affects each post-exercise dimension rating (Lance, model parameters that may affect fit; number of Lambert, Gewin, Lievens, & Conway, 2004) though items and number of factors, multiple data conditions such factors are typically omitted from the model. were created. As described, our primary data Similarly, halo or partial halo rating errors may be consisted of 16 items with four items loading onto represented by general rating tendency factors that each of four factors. To investigate the effect of affect multiple performance dimension items number of items on AFIs, we also simulated data (Solomonson & Lance, 1997). Lastly, common using 32 items and four factors, with eight indicators methods variance present among a group of similarly per factor. In order to create the additional items, we measured variables can be represented as an simply applied the same factor loadings used in the unmodeled factor (cf. Podsakoff & MacKenzie, primary sixteen-item data to the additional eight 1994; Podsakoff, MacKenzie, Moorman, & Fetter, indicators. One issue that arose, however, is that the 1990). In sum, there are several common situations in addition of items resulted in data that were more organizational research in which an unmodeled latent reliable than in the primary data. Differences in factor factor may impact a measurement model, yet the reliability can influence the precision of estimated impact of such a factor on model fit is unknown. model parameters (Meade & Bauer, in press). Fornell and Larcker (1981) give factor reliability as: Current Study p The current sought to determine the extent ∑ λ ip to which the presence of an unmodeled factor with ρη = p i =1 (1) significant cross-loadings on observed items would p affect CFA model fit. In order to evaluate the ∑ λ ip + ∑ var (ε i ) performance of the chi-square and AFIs, a baseline i =1 i =1 model with good (but imperfect) population model fit where λip indicates the factor loadings of the was first specified. Next, a series of models with indicators of the factor and εi is the item uniqueness. sequentially increasing levels of model In order to equalize reliability for each of the four misspecification were simulated. Model fit statistics factors, the 32 item factor loadings were created by were then recorded as were the percentage of subtracting a constant from item factor loadings in AFIs and CFA Misfit 3 the 16-item condition. This constant varied across the For our smallest level of misspecification, a single four factors but was the same for all items within a item loaded onto both its original factor and the given factor. The value of the constant was derived unmodeled factor. In order to create the via an iterative process using the Generalized misspecification, the magnitude of the original factor Reduced Gradient (GRG2) nonlinear optimization loading was split between the original factor and the implemented in Microsoft Excel’s “Solver” unmodeled factor (cf. Butts, Vandenberg, & subroutine (see Fylstra, Lasdon, Watson, & Waren, Williams, 2006). For example, Item 2’s population 1998 for a description). As a result, item factor factor loading value was .76 in baseline data. For loadings in the 32-item condition were lower than misspecified data, the factor loading was .38 for both their 16-item counterparts, but the reliability of each the original factor (Factor 1) and the unmodeled factor was equal across the two conditions. As with factor (with item uniqueness terms, and therefore the 16-item data, item uniqueness terms were item variance, being equal in both misspecified and simulated for 32-item data such that each item had a baseline data). For subsequent levels of variance of 1.0. Population item factor loadings for misspecification, one additional item per condition the 32-item data appear in Table 2. was specified to load onto both its original factor and Number of factors. Having generated two the unmodeled factor. These items are referred to as sets of population parameters, number of factors (2 or misspecified items. An equal number of items per 4) was manipulated by either selecting all four factors factor were designated as misspecified items to the for analysis in the CFA model, or by only analyzing extent possible (i.e., for 8 misspecified items, there items that loaded onto the first two of the four were two misspecified items from each of the four factors. As a result, there were four potential factors). Note that reference indicators were used in conditions of number of factors and number of this study to provide a metric for the latent variables indicators (with these two variables not being fully during the data analyses. In all study conditions, crossed). Note that the reliability of the factors were factor loadings for reference indicators were held equal across all four conditions: constant in the population. We then simulated up to 66% of the freely estimated factor loadings as Condition A: 4 items for each of 4 factors (i.e., 16 misspecified items per condition. Nested within each items and 4 factors) of these conditions were the six sample size conditions described earlier. Condition B: 8 items for each of 4 factors (i.e., 32 items and 4 factors) Analyses A CFA model was estimated in which the Condition C: 4 items for each of 2 factors (i.e., 8 population baseline model factor structure was items and 2 factors) specified. Covariance matrices were analyzed with the first item in each factor serving as the reference Condition D: 8 items for each of 2 factors (i.e., 16 indicator. ML estimation was used for all analyses items and 2 factors) using LISREL 8.53 (Jöreskog & Sörbom, 1996). AFIs. We examined the chi-square statistic Study Variables and several AFIs recommended by Hu and Bentler We manipulated several study variables: (1999). Specifically, We examined the TLI (NNFI), sample size, number of factors, number of indicators, IFI (BL89), RNI, CFI, Gamma-Hat, McDonald’s and the level of misspecification. Sample size Centrality Index (Mc), SRMR, and RMSEA. conditions included 100, 200, 400, 800, 1600, and The extent to which each AFI was 6400. These values were chosen in order to represent influenced by amount of misspecification, sample both commonly occurring sample sizes and large size, and model complexity was evaluating via sample sizes that would likely give rise to a highly ANOVAs using SAS’s Proc GLM. In each model, sensitive chi-square statistic. With such large sample the AFI was entered as the dependent variable, with sizes (as sometimes encountered in organizational the number of misspecified items, sample size, research), researchers may be more likely to rely on number of factors, and number of items as the AFIs rather than chi-square indices to evaluate model predictors. We then calculated ω2 effect size fit. As such, there were a total of four baseline estimates for these variables as well as the population models (Conditions A-D), with six sample interactions among them. AFIs were considered to be size conditions simulated for each. optimal if they displayed large ω2 values for level of In order to manipulate varying magnitudes misspecification and small ω2 values for other study of misspecification, we created an unmodeled variables. We evaluated redundancy between AFIs by additional factor upon which items could cross-load. computing correlations among the AFIs. AFIs and CFA Misfit 4 Perhaps the most important criteria is the misspecified items, this number represents Type I extent to which researchers would accurately error. The number of misspecified items is depicted conclude that misspecification was present in their on the X axis. The slope of the S-shaped curves data. While not intended, the recommendations for represents the sensitivity of the fit index to detect adequate model fit put forth by Hu and Bentler misspecification. Ideally, with no misspecified items, (1999) have turned into de-facto significance tests the percentage of replications in which misfit was (Marsh et al., 2004) in which failure to reach the indicated will be at the nominal level (i.e., < 5%). values specified leads to the conclusion of inadequate However, the curve should then rise sharply to fit. For each replication in each condition, we indicate misfit when misspecified items were computed whether or not the AFI in question simulated. Also, ideally power curves should be exceeded the AFI values recommended by Hu and similar across conditions and sample sizes. Bentler (1999). Hu and Bentler proposed an AFI As can be seen in Figure 1, chi-square was value greater than or equal to .95 be used for TLI, very sensitive to sample size, with Type I error rates IFI, CFI, RNI, and Gamma-hat, while .90 was at 100% with sample sizes of 6400. The very minor recommended for Mc. SRMR less than or equal to cross-loadings simulated in the data were enough to .08 and RMSEA less than or equal to .06 were also indicate misfit using chi-square for large sample examined. sizes. Type I error was also high in Condition B across all sample sizes. The effect of sample size on Results chi-square behavior is obvious and not ideal. Figure 2 reveals somewhat more desirable Convergence errors or inadmissible performance for TLI. Type I error rates are typically solutions were present in less than 2% of the low (with the exception of Condition B, N=100) and replications in each of the conditions, with condition power curves tend to have a large slope. However, B having somewhat more than the other three the location of the power curves is shifted to the right conditions. Additionally, these were somewhat more somewhat more than would be desired. In other common for small sample sizes, yet never exceeded words, a sizable percentage of items can be 4% in any one condition. Replications in which misspecified yet the TLI would continue to indicate estimation errors occurred were removed from adequate fit. In condition B, up to 6 items could load further analyses. onto an unspecified factor yet TLI would be > .95 Results of the decomposition of variance can when sample size is 800 or greater. Given the high be found in Table 3. As can be seen, no index was correlation between the indices, it is unsurprising that particularly sensitive to model misspecification while Figures 3-5 show very similar findings for CFI, IFI, several were sensitive to either sample size or an and RNI, respectively. interaction between sample size and other simulated Figure 6 presents the results for gamma-hat. conditions. As expected, chi-square and SRMR were Echoing the findings in Table 3, gamma-hat was very particularly egregious offenders with respect to insensitive to this type of misspecification. sample size. Among AFIs, Mc displayed the largest While Table 3 suggested that Mc was most effect size due to misspecification and among the sensitive to misspecification, the strong influence of smallest due to sample size. However, it was more number of factors on Mc can be found in Figure 7. strongly affected by number of factors than were Mc had somewhat high Type I error rates in other indices. RMSEA had no sensitivity whatsoever condition A and very high Type I error rates in due to misspecification and was most strongly Condition B with small sample sizes. Conversely, affected by the number of items simulated. power was very low in Conditions C and D. Correlations among the fit indices can be Figure 8 shows that SRMR is very poorly found in Table 4. As can be seen, several of the suited to detect this type of misspecification. Number indices correlated nearly perfectly (TLI, CFI, IFI, of samples in which misspecification was detected RNI). As such, very little information is to be gained was extremely low for nearly all conditions, except by reporting more than one of these indices. Gamma- that of Condition B with N=100. For that condition, hat also correlated between .94 and .95 with these Type I error was in excess of 95%. indices. The relationship between chi-square and Finally, RMSEA performed very poorly in AFIs was somewhat low. conditions B and D, nearly always indicating The percentage of replications exceeding the adequate fit. Conversely, while shifted somewhat Hu and Bentler (1999) recommended AFI values for more to the right than would be desired, power good fit can be found in Figures 1-9. In these figures, curves had excellent slopes in Conditions A and C. power is the percentage of samples in which poor fit was indicated, though for conditions of zero AFIs and CFA Misfit 5 Discussion such indices in broader CFA and SEM research. While several previous studies have established criteria for acceptable model fit using AFIs for various types of model misspecification, this study was the first to examine the effect of References misspecification due to an unmodeled factor. Based on an extensive simulation of misspecification of Barrett, P. (2007). Structural equation modelling: factor covariances and cross-loading items, Hu and Adjudging model fit. Personality and Bentler (1999) recommended several AFIs that could Individual Differences, 42, 815-824. be used to successfully highlight model Beauducel, A., & Wittmann, W. W. (2005). misspecification. Using those recommended values, Simulation Study on Fit Indexes in CFA we evaluated the percentage of replications in which Based on Data With Slightly Distorted simulated misspecification due to an unmodeled Simple Structure. Structural Equation factor was identified. We found inadequate Modeling, 12, 41-75. performance for chi-square, as well as the AFIs we Butts, M. M., Vandenberg, R. J., & Williams, L. J. examined: TLI, CFI, IFI, RNI, Gamma-hat, Mc, (2006, August). Investigating susceptibility SRMR, and RMSEA. Most indices showed of measurement invariance tests: The somewhat inadequate sensitivity to misspecification effects of common method variance. Paper with many also showing sensitivity to sample size presented at the annual meeting of the (e.g., chi-square, SRMR), numbers of factors (e.g., Academy of Management, Atlanta, GA. Mc), numbers of items (e.g., RMSEA), or some Donnellan, M. B., Oswald, F. L., Baird, B. M., & combination of these model conditions. Among those Lucas, R. E. (2006). The Mini-IPIP Scales: indices not excessively prone to the influence of Tiny-Yet-Effective Measures of the Big these simulated conditions (e.g., TLI, CFI, IFI, and Five Factors of Personality. Psychological RNI), power was still quite low to detect Assessment, 18, 192-203. misspecification unless the percentage of items that Fan, X., & Sivo, S. A. (2005). Sensitivity of Fit was misspecified was quite high (typically > 30%). Indexes to Misspecified Structural or Measurement Model Components: Rationale Summary and Recommendations of Two-Index Strategy Revisited. Structural Given the poor performance of both chi- Equation Modeling, 12, 343-367. square and the AFIs examined in this study, CFA Fornell, C., & Larcker, D. F. (1981). Evaluating cannot be recommended to reliably indicate whether structural equation models with an unmodeled factor is present in the data. While we unobservable variables and measurement did not examine EFA directly in this study, our error. Journal of Marketing Research, results suggest that if there is any doubt as to the 18(February), 39-50. appropriate number of factors underlying a dataset, Fylstra, D., Lasdon, L., Watson, J., & Waren, A. EFA should be conducted prior to CFA to ensure that (1998). Design and use of the Microsoft the correct number of factors are modeled. Note that Excel Solver. INTERFACES, 28 (5), 29-55. in this study, our unmodeled factor had cross- Goldberg, L. R. (1999). A broad-bandwidth, public loadings with items that also loaded onto their domain, personality inventory measuring the specified factor. This scenario might be indicative of lower-level facets of several five-factor one in which a general ability underlies a series of models. In I. Mervielde, I. Deary, F. D. related ability tests, halo error in performance ratings, Fruyt & F. Ostendorf (Eds.), Personality or common methods effects in survey responses. Psychology In Europe (Vol. 7, pp. 7-28). Despite the poor performance of the AFIs in Tilburg, The Netherlands: Tilburg the current study, we do not support Barrett’s (2007) University Press. recent call to abandon AFIs. Rather, our results for Hinkin, T. R. (1998). A brief tutorial on the the chi-square statistic indicate exactly why AFIs are development of measures for use in survey needed. With very large sample sizes sometime questionnaires. Organizational Research found in organizational research, the chi-square test Methods, 1, 104-121. was significant nearly 100% of the time even when Hu, L. T., & Bentler, P. M. (1998). Fit indices in no misspecification was simulated beyond the covariance structure modeling: Sensitivity to nominal misfit of trivially small cross-loadings of underparameterized model misspecification. items. Even though AFIs did not perform well under Psychological Methods, 3, 424-453. the current study conditions, we see a great need for AFIs and CFA Misfit 6 Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for H., & Fetter, R. (1990). Transformational fit indexes in covariance structure analysis: leader behaviors and their effects on Conventional criteria versus new followers' trust in leader, satisfaction, and alternatives. Structural Equation Modeling, organizational citizenship behaviors. 6, 1-55. Leadership Quarterly, 1, 107-142. Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: Solomonson, A. L., & Lance, C. E. (1997). Users Reference Guide. Chicago: Scientific Examination of the relationship between Software International. true halo and halo error in performance Lance, C. E., Lambert, T. A., Gewin, A. G., Lievens, ratings. Journal of Applied Psychology, 82, F., & Conway, J. M. (2004). Revised 665-674. estimates of dimension and exercise Steiger, J. H. (2007). Understanding the limitations of variance components in assessment center global fit assessment in structural equation postexercise dimension ratings. Journal of modeling. Personality and Individual Applied Psychology, 89, 377-385. Differences, 42, 893-898. Marsh, H., Balla, J. R., & Hau, K. (1996). An Tanaka, J. S. (1993). Multifaceted conceptions of fit evaluation of incremental fit indices: a in structure equation models. In K. A. clarification of mathematical and empirical Bollen & J. S. Long (Eds.) Testing structural properties. In G. A. Marcoulides & R. E. equation models (pp. 136-162). Newbury Schumacker (Eds.), Advanced structural Park, CA: Sage. equation modeling: Issues and techniques. Mahwah NJ: Erlbaum. Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers Author Contact Info in Overgeneralizing Hu and Bentler's (1999) Findings. Structural Equation Modeling, 11, Adam W. Meade 320-341. Department of Psychology Meade, A. W., & Bauer, D. J. (in press). Power and North Carolina State University precision in confirmatory factor analytic Campus Box 7801 tests of measurement invariance. Structural Raleigh, NC 27695-7801 Equation Modeling. Phone: 919-513-4857 Podsakoff, P. M., & MacKenzie, S. B. (1994). An Fax: 919-515-1716 examination of the psychometric properties E-mail: adam_meade@ncsu.edu and nomological validity of some revised and reduced substitutes for leadership scales. Journal of Applied Psychology, 79, 702-713. Podsakoff, P. M., MacKenzie, S. B., Moorman, R. 2 Table 1 Population Factor Loadings for Baseline Model – Sixteen-Item Condition Item Extraversion Agreeableness Neuroticism Cons 1 0.68 -0.03 -0.02 -0.11 2 0.76 0.02 -0.03 -0.03 3 0.74 -0.04 -0.03 0.00 4 0.75 0.00 -0.01 0.08 5 0.04 0.76 0.07 0.00 6 -0.06 0.56 -0.03 0.04 7 -0.08 0.75 0.07 0.06 8 -0.02 0.72 0.05 -0.03 9 0.07 -0.01 0.80 0.00 10 -0.01 -0.03 0.58 -0.02 11 -0.04 0.06 0.80 0.03 12 0.04 0.00 0.39 0.05 13 -0.02 -0.02 -0.01 0.65 14 0.00 -0.13 -0.03 0.67 15 0.00 0.03 -0.01 0.59 16 0.00 0.03 0.02 0.67 Table 2 Population Factor Loadings for Baseline Model –Thirty Two-Item Condition Item Extraversion Agreeableness Neuroticism Cons 1 0.540 -0.03 -0.02 -0.11 2 0.620 0.02 -0.03 -0.03 3 0.600 -0.04 -0.03 0.00 4 0.610 0.00 -0.01 0.08 5 0.040 0.609 0.07 0.00 6 -0.060 0.409 -0.03 0.04 7 -0.080 0.599 0.07 0.06 8 -0.020 0.569 0.05 -0.03 9 0.070 -0.01 0.650 0.00 10 -0.010 -0.03 0.430 -0.02 11 -0.040 0.06 0.650 0.03 12 0.040 0.00 0.240 0.05 13 -0.020 -0.02 -0.01 0.505 14 0.000 -0.13 -0.03 0.525 15 0.000 0.03 -0.01 0.445 16 0.000 0.03 0.02 0.525 Note: Note, Items 17-32 are identical to items 1-16. 3 Table 3. Effects of Simulation Variables on Fit Index Chi- Gamma- Study Variable Square TLI CFI IFI RNI Hat Mc SRMR RMSEA Misspecification (M) 0.14 0.24 0.28 0.29 0.28 0.15 0.35 0.06 0.00 Sample Size (N) 0.49 0.11 0.12 0.10 0.11 0.12 0.11 0.67 0.01 Number of Factors (J) 0.07 0.04 0.05 0.05 0.05 0.10 0.17 0.03 0.00 Number of Items (K) 0.01 0.08 0.05 0.06 0.05 0.07 0.01 0.05 0.39 M*N 0.12 0.04 0.05 0.04 0.05 0.02 0.02 0.00 0.00 M*J 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 M*K 0.01 0.08 0.06 0.07 0.06 0.09 0.02 0.06 0.12 N*J 0.05 0.01 0.01 0.01 0.01 0.02 0.02 0.00 0.00 N*K 0.00 0.02 0.02 0.02 0.02 0.04 0.05 0.02 0.00 M*N*J 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 M*N*K 0.01 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 R-Square 0.91 0.62 0.64 0.63 0.64 0.61 0.75 0.89 0.52 Table 4. Correlations among Fit Indices. Gamma- Chi-Square TLI CFI IFI RNI hat Mc SRMR RMSEA Chi-Square 1.00 TLI -0.19 1.00 CFI -0.20 0.99 1.00 IFI -0.22 1.00 1.00 1.00 RNI -0.21 1.00 1.00 1.00 1.00 Gamma-hat -0.18 0.95 0.94 0.95 0.94 1.00 Mc -0.31 0.84 0.87 0.88 0.88 0.88 1.00 SRMR -0.18 -0.72 -0.71 -0.70 -0.71 -0.74 -0.63 1.00 RMSEA 0.09 -0.62 -0.56 -0.58 -0.57 -0.65 -0.28 0.37 1.00 4 Figure 1. Performance of chi-square under conditions of model misspecification. Chi-Square, Condition A Chi-Square, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items Chi-Square, Condition C Chi-Square, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 5 Figure 2. Performance of TLI under conditions of model misspecification. TLI, Condition A TLI, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items TLI, Condition C TLI, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 6 Figure 3. Performance of CFI under conditions of model misspecification. CFI, Condition A CFI, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items CFI, Condition C CFI, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 7 Figure 4. Performance of IFI under conditions of model misspecification. IFI, Condition A IFI, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items IFI, Condition C IFI, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 8 Figure 5. Performance of RNI under conditions of model misspecification. RNI, Condition A RNI, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items RNI, Condition C RNI, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 9 Figure 6. Performance of Gamma-hat under conditions of model misspecification. Gamma-Hat, Condition A Gamma-Hat, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items Gamma-Hat, Condition C Gamma-Hat, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 10 Figure 7. Performance of Mc under conditions of model misspecification. Mc, Condition A Mc, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items Mc, Condition C Mc, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 11 Figure 8. Performance of SRMR under conditions of model misspecification. SRMR, Condition A SRMR, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items SRMR, Condition C SRMR, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items 12 Figure 9. Performance of RMSEA under conditions of model misspecification. RMSEA, Condition A RMSEA, Condition B 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 2 4 6 8 0 2 4 6 8 10 12 14 16 18 20 # Misspecified Items # Misspecified Items RMSEA, Condition C RMSEA, Condition D 1.0 1.0 0.8 100 0.8 100 200 200 Power Power 0.6 0.6 400 400 0.4 800 0.4 800 1600 1600 0.2 6400 0.2 6400 0.0 0.0 0 1 2 3 4 0 2 4 6 8 10 # Misspecified Items # Misspecified Items