; Power of AFIs to Detect CFA Model Misfit
Documents
Resources
Learning Center
Upload
Plans & pricing Sign in
Sign Out
Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

Power of AFIs to Detect CFA Model Misfit

VIEWS: 52 PAGES: 17

  • pg 1
									Meade, A. W. (2008, April). Power of AFIs to Detect CFA Model Misfit. Paper presented at the 23th Annual Conference of the Society for
Industrial and Organizational Psychology, San Francisco, CA.




                          Power of AFIs to Detect CFA Model Misfit

                                                       Adam W. Meade
                                               North Carolina State University

                Hu and Bentler (1999) have derived guidelines for approximate fit indices (AFIs) that are
                indicative of adequate model fit. The current study evaluated these guidelines for data in which an
                unmodeled factor was present. Results indicated poor power to detect model misspecification for
                all AFIs examined.




          Confirmatory factor analysis (CFA) has                           2007). Many of these AFIs are derived from the same
become a primary tool in scale development and                             fit function used to calculate the chi-square statistic
measure       evaluation    in    psychological      and                   (e.g., CFI, IFI, NFI, RNI, TLI), while others index
organizational research (Hinkin, 1998). While there                        average discrepancy between reproduced and
are several methods of evaluating CFA model fit, the                       observed correlations (e.g., RMSR). Excellent
excessive sensitivity of the chi-square statistic has led                  overviews of the AFIs are available in the extant
to the development and application of many                                 literature (e.g., Hu & Bentler, 1998; Marsh et al.,
approximate fit indices (AFIs; see Marsh, Bella, and                       1996; Tanaka, 1993) and will not be discussed in
Hau, 1996). Previous simulation studies have                               detail.
indicated that reasonable assurance of adequate                                      There have been many simulation studies
model fit may be found when some fit indices are                           over the years in which various model
meet or exceed particular values. A recent such study                      misspecification has been simulated in order to
by Hu and Benter (1999) has been enormously                                determine the performance of the AFIs. The most
influential, cited over 2000 times (as of September                        influential article on AFI performance has been that
10, 2007), to the extent that Barrett (2007) has                           by Hu and Bentler (1999). In their study,
recently stated that this work has become the “bible”                      misspecification was based on improperly
of the “Golden Rules” of fit.                                              constrained factor covariances or factor cross-
          In their simulation work, Hu and Bentler                         loadings. Based on these simulations, Hu and Bentler
(1999) simulated several types of model                                    proposed evaluating model fit using SRMR values
misspecification in order to derive their suggested                        less than or equal to .08 as that index was most
AFI values that indicate acceptable fit. These                             sensitive to misspecification among factor
misspecifications included improperly specified                            covariances. Additionally they suggested evaluating
covariances among latent factors and improperly                            model fit using one of the indices more sensitive to
constrained factor loadings (i.e., erroneously omitting                    factor loading misspecification. These include TLI,
cross-loadings). Despite the sizable nature of their                       IFI, CFI, RNI, and Gamma-hat values greater than or
simulations, they failed to investigate the effect of                      equal to .95, while .90 was recommended for Mc,
model misspecification due to an unmodeled latent                          with a recommended RMSEA value less than or
factor. In this study, we simulate a number of models                      equal to .06.
with a misspecified factor structure in which an                                     More recently, other authors have continued
additional latent variable is omitted, to evaluate                         to index the performance of AFIs under certain data
behavior of both the chi-square statistic and AFIs.                        conditions. For instance, Marsh, Hau, and Wen
                                                                           (2004) sought to replicate many of Hu and Benter’s
Prior Fit Index Simulation Research                                        (1999) conditions. They came to the general
          The excessive sensitivity of the chi-square                      conclusion that the Hu and Bentler recommended
statistic with large samples has been known for some                       cutoffs were somewhat conservative for some types
time, which rapidly gave rise to the development of                        of models. Beauducel and Wittmann (2005) sought to
several AFIs in order to better index the extent to                        simulate data more similar to personality data than
which models “approximately” fit the data (Steiger,                        were Hu and Bentler’s (1999) simulations. They used
                                                AFIs and CFA Misfit                                                2


lower correlations and factor loadings than Hu &             replications that exceeded the values suggested by Hu
Bentler (i.e., lower communality and factor                  and Bentler (1999).
reliability) and simulated data with secondary
loadings on modeled factors. They found that misfit                                 Method
under typical conditions encountered in personality
research may be less likely to be detected with                        An initial structural model was developed
RMSEA and RMSR than other AFIs.                              for four correlated factors. Factor loadings and factor
          Fan & Silvo (2005) did not evaluate                correlations were taken from a recently published
individual AFIs, but rather the strategy put forth by        study (Donnellan, Oswald, Baird, & Lucas, 2006) of
Hu and Benter (1999) of reporting two indices. Their         a short form of a commonly used personality survey,
results question Hu and Benter’s finding that RMSR           the International Personality Item Pool (IPIP;
was more sensitive to factor covariance                      Goldberg, 1999). While the IPIP assesses the Big
misspecification while other AFIs were more                  Five personality dimensions, we utilized factor
sensitive to measurement model misspecifications.            loadings and factor correlations from only four of
They suggested reporting multiple AFIs.                      these factors, in order to keep the model of realistic
                                                             and manageable size. Thus, our initial data contained
Unmodeled Factors                                            16 items with four items loading onto each of the
          In sum, while many excellent simulation            four factors. These factor loadings were treated as
studies on AFIs have been conducted recently, none           error-free population parameters for our data (see
have evaluated the potential impact of an unmodeled          Table 1). Item uniqueness terms were created such
factor. There are several instances in which such a          that item variance was equal to unity. We also
factor may be present. First, an unmodeled factor            introduced nominal model misfit by simulating cross-
affecting several items may be present when several          loadings for items onto all three secondary factors.
related measures of a broader construct are modeled.         These cross-loadings were sampled from a normal
For instance, a general mental ability factor may be         distribution (µ=0.0, σ=.05) such that the expected
present among a group of related but distinct ability        range of such cross-loadings was between -.10 and
tests (e.g., mechanical, quantitative). Similarly, some      +.10. Once population data were simulated, sampling
models of assessment center performance have been            error was introduced into 500 sample replications.
put forth in which a general performance factor that                   Because we were interested in two types of
affects each post-exercise dimension rating (Lance,          model parameters that may affect fit; number of
Lambert, Gewin, Lievens, & Conway, 2004) though              items and number of factors, multiple data conditions
such factors are typically omitted from the model.           were created. As described, our primary data
Similarly, halo or partial halo rating errors may be         consisted of 16 items with four items loading onto
represented by general rating tendency factors that          each of four factors. To investigate the effect of
affect multiple performance dimension items                  number of items on AFIs, we also simulated data
(Solomonson & Lance, 1997). Lastly, common                   using 32 items and four factors, with eight indicators
methods variance present among a group of similarly          per factor. In order to create the additional items, we
measured variables can be represented as an                  simply applied the same factor loadings used in the
unmodeled factor (cf. Podsakoff & MacKenzie,                 primary sixteen-item data to the additional eight
1994; Podsakoff, MacKenzie, Moorman, & Fetter,               indicators. One issue that arose, however, is that the
1990). In sum, there are several common situations in        addition of items resulted in data that were more
organizational research in which an unmodeled latent         reliable than in the primary data. Differences in factor
factor may impact a measurement model, yet the               reliability can influence the precision of estimated
impact of such a factor on model fit is unknown.             model parameters (Meade & Bauer, in press). Fornell
                                                             and Larcker (1981) give factor reliability as:
Current Study                                                                         p      
         The current sought to determine the extent                                   ∑ λ ip 
                                                                                             
to which the presence of an unmodeled factor with
                                                                          ρη = p      i =1                     (1)
significant cross-loadings on observed items would
                                                                                       p
affect CFA model fit. In order to evaluate the                                 ∑ λ ip  + ∑ var (ε i )
                                                                                      
performance of the chi-square and AFIs, a baseline                             i =1  i =1
model with good (but imperfect) population model fit         where λip indicates the factor loadings of the
was first specified. Next, a series of models with           indicators of the factor and εi is the item uniqueness.
sequentially     increasing   levels   of     model          In order to equalize reliability for each of the four
misspecification were simulated. Model fit statistics        factors, the 32 item factor loadings were created by
were then recorded as were the percentage of                 subtracting a constant from item factor loadings in
                                                AFIs and CFA Misfit                                               3


the 16-item condition. This constant varied across the       For our smallest level of misspecification, a single
four factors but was the same for all items within a         item loaded onto both its original factor and the
given factor. The value of the constant was derived          unmodeled factor. In order to create the
via an iterative process using the Generalized               misspecification, the magnitude of the original factor
Reduced Gradient (GRG2) nonlinear optimization               loading was split between the original factor and the
implemented in Microsoft Excel’s “Solver”                    unmodeled factor (cf. Butts, Vandenberg, &
subroutine (see Fylstra, Lasdon, Watson, & Waren,            Williams, 2006). For example, Item 2’s population
1998 for a description). As a result, item factor            factor loading value was .76 in baseline data. For
loadings in the 32-item condition were lower than            misspecified data, the factor loading was .38 for both
their 16-item counterparts, but the reliability of each      the original factor (Factor 1) and the unmodeled
factor was equal across the two conditions. As with          factor (with item uniqueness terms, and therefore
the 16-item data, item uniqueness terms were                 item variance, being equal in both misspecified and
simulated for 32-item data such that each item had a         baseline data). For subsequent levels of
variance of 1.0. Population item factor loadings for         misspecification, one additional item per condition
the 32-item data appear in Table 2.                          was specified to load onto both its original factor and
          Number of factors. Having generated two            the unmodeled factor. These items are referred to as
sets of population parameters, number of factors (2 or       misspecified items. An equal number of items per
4) was manipulated by either selecting all four factors      factor were designated as misspecified items to the
for analysis in the CFA model, or by only analyzing          extent possible (i.e., for 8 misspecified items, there
items that loaded onto the first two of the four             were two misspecified items from each of the four
factors. As a result, there were four potential              factors). Note that reference indicators were used in
conditions of number of factors and number of                this study to provide a metric for the latent variables
indicators (with these two variables not being fully         during the data analyses. In all study conditions,
crossed). Note that the reliability of the factors were      factor loadings for reference indicators were held
equal across all four conditions:                            constant in the population. We then simulated up to
                                                             66% of the freely estimated factor loadings as
Condition A: 4 items for each of 4 factors (i.e., 16         misspecified items per condition. Nested within each
items and 4 factors)                                         of these conditions were the six sample size
                                                             conditions described earlier.
Condition B: 8 items for each of 4 factors (i.e., 32
items and 4 factors)                                         Analyses
                                                                       A CFA model was estimated in which the
Condition C: 4 items for each of 2 factors (i.e., 8          population baseline model factor structure was
items and 2 factors)                                         specified. Covariance matrices were analyzed with
                                                             the first item in each factor serving as the reference
Condition D: 8 items for each of 2 factors (i.e., 16         indicator. ML estimation was used for all analyses
items and 2 factors)                                         using LISREL 8.53 (Jöreskog & Sörbom, 1996).
                                                                       AFIs. We examined the chi-square statistic
Study Variables                                              and several AFIs recommended by Hu and Bentler
         We manipulated several study variables:             (1999). Specifically, We examined the TLI (NNFI),
sample size, number of factors, number of indicators,        IFI (BL89), RNI, CFI, Gamma-Hat, McDonald’s
and the level of misspecification. Sample size               Centrality Index (Mc), SRMR, and RMSEA.
conditions included 100, 200, 400, 800, 1600, and                      The extent to which each AFI was
6400. These values were chosen in order to represent         influenced by amount of misspecification, sample
both commonly occurring sample sizes and large               size, and model complexity was evaluating via
sample sizes that would likely give rise to a highly         ANOVAs using SAS’s Proc GLM. In each model,
sensitive chi-square statistic. With such large sample       the AFI was entered as the dependent variable, with
sizes (as sometimes encountered in organizational            the number of misspecified items, sample size,
research), researchers may be more likely to rely on         number of factors, and number of items as the
AFIs rather than chi-square indices to evaluate model        predictors. We then calculated ω2 effect size
fit. As such, there were a total of four baseline            estimates for these variables as well as the
population models (Conditions A-D), with six sample          interactions among them. AFIs were considered to be
size conditions simulated for each.                          optimal if they displayed large ω2 values for level of
         In order to manipulate varying magnitudes           misspecification and small ω2 values for other study
of misspecification, we created an unmodeled                 variables. We evaluated redundancy between AFIs by
additional factor upon which items could cross-load.         computing correlations among the AFIs.
                                                AFIs and CFA Misfit                                                4


         Perhaps the most important criteria is the          misspecified items, this number represents Type I
extent to which researchers would accurately                 error. The number of misspecified items is depicted
conclude that misspecification was present in their          on the X axis. The slope of the S-shaped curves
data. While not intended, the recommendations for            represents the sensitivity of the fit index to detect
adequate model fit put forth by Hu and Bentler               misspecification. Ideally, with no misspecified items,
(1999) have turned into de-facto significance tests          the percentage of replications in which misfit was
(Marsh et al., 2004) in which failure to reach the           indicated will be at the nominal level (i.e., < 5%).
values specified leads to the conclusion of inadequate       However, the curve should then rise sharply to
fit. For each replication in each condition, we              indicate misfit when misspecified items were
computed whether or not the AFI in question                  simulated. Also, ideally power curves should be
exceeded the AFI values recommended by Hu and                similar across conditions and sample sizes.
Bentler (1999). Hu and Bentler proposed an AFI                         As can be seen in Figure 1, chi-square was
value greater than or equal to .95 be used for TLI,          very sensitive to sample size, with Type I error rates
IFI, CFI, RNI, and Gamma-hat, while .90 was                  at 100% with sample sizes of 6400. The very minor
recommended for Mc. SRMR less than or equal to               cross-loadings simulated in the data were enough to
.08 and RMSEA less than or equal to .06 were also            indicate misfit using chi-square for large sample
examined.                                                    sizes. Type I error was also high in Condition B
                                                             across all sample sizes. The effect of sample size on
                       Results                               chi-square behavior is obvious and not ideal.
                                                                       Figure 2 reveals somewhat more desirable
          Convergence      errors   or    inadmissible       performance for TLI. Type I error rates are typically
solutions were present in less than 2% of the                low (with the exception of Condition B, N=100) and
replications in each of the conditions, with condition       power curves tend to have a large slope. However,
B having somewhat more than the other three                  the location of the power curves is shifted to the right
conditions. Additionally, these were somewhat more           somewhat more than would be desired. In other
common for small sample sizes, yet never exceeded            words, a sizable percentage of items can be
4% in any one condition. Replications in which               misspecified yet the TLI would continue to indicate
estimation errors occurred were removed from                 adequate fit. In condition B, up to 6 items could load
further analyses.                                            onto an unspecified factor yet TLI would be > .95
          Results of the decomposition of variance can       when sample size is 800 or greater. Given the high
be found in Table 3. As can be seen, no index was            correlation between the indices, it is unsurprising that
particularly sensitive to model misspecification while       Figures 3-5 show very similar findings for CFI, IFI,
several were sensitive to either sample size or an           and RNI, respectively.
interaction between sample size and other simulated                    Figure 6 presents the results for gamma-hat.
conditions. As expected, chi-square and SRMR were            Echoing the findings in Table 3, gamma-hat was very
particularly egregious offenders with respect to             insensitive to this type of misspecification.
sample size. Among AFIs, Mc displayed the largest                      While Table 3 suggested that Mc was most
effect size due to misspecification and among the            sensitive to misspecification, the strong influence of
smallest due to sample size. However, it was more            number of factors on Mc can be found in Figure 7.
strongly affected by number of factors than were             Mc had somewhat high Type I error rates in
other indices. RMSEA had no sensitivity whatsoever           condition A and very high Type I error rates in
due to misspecification and was most strongly                Condition B with small sample sizes. Conversely,
affected by the number of items simulated.                   power was very low in Conditions C and D.
          Correlations among the fit indices can be                    Figure 8 shows that SRMR is very poorly
found in Table 4. As can be seen, several of the             suited to detect this type of misspecification. Number
indices correlated nearly perfectly (TLI, CFI, IFI,          of samples in which misspecification was detected
RNI). As such, very little information is to be gained       was extremely low for nearly all conditions, except
by reporting more than one of these indices. Gamma-          that of Condition B with N=100. For that condition,
hat also correlated between .94 and .95 with these           Type I error was in excess of 95%.
indices. The relationship between chi-square and                       Finally, RMSEA performed very poorly in
AFIs was somewhat low.                                       conditions B and D, nearly always indicating
          The percentage of replications exceeding the       adequate fit. Conversely, while shifted somewhat
Hu and Bentler (1999) recommended AFI values for             more to the right than would be desired, power
good fit can be found in Figures 1-9. In these figures,      curves had excellent slopes in Conditions A and C.
power is the percentage of samples in which poor fit
was indicated, though for conditions of zero
                                                  AFIs and CFA Misfit                                               5


                      Discussion                               such indices in broader CFA and SEM research.

          While several previous studies have
established criteria for acceptable model fit using
AFIs for various types of model misspecification, this
study was the first to examine the effect of                                        References
misspecification due to an unmodeled factor. Based
on an extensive simulation of misspecification of              Barrett, P. (2007). Structural equation modelling:
factor covariances and cross-loading items, Hu and                       Adjudging model fit. Personality and
Bentler (1999) recommended several AFIs that could                       Individual Differences, 42, 815-824.
be used to successfully highlight model                        Beauducel, A., & Wittmann, W. W. (2005).
misspecification. Using those recommended values,                        Simulation Study on Fit Indexes in CFA
we evaluated the percentage of replications in which                     Based on Data With Slightly Distorted
simulated misspecification due to an unmodeled                           Simple Structure. Structural Equation
factor was identified. We found inadequate                               Modeling, 12, 41-75.
performance for chi-square, as well as the AFIs we             Butts, M. M., Vandenberg, R. J., & Williams, L. J.
examined: TLI, CFI, IFI, RNI, Gamma-hat, Mc,                             (2006, August). Investigating susceptibility
SRMR, and RMSEA. Most indices showed                                     of measurement invariance tests: The
somewhat inadequate sensitivity to misspecification                      effects of common method variance. Paper
with many also showing sensitivity to sample size                        presented at the annual meeting of the
(e.g., chi-square, SRMR), numbers of factors (e.g.,                      Academy of Management, Atlanta, GA.
Mc), numbers of items (e.g., RMSEA), or some                   Donnellan, M. B., Oswald, F. L., Baird, B. M., &
combination of these model conditions. Among those                       Lucas, R. E. (2006). The Mini-IPIP Scales:
indices not excessively prone to the influence of                        Tiny-Yet-Effective Measures of the Big
these simulated conditions (e.g., TLI, CFI, IFI, and                     Five Factors of Personality. Psychological
RNI), power was still quite low to detect                                Assessment, 18, 192-203.
misspecification unless the percentage of items that           Fan, X., & Sivo, S. A. (2005). Sensitivity of Fit
was misspecified was quite high (typically > 30%).                       Indexes to Misspecified Structural or
                                                                         Measurement Model Components: Rationale
Summary and Recommendations                                              of Two-Index Strategy Revisited. Structural
         Given the poor performance of both chi-                         Equation Modeling, 12, 343-367.
square and the AFIs examined in this study, CFA                Fornell, C., & Larcker, D. F. (1981). Evaluating
cannot be recommended to reliably indicate whether                       structural equation models with
an unmodeled factor is present in the data. While we                     unobservable variables and measurement
did not examine EFA directly in this study, our                          error. Journal of Marketing Research,
results suggest that if there is any doubt as to the                     18(February), 39-50.
appropriate number of factors underlying a dataset,            Fylstra, D., Lasdon, L., Watson, J., & Waren, A.
EFA should be conducted prior to CFA to ensure that                      (1998). Design and use of the Microsoft
the correct number of factors are modeled. Note that                     Excel Solver. INTERFACES, 28 (5), 29-55.
in this study, our unmodeled factor had cross-                 Goldberg, L. R. (1999). A broad-bandwidth, public
loadings with items that also loaded onto their                          domain, personality inventory measuring the
specified factor. This scenario might be indicative of                   lower-level facets of several five-factor
one in which a general ability underlies a series of                     models. In I. Mervielde, I. Deary, F. D.
related ability tests, halo error in performance ratings,                Fruyt & F. Ostendorf (Eds.), Personality
or common methods effects in survey responses.                           Psychology In Europe (Vol. 7, pp. 7-28).
         Despite the poor performance of the AFIs in                     Tilburg, The Netherlands: Tilburg
the current study, we do not support Barrett’s (2007)                    University Press.
recent call to abandon AFIs. Rather, our results for           Hinkin, T. R. (1998). A brief tutorial on the
the chi-square statistic indicate exactly why AFIs are                   development of measures for use in survey
needed. With very large sample sizes sometime                            questionnaires. Organizational Research
found in organizational research, the chi-square test                    Methods, 1, 104-121.
was significant nearly 100% of the time even when              Hu, L. T., & Bentler, P. M. (1998). Fit indices in
no misspecification was simulated beyond the                             covariance structure modeling: Sensitivity to
nominal misfit of trivially small cross-loadings of                      underparameterized model misspecification.
items. Even though AFIs did not perform well under                       Psychological Methods, 3, 424-453.
the current study conditions, we see a great need for
                                                AFIs and CFA Misfit                                               6


Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for                H., & Fetter, R. (1990). Transformational
         fit indexes in covariance structure analysis:                 leader behaviors and their effects on
         Conventional criteria versus new                              followers' trust in leader, satisfaction, and
         alternatives. Structural Equation Modeling,                   organizational citizenship behaviors.
         6, 1-55.                                                      Leadership Quarterly, 1, 107-142.
Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8:              Solomonson, A. L., & Lance, C. E. (1997).
         Users Reference Guide. Chicago: Scientific                    Examination of the relationship between
         Software International.                                       true halo and halo error in performance
Lance, C. E., Lambert, T. A., Gewin, A. G., Lievens,                   ratings. Journal of Applied Psychology, 82,
         F., & Conway, J. M. (2004). Revised                           665-674.
         estimates of dimension and exercise                 Steiger, J. H. (2007). Understanding the limitations of
         variance components in assessment center                      global fit assessment in structural equation
         postexercise dimension ratings. Journal of                    modeling. Personality and Individual
         Applied Psychology, 89, 377-385.                              Differences, 42, 893-898.
Marsh, H., Balla, J. R., & Hau, K. (1996). An                Tanaka, J. S. (1993). Multifaceted conceptions of fit
         evaluation of incremental fit indices: a                      in structure equation models. In K. A.
         clarification of mathematical and empirical                   Bollen & J. S. Long (Eds.) Testing structural
         properties. In G. A. Marcoulides & R. E.                      equation models (pp. 136-162). Newbury
         Schumacker (Eds.), Advanced structural                        Park, CA: Sage.
         equation modeling: Issues and techniques.
         Mahwah NJ: Erlbaum.
Marsh, H. W., Hau, K. T., & Wen, Z. (2004). In
         Search of Golden Rules: Comment on
         Hypothesis-Testing Approaches to Setting
         Cutoff Values for Fit Indexes and Dangers                          Author Contact Info
         in Overgeneralizing Hu and Bentler's (1999)
         Findings. Structural Equation Modeling, 11,                          Adam W. Meade
         320-341.                                                        Department of Psychology
Meade, A. W., & Bauer, D. J. (in press). Power and                     North Carolina State University
         precision in confirmatory factor analytic                            Campus Box 7801
         tests of measurement invariance. Structural                      Raleigh, NC 27695-7801
         Equation Modeling.                                                Phone: 919-513-4857
Podsakoff, P. M., & MacKenzie, S. B. (1994). An                              Fax: 919-515-1716
         examination of the psychometric properties                    E-mail: adam_meade@ncsu.edu
         and nomological validity of some revised
         and reduced substitutes for leadership scales.
         Journal of Applied Psychology, 79, 702-713.
Podsakoff, P. M., MacKenzie, S. B., Moorman, R.
                                                                                       2


Table 1
Population Factor Loadings for Baseline Model – Sixteen-Item Condition
        Item           Extraversion     Agreeableness   Neuroticism    Cons
          1                0.68             -0.03          -0.02       -0.11
          2                0.76              0.02          -0.03       -0.03
          3                0.74             -0.04          -0.03        0.00
          4                0.75              0.00          -0.01        0.08
          5                0.04              0.76           0.07        0.00
          6               -0.06              0.56          -0.03        0.04
          7               -0.08              0.75           0.07        0.06
          8               -0.02              0.72           0.05       -0.03
          9                0.07             -0.01           0.80        0.00
         10               -0.01             -0.03           0.58       -0.02
         11               -0.04              0.06           0.80        0.03
         12                0.04              0.00           0.39        0.05
         13               -0.02             -0.02          -0.01        0.65
         14                0.00             -0.13          -0.03        0.67
         15                0.00              0.03          -0.01        0.59
         16                0.00              0.03           0.02        0.67




Table 2
Population Factor Loadings for Baseline Model –Thirty Two-Item Condition
        Item             Extraversion      Agreeableness     Neuroticism       Cons
          1                 0.540              -0.03            -0.02          -0.11
          2                 0.620               0.02            -0.03          -0.03
          3                 0.600              -0.04            -0.03           0.00
          4                 0.610               0.00            -0.01           0.08
          5                 0.040              0.609             0.07           0.00
          6                 -0.060             0.409            -0.03           0.04
          7                 -0.080             0.599             0.07           0.06
          8                 -0.020             0.569             0.05          -0.03
          9                 0.070              -0.01           0.650            0.00
         10                 -0.010             -0.03           0.430           -0.02
         11                 -0.040              0.06           0.650            0.03
         12                 0.040               0.00           0.240            0.05
         13                 -0.020             -0.02            -0.01          0.505
         14                 0.000              -0.13            -0.03          0.525
         15                 0.000               0.03            -0.01          0.445
         16                 0.000               0.03             0.02          0.525

Note: Note, Items 17-32 are identical to items 1-16.
                                                                                                                              3

Table 3.
Effects of Simulation Variables on Fit Index
                               Chi-                                                 Gamma-
       Study Variable         Square     TLI      CFI       IFI      RNI              Hat         Mc        SRMR      RMSEA
Misspecification (M)           0.14      0.24     0.28     0.29      0.28            0.15         0.35       0.06      0.00
Sample Size (N)                0.49      0.11     0.12     0.10      0.11            0.12         0.11       0.67      0.01
Number of Factors (J)          0.07      0.04     0.05     0.05      0.05            0.10         0.17       0.03      0.00
Number of Items (K)            0.01      0.08     0.05     0.06      0.05            0.07         0.01       0.05      0.39
M*N                            0.12      0.04     0.05     0.04      0.05            0.02         0.02       0.00      0.00
M*J                            0.00      0.00     0.00     0.00      0.00            0.00         0.00       0.00      0.00
M*K                            0.01      0.08     0.06     0.07      0.06            0.09         0.02       0.06      0.12
N*J                            0.05      0.01     0.01     0.01      0.01            0.02         0.02       0.00      0.00
N*K                            0.00      0.02     0.02     0.02      0.02            0.04         0.05       0.02      0.00
M*N*J                          0.00      0.00     0.00     0.00      0.00            0.00         0.00       0.00      0.00
M*N*K                          0.01      0.00     0.00     0.00      0.00            0.00         0.00       0.00      0.00
R-Square                       0.91      0.62     0.64     0.63      0.64            0.61         0.75       0.89      0.52


Table 4.
Correlations among Fit Indices.
                                                                       Gamma-
                Chi-Square     TLI        CFI       IFI       RNI        hat                 Mc          SRMR       RMSEA
 Chi-Square         1.00
 TLI               -0.19       1.00
 CFI               -0.20       0.99       1.00
 IFI               -0.22       1.00       1.00     1.00
 RNI               -0.21       1.00       1.00     1.00       1.00
 Gamma-hat         -0.18       0.95       0.94     0.95       0.94          1.00
 Mc                -0.31       0.84       0.87     0.88       0.88          0.88         1.00
 SRMR              -0.18       -0.72      -0.71    -0.70     -0.71          -0.74        -0.63           1.00
 RMSEA              0.09       -0.62      -0.56    -0.58     -0.57          -0.65        -0.28           0.37        1.00
                                                                                                                                                 4

Figure 1. Performance of chi-square under conditions of model misspecification.


                         Chi-Square, Condition A                                                   Chi-Square, Condition B

           1.0                                                               1.0

           0.8                                               100             0.8                                                           100
                                                             200                                                                           200
   Power




                                                                     Power
           0.6                                                               0.6
                                                             400                                                                           400

           0.4                                               800             0.4                                                           800
                                                             1600                                                                          1600
           0.2                                               6400            0.2                                                           6400

           0.0                                                               0.0
                 0       2        4            6       8                           0       2   4    6   8   10   12   14   16   18    20

                         # Misspecified Items                                                      # Misspecified Items



                         Chi-Square, Condition C                                                   Chi-Square, Condition D

           1.0                                                               1.0

           0.8                                               100             0.8                                                           100
                                                             200                                                                           200
   Power




                                                                     Power
           0.6                                                               0.6
                                                             400                                                                           400

           0.4                                               800             0.4                                                           800
                                                             1600                                                                          1600
           0.2                                               6400            0.2                                                           6400

           0.0                                                               0.0
                     0       1    2        3       4                                   0       2        4        6         8         10

                         # Misspecified Items                                                      # Misspecified Items
                                                                                                                                                   5

Figure 2. Performance of TLI under conditions of model misspecification.


                                 TLI, Condition A                                                       TLI, Condition B

           1.0                                                               1.0

           0.8                                                100            0.8                                                             100
                                                              200                                                                            200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                            400

           0.4                                                800            0.4                                                             800
                                                              1600                                                                           1600
           0.2                                                6400           0.2                                                             6400

           0.0                                                               0.0
                 0       2            4           6       8                        0       2   4    6     8   10   12   14   16   18    20

                         # Misspecified Items                                                      # Misspecified Items



                                 TLI, Condition C                                                       TLI, Condition D

           1.0                                                               1.0

           0.8                                                100            0.8                                                             100
                                                              200                                                                            200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                            400

           0.4                                                800            0.4                                                             800
                                                              1600                                                                           1600
           0.2                                                6400           0.2                                                             6400

           0.0                                                               0.0
                     0       1        2       3       4                                0       2          4        6         8         10

                         # Misspecified Items                                                      # Misspecified Items
                                                                                                                                                  6

Figure 3. Performance of CFI under conditions of model misspecification.


                              CFI, Condition A                                                         CFI, Condition B

           1.0                                                              1.0

           0.8                                              100             0.8                                                             100
                                                            200                                                                             200
   Power




                                                                    Power
           0.6                                                              0.6
                                                            400                                                                             400

           0.4                                              800             0.4                                                             800
                                                            1600                                                                            1600
           0.2                                              6400            0.2                                                             6400

           0.0                                                              0.0
                 0       2          4          6       8                          0       2   4    6     8   10   12   14   16   18    20

                         # Misspecified Items                                                     # Misspecified Items



                              CFI, Condition C                                                         CFI, Condition D

           1.0                                                              1.0

           0.8                                              100             0.8                                                             100
                                                            200                                                                             200
   Power




                                                                    Power
           0.6                                                              0.6
                                                            400                                                                             400

           0.4                                              800             0.4                                                             800
                                                            1600                                                                            1600
           0.2                                              6400            0.2                                                             6400

           0.0                                                              0.0
                     0    1        2       3       4                                  0       2          4        6         8         10

                         # Misspecified Items                                                     # Misspecified Items
                                                                                                                                                   7

Figure 4. Performance of IFI under conditions of model misspecification.


                                 IFI, Condition A                                                        IFI, Condition B

           1.0                                                                1.0

           0.8                                                 100            0.8                                                            100
                                                               200                                                                           200
   Power




                                                                      Power
           0.6                                                                0.6
                                                               400                                                                           400

           0.4                                                 800            0.4                                                            800
                                                               1600                                                                          1600
           0.2                                                 6400           0.2                                                            6400

           0.0                                                                0.0
                 0       2            4            6       8                        0       2   4    6    8   10   12   14   16   18    20

                         # Misspecified Items                                                       # Misspecified Items



                                 IFI, Condition C                                                        IFI, Condition D

           1.0                                                                1.0

           0.8                                                 100            0.8                                                            100
                                                               200                                                                           200
   Power




                                                                      Power
           0.6                                                                0.6
                                                               400                                                                           400

           0.4                                                 800            0.4                                                            800
                                                               1600                                                                          1600
           0.2                                                 6400           0.2                                                            6400

           0.0                                                                0.0
                     0       1        2        3       4                                0       2         4        6         8         10

                         # Misspecified Items                                                       # Misspecified Items
                                                                                                                                                   8

Figure 5. Performance of RNI under conditions of model misspecification.


                                 RNI, Condition A                                                       RNI, Condition B

           1.0                                                               1.0

           0.8                                                100            0.8                                                             100
                                                              200                                                                            200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                            400

           0.4                                                800            0.4                                                             800
                                                              1600                                                                           1600
           0.2                                                6400           0.2                                                             6400

           0.0                                                               0.0
                 0       2            4           6       8                        0       2   4    6     8   10   12   14   16   18    20

                         # Misspecified Items                                                      # Misspecified Items



                                 RNI, Condition C                                                       RNI, Condition D

           1.0                                                               1.0

           0.8                                                100            0.8                                                             100
                                                              200                                                                            200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                            400

           0.4                                                800            0.4                                                             800
                                                              1600                                                                           1600
           0.2                                                6400           0.2                                                             6400

           0.0                                                               0.0
                     0       1        2       3       4                                0       2          4        6         8         10

                         # Misspecified Items                                                      # Misspecified Items
                                                                                                                                                9

Figure 6. Performance of Gamma-hat under conditions of model misspecification.


                         Gamma-Hat, Condition A                                                  Gamma-Hat, Condition B

           1.0                                                             1.0

           0.8                                             100             0.8                                                            100
                                                           200                                                                            200
   Power




                                                                   Power
           0.6                                                             0.6
                                                           400                                                                            400

           0.4                                             800             0.4                                                            800
                                                           1600                                                                           1600
           0.2                                             6400            0.2                                                            6400

           0.0                                                             0.0
                 0       2        4            6       8                         0       2   4     6   8   10   12   14   16   18    20

                         # Misspecified Items                                                    # Misspecified Items



                         Gamma-Hat, Condition C                                                  Gamma-Hat, Condition D

           1.0                                                             1.0

           0.8                                             100             0.8                                                            100
                                                           200                                                                            200
   Power




                                                                   Power
           0.6                                                             0.6
                                                           400                                                                            400

           0.4                                             800             0.4                                                            800
                                                           1600                                                                           1600
           0.2                                             6400            0.2                                                            6400

           0.0                                                             0.0
                     0       1    2        3       4                                 0       2         4        6         8         10

                         # Misspecified Items                                                    # Misspecified Items
                                                                                                                                              10

Figure 7. Performance of Mc under conditions of model misspecification.


                                 Mc, Condition A                                                        Mc, Condition B

           1.0                                                               1.0

           0.8                                                100            0.8                                                            100
                                                              200                                                                           200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                           400

           0.4                                                800            0.4                                                            800
                                                              1600                                                                          1600
           0.2                                                6400           0.2                                                            6400

           0.0                                                               0.0
                 0       2            4           6       8                        0       2   4    6    8   10   12   14   16   18    20

                         # Misspecified Items                                                      # Misspecified Items



                                 Mc, Condition C                                                        Mc, Condition D

           1.0                                                               1.0

           0.8                                                100            0.8                                                            100
                                                              200                                                                           200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                           400

           0.4                                                800            0.4                                                            800
                                                              1600                                                                          1600
           0.2                                                6400           0.2                                                            6400

           0.0                                                               0.0
                     0       1        2       3       4                                0       2         4        6         8         10

                         # Misspecified Items                                                      # Misspecified Items
                                                                                                                                             11

Figure 8. Performance of SRMR under conditions of model misspecification.


                                 SRMR, Condition A                                                   SRMR, Condition B

           1.0                                                               1.0

           0.8                                                100            0.8                                                           100
                                                              200                                                                          200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                          400

           0.4                                                800            0.4                                                           800
                                                              1600                                                                         1600
           0.2                                                6400           0.2                                                           6400

           0.0                                                               0.0
                 0       2             4          6       8                        0       2   4    6   8   10   12   14   16   18    20

                         # Misspecified Items                                                      # Misspecified Items



                                 SRMR, Condition C                                                   SRMR, Condition D

           1.0                                                               1.0

           0.8                                                100            0.8                                                           100
                                                              200                                                                          200
   Power




                                                                     Power
           0.6                                                               0.6
                                                              400                                                                          400

           0.4                                                800            0.4                                                           800
                                                              1600                                                                         1600
           0.2                                                6400           0.2                                                           6400

           0.0                                                               0.0
                     0       1         2      3       4                                0       2        4        6         8         10

                         # Misspecified Items                                                      # Misspecified Items
                                                                                                                                          12

Figure 9. Performance of RMSEA under conditions of model misspecification.


                             RMSEA, Condition A                                                  RMSEA, Condition B

           1.0                                                            1.0

           0.8                                             100            0.8                                                           100
                                                           200                                                                          200
   Power




                                                                  Power
           0.6                                                            0.6
                                                           400                                                                          400

           0.4                                             800            0.4                                                           800
                                                           1600                                                                         1600
           0.2                                             6400           0.2                                                           6400

           0.0                                                            0.0
                 0       2          4          6       8                        0       2   4    6   8   10   12   14   16   18    20

                         # Misspecified Items                                                   # Misspecified Items



                             RMSEA, Condition C                                                  RMSEA, Condition D

           1.0                                                            1.0

           0.8                                             100            0.8                                                           100
                                                           200                                                                          200
   Power




                                                                  Power
           0.6                                                            0.6
                                                           400                                                                          400

           0.4                                             800            0.4                                                           800
                                                           1600                                                                         1600
           0.2                                             6400           0.2                                                           6400

           0.0                                                            0.0
                     0       1      2      3       4                                0       2        4        6         8         10

                         # Misspecified Items                                                   # Misspecified Items

								
To top