All Things in Moderation, Including Tests of Mediation

Document Sample
All Things in Moderation, Including Tests of Mediation Powered By Docstoc
					   All Things in
Moderation, Including
 Tests of Mediation
          Jeffrey B. Vancouver
          Charlie M. Thompson
              Bruce Carlson

                                 1
              Abstract
Covariance tests of mediation are
questioned regarding their internal
validity. Specifically, eight alternative
explanations for a finding of mediation
are described. An alternative method
using a moderation design is proposed.
Implications regarding the potential use,
and advantages and disadvantages of this
approach are described.
                                            2
     Exogenous/                            Endogenous/
                               Mediator
     Independent                            Dependent


                                  a



     Exogenous/                            Endogenous/
     Independent                            Dependent




                              Moderator



                                  b


Figure 1. Mediator and moderator models.                 3
                 Introduction
• Few concepts confuse researchers more readily than the
  distinction between mediation and moderation.
   – Mediation refers to a process or mechanism through which
     one variable (i.e., exogenous) causes variation in another
     variable (i.e., endogenous) (see Figure 1a).
   – Moderation refers to the influence of a process or mechanism
     on the degree or kind of co-variation between the exogenous
     and endogenous variables (see Figure 1b).
• Conceptually and analytically important
   – Mediation is analyzed via differences in partial and whole
     covariances.
   – Moderation is analyzed via interactions.
                                                                    4
• We argue that studies designed to test for moderation
  may provide stronger tests of mediation than the partial
  and whole covariance approaches typically used (e.g.,
  Baron & Kenny, 1986; Bing, Davison, LeBreton, &
  LeBreton, 2002; James & Brett, 1984).
• Problem:
   – The fundamental issue is the weak internal validity of passive
     observational methods (Runkel & McGrath, 1972; Shadish,
     Cook, & Campbell, 2002).
   – Specifically, the issue is that the covariance methods for
     testing mediation are susceptible to numerous alternative
     explanations (Shadish, et al., 2002).
      • Figure 2 shows alternative models when the exogenous/independent
        variable is manipulated.
      • Figure 3 shows alternative models when all variables are passively
        observed.
                                                                             5
• Incidence of problem
   – Use of the covariance methods are widespread, particularly in
     the applied psychological literature (Bing, et al., 2002;
     MacKinnon et al., 2002),
   – whereas use of the moderation method, either experimentally
     or passively, to assess mediation is rare
   – Figure 4 shows the results of a review of the Journal of
     Applied Psychology from 1999 through 2002.
• Potential solution
   – Use moderation (especially manipulated) to assess hypothetical
     mediators (see Figure 5).
   – Consider “automotive science example.”




                                                                      6
           Alternatives Explanations for
            Positive Test of Mediation

         Exogenous/                    Endogenous/
                                                     Mediator
        Independent                     Dependent


A. "Endogenous" variable causes "mediator"                      7
                                                                     Mediator


          Exogenous/
         Independent
                                                                    Endogenous/
                                                                     Dependent

B. "Exogenous" variable causes "mediator" and endogenous variable independently.




                                             Mediator
         Exogenous/
         Independent
                                                                    Endogenous/
                                                                     Dependent

C. "Exogenous" variable causes "mediator" more quickly then it causes the endogenous
    variable.

Figure 2. Alternative models when the exogenous/independent variable is
manipulated.                                                                           8
        Endogenous/                      Exogenous/
                                                                          Mediator
         Dependent                       Independent


A. "Endogenous" variable causes the "exogenous" variable, which causes the "mediator".




        Endogenous/                                                     Exogenous/
                                           Mediator
         Dependent                                                      Independent


B. The “mediator” mediates effect of the "endogenous" variable on the "exogenous" variable.




                                         Endogenous/                    Exogenous/
           Mediator
                                          Dependent                     Independent


C. The “mediator” causes the "endogenous" variable, which causes the "exogenous“ variable.
                                                                                          9
                                        Exogenous/                    Endogenous/
          Mediator
                                        Independent                    Dependent


D. The “mediator” causes "exogenous" variable, which causes the "endogenous" variable.


                                                                       Mediator


                                                                      Exogenous/
         4th variable
                                                                     Independent


                                                                     Endogenous/
                                                                      Dependent

E. Some fourth variable causes two or more of the others in some order other than the one
   hypothesized.

Figure 3. Additional alternative models when no variable is manipulated.


                                                                                            10
                                280 articles



                   76 (27%)                99 (35%)

                   tested for       19     tested for

                   mediation              moderation

                        mentioned the moderator
                          implied a mediator




Figure 4. Incidence of tests of mediation and moderation in the Journal
of Applied Psychology (volumes 84-86).                                    11
                     The Automotive Science Example
Research Question: Why does pressing the gas pedal increase speed of car?

   Hypothesis 1: Drive train mediates relationship between pedal and speed.

   Hypothesis 2: Alternator mediates relationship between pedal and speed.

Results of standard covariance approach to tests of mediation:
   Both hypotheses are supported.

Moderation research designs:
  Hypothesis 3: Wrench in drive train will affect
          relationship between pedal and speed.
  Hypothesis 4: Disconnecting alternator from car will affect relationship.

Results of moderation tests:
   Hypothesis 3 was supported but Hypothesis 4 was not!

Conclusion: Drive train is a mediator but alternator is not.
                                                                              12
     Exogenous/                                 Endogenous/
                                Mediator
    Independent                                  Dependent




                               Moderator




Figure 5. Test of mediation using moderation.                 13
                             Issues
• Tests of mediation through manipulated moderation are
  not fool-proof.
   – Confounds in manipulation can lead to invalid interpretations.
   – No manipulation possible or possible manipulation weak.
• Tests of mediation through passive observation are
  likely to be . . .
   – heavily confounded or
   – analytically suspect (power, analysis issues).
• Nonetheless,
   – manipulation studies are generally more internally valid,
   – require clever designs (e.g., task load paradigm to moderate
     attentional resources available), and
   – represent many of the issues I/O Psychologists find important
      • E.g., why do context/job dimensions matter?
      • Or, when are they likely to matter given the underlying mechanisms?   14
                      Conclusions
• We are not suggesting that covariance techniques for
  assessing mediators should be abandoned. There will
  always be cases where it is the only viable choice or other
  considerations (e.g., external validity) suggest favoring
  the approach. Rather, our objectives are threefold:
   – First, we wanted to add some humility to the discussion
     sections of covariance mediator studies.
   – Second, we wanted to motivate researchers to develop
     manipulations or measures of moderators that might allow for
     cleaner inferences of mediation.
   – Finally, we wanted to increase the likelihood that researchers
     will consider the understandings their findings of moderation
     might provide regarding mediating mechanisms.
                                                                      15
                             References
Baron, R. M. & Kenny, D. A. (1986). The moderator-mediator variable distinction in
   social psychological research: Conceptual, strategic, and statistical considerations.
   Journal of Personality and Social Psychology, 51, 1173-1182.
Bing, M. N., Davison, H. K., LeBreton, D. L., & LeBreton, J. M. (2002, April). Issues
   and improvements in tests of mediation. Poster presented at the annual meeting of the
   Society for Industrial and Organizational Psychology, Toronto, Canada.
James, L. R. & Brett, J. M. (1984). Mediators, moderators and tests of mediation.
   Journal of Applied Psychology, 69, 307-321.
MacKinnon, D. P.., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002).
   A comparison of methods to test mediation and other intervening variable effects.
   Psychological Methods, 7, 83-104.
Runkel, P. J. & McGrath, J. E. (1972). Research on human behavior: A systematic guide
   to method. New York: Holt, Rinehart, and Winston.
Shadish, W, R, Cook, T. D. & Campbell, D. T. (2002). Experimental and Quasi-
   Experimentation Designs for generalized causal inference. Boston: Houghton
   Mifflin.

                                                                                           16