# Selecting How to Handle Selection

Document Sample

```					           Selecting How to Handle Selection Effects (or Emending Endogeneity)
Nigel Lo

A selection effect (or endogeneity) occurs when unobserved
factors or exogenous shocks are correlated with both the                       If selection is a problem, use a selection model.             The Data Generating Process
treatment and the outcome. Researchers have an increasing
variety of tools available to them and I use Monte Carlo
analysis to test three different methods — probit, bivariate
Do NOT match.                                The data generating process follows a latent variable approach with a
selection into treatment equation:
*
probit, and probit using preprocessed “matched” data. I find                                                                                          y    Selection    = " o + "1 x1 + " ER x ER + #1
that bivariate probit is the least biased as long as the exclusion
Normal Errors     Nonnormal Errors
with xER being an exclusion restriction, and an outcome equation:
restriction is strong. I also find that probit performs as well as
*
and often better than probit following matching. There appears                                                                                          y       Outcome   = \$ o + \$1 x1 + \$ T ySel + #2
to be no utility in using matching as a way of correcting for
with ySel being the treatment (and the selection outcome) where
selection effects.
&1 if y * > 0
Strong, Exogenous                                                    ySelection,Outcome = '
A Debate in the Literature                                                                                                                                                           *
When unobserved factors that are correlated with the treatment and
Restriction                                                                           (0 if y % 0
the outcome are present, what happens? These unobservables are
and
often referred to as selection effects or endogeneity and are distinct from
sample selection bias. With selection effects, we fully observe the
dependent variable for those who receive the treatment and those who do
#1,2 ~ Normal2 (µ,)) where µ = (0,0),
not. We should be concerned about selection effects because nonrandom
assignment of the treatment threatens to bias estimates and lead to                                                                                                 +1 *.
incorrect inferences. Selection effects are potentially a problem for a                                                                                           )=-   0             and   * 1 {-.8,-.5,0,.5,.8}
variety of research questions including the effects of treaty or institutional                                                                                      ,* 1/
membership, the effectiveness of third-party conflict management, and
the escalation of conflict.                                                                                                                     in the bivariate normal simulations and

What to do: Probit, Bivariate Probit, or                                                                                                             #1,2 ~ Skew Normal2 (2 ,3," ) where 2 = (0,0),
Matching?                                                                           Weak, Exogenous                                                                 3=)        and   " = (5,5)
•Include control variables that are correlated with the treatment                   Restriction                                                 in the nonnormal simulations in order weaken the assumption of normal
variable and the outcome and estimate the usual probit model. This                                                                              errors. The correlation between the errors can be thought of as the
approach essentially ignores the problem.                                                                                                       unobservable factors that influence both the selection of the treatment
and the outcome.
•Model the selection into treatment and the outcome simultaneously                                                                         !
using a bivariate probit model. This approach assumes that
unobserved factors manifest themselves in the correlation of the errors of
Results
the two equations and the errors are distributed bivariate normal.                                                                              The estimates using a matching method are biased – often more so
Identification requires a good exclusion restriction.                                                                                           than bivariate probit and probit. As expected, bivariate probit performs
•Use a nonparametric matching method to preprocess the data and                                                                                 the best when the data generating process follows a bivariate probit
proceed with the usual estimation method (with control variables).                                                                              process. When the errors are distributed nonnormally, the performance
Through achieving covariate balance, the influence of any confounding                                                                           of bivariate probit relative to the other estimators decreases. However,
variables should be nil. There are a number of different ways to match but                                                                      when the restriction is strong, bivariate probit remains the least biased
one can only match on observable variables and matching assumes                                                                                 except for when there is no correlation in the errors. As the strength of
belonging in the control group is independent of treatment, conditional on                                                                      the exclusion restriction decreases (regardless of its exogeneity),
observed variables without making any distributional assumptions.                Strong, Correlated                                             bivariate probit becomes more biased than either matching or probit at
Restriction                                                small positive values of ρ. At negative values of ρ, bivariate probit
A Monte Carlo Experiment                                                                                                                          continues to be the least biased estimator. Ordinary probit outperforms
the matching methods despite not having preprocessed the data but only
Which method should we use and under what conditions? If there are                                                                              when the exclusion restriction is strong, regardless of the error
unobserved factors present, then matching may not be the panacea                                                                                distribution. Otherwise, probit does just as well as matching.
researchers are looking for. However, the assumptions of bivariate probit
models may compromise inferences. Comparing estimators using                                                                                    The 95% confidence interval often shows overlap with the other
existing observational data is problematic because there may be other                                                                           estimators. While bivariate probit may have mean estimates close to the
problems, such as dependencies (model, spatial, or temporal) that must                                                                          true value, the estimates sometimes have a large variance.
be dealt with first. Monte Carlo simulation is especially useful for
comparing competing methods of estimation because we can control
various aspects of the data in simulating the data.
Conclusions
Since matching cannot match on unobservable factors, we should be
There are three key factors that must be managed:                                                                                               cautious in applying it to studies that involve selection effects.
•the exogeneity of the exclusion restriction – the correlation between            Weak, Correlated                                              Researchers should continue to apply bivariate probit estimators to
the exclusion restriction and other variables in the model (0 and .8).                                                                          these problems even though they make a distributional assumption
•the strength of the exclusion restriction – the size of the coefficient            Restriction                                                 regarding the errors. In fact, the results here suggest that even a regular
of the exclusion restriction (.1 and 1.5).                                                                                                      probit approach is preferred over matching. Although Simmons and
•the distribution of errors – bivariate probit assumes a bivariate normal                                                                       Hopkins stress that “every effort should be made to theorize and
distribution of errors, thus it is necessary to weaken this assumption.                                                                         measure purported unobservables,” the fact remains that is unlikely that
we will be able to “catch ‘em all.” Therefore, it is necessary to control for
Varying these factors ensures that no one method is favored over the                                                                            these statistically and develop strong theoretically compelling and novel
others by design.                                                                                                                               exclusion restrictions.

```
DOCUMENT INFO
Shared By:
Categories:
Stats:
 views: 14 posted: 3/18/2010 language: English pages: 1