# Regression Discontinuity

Document Sample

```					Regression Discontinuity

10/13/08
What is R.D.?
• Regression--the econometric/statistical
tool social scientists use to analyze
multivariate correlations

Yi    X1i1  X2i2  ei
Where Y is some sort of dependent variable,
alpha’s a constant, the X’s are a bunch of
independent variables, the beta’s are coefficients,
and the e is the error term.
Discontinuity
Some sort of arbitrary jump/change
thanks to a quirk in law or nature.

We’re interested in the ones that
make very similar people get very
dissimilar results.
Discontinuity Examples
• PSAT/NMSQT
– Basically the top 16,000 test-takers
get a scholarship.
– A small difference in test score can
means a discontinuous jump in
scholarship amount.
Discontinuity Examples
• School Class Size
– Maimonides’ Rule--No more than 40
kids in a class in Israel.
– 40 kids in school means 40 kids per
class. 41 kids means two classes
with 20 and 21.
(Angrist & Lavy, QJE 1999)
Discontinuity Examples
• Union Elections
– If employers want to unionize, NLRB
holds election. 50% means the employer
doesn’t have to recognize the union, and
50% + 1 means the employer is required
to “bargain in good faith” with the union.

(DiNardo & Lee, QJE 2004)
Discontinuity Examples
• U.S. House Elections
– Incumbency advantage. If you’re first
past the pole in the previous election,
even by just one vote, you get a huge

(David Lee, Journal of Econometrics 2007)
Discontinuity Examples
• Air Pollution and Home Values
– The Clean Air Act’s National Ambient Air
Quality Standards say if the geometric mean
concentration of 5 pollutant particulates is
75 micrograms per cubic meter or greater,
county is classified as “non-attainment” and
are subject to much more stringent
regulation.
(Ken Chay, Michael Greenstone, JPE 2005)
Combine the “R” and the “D”
Run a regression based on a situation
where you’ve got a discontinuity.

Treat above-the-cutoff and below-the-
cutoff like the treatment and control
groups from a randomization.
Why are we doing this?
Why do we have to look for quirks like this?
Can’t we just control for whatever we want
using OLS or some other line-fitting tool?

Yi    X1i1  X2i2  ei
Just get a bunch of people’s salaries and PSAT
scores. PSAT’s are X, income is Y, run a
regression in SPSS/Stata, or heck, even Excel, and
we have causal inference, right? Higher test
scores cause people to earn more later in life.
No.
The statistical methods we use are based on lot of
assumptions. Importantly, the error terms (which is really
full of things we can’t measure, the unobservables) are
supposed to be uncorrelated with the X’s and normally
distributed.
In reality, those conditions probably hasn’t been met in any of
the previous situations.

For example, class size is probably correlated with some type
of neighborhood quality.

Please turn to your neighbor and discuss what is probably wrong with
each of the previous 5 examples (PSAT, class size, union elections,
house elections, air pollution)
No.
The statistical methods we use are based on lot of
assumptions. Importantly, the error terms (which is really
full of things we can’t measure, the unobservables) are
supposed to be uncorrelated with the X’s and normally
distributed.
In reality, those conditions probably hasn’t been met in any of
the previous situations.

•   Higher PSAT kids might have higher ability.
•   Crowded classrooms might be in poorer schools.
•   Unionized workers might work for certain types of firms.
•   Incumbent politicians might be better. They won before, didn’t they?
•   Pollution might be correlated to economic growth, which could
increase home values.
Controlling for everything?
Focus on the Israeli schools for a second.
We can try and control for neighborhood poverty level.
Does that solve the problem?
No.
If neighborhood poverty level is correlated with the X of
interest (class size) why would you think it’s safe to assume
that the unobservables aren’t correlated? Have you really
magically controlled for every single thing that’s correlated
with the X of interest? Probably not.

So let’s find a bandwidth in which these things are
uncorrelated.
A Bandwidth of Randomness
Test scores aren’t random, and neither is class size,
nor air pollution.

But is a kid in the 94.9th percentile really that different
from the 95th percentile kid?
Is a school with 40 kids that different from a school
with 41?

Right around the cutoff, there’s a good chance things
are random.
No Sorting - Observables
But don’t take my word for it. Look at the averages of the
observables in your below cutoff group, and the averages of
the observables in the above cutoff group. Are they the
same? Hopefully, but maybe not.

endogenous sorting? When deciding where to live, did
good moms look for schools where their kids would be the
41st kid? Did certain types of polluters look for counties
where they’d be below the cutoff?

These things can be checked to some degree--look at the
average observables above and below the cutoff.
No Sorting - Clumping
In addition to checking the observables on either side
of the cutoff, we should check the density of the
distribution. Is it unusually low/high right around
the cutoff?

If there’s some abnormally large portion of people
right around the cutoff, it’s quite possible that you
don’t have random assignment.
No Sorting - Clumping

QuickTime™ an d a
decompressor
are need ed to see this picture.

Dude, you’re totally cheating. Please stop.

Emily Conover & Adriana Camacho “Manipulation of Social Program Eligibility”
GSP--Multiple Analyses
“Incentives to Learn,” Ted Miguel, Michael Kremer, Rebecca Thornton

Girls Scholarship Program, Busia Kenya.

Randomize holding a scholarship competition across schools in Busia
and Teso districts.

Treatment: If a girl finishes in the top 15% in her district on the end-of-
year exam, she wins a two-year scholarship.

Randomization Analysis: Does attending a school with the competition
make you work harder/improve schooling outcomes?

RD Analysis: Does winning the award improve schooling outcomes?
P-900 in Chile
“The Central Role of Noise in Evaluating Interventions That Use Test Scores to Rank Schools” Kenneth
Y. Chay, Patrick J. Mcewan, Miguel Urquiola, AER 2005

Mean Reversion: Sophomore Slump, SI Cover Curse, Heisman Trophy Curse, Madden curse, and in the
opposite direction.

QuickTime™ a nd a
decompressor
are need ed to see this picture.
THIS IS THE MOST AMAZING
THING EVER!
HOLY CRAP! Look at the educational outcomes of
treatment schools in 1990, compared to those
same schools in 1988, before the program.
AMAZING! FANTABULOUS!

QuickTime™ and a
decompressor
are needed to see this picture.
Oh, wait.
Hmm. That’s kind of disappointing.

QuickTime™ an d a
decompressor
are need ed to see this p icture .
So how do we actually do this?
1.        Draw two pretty pictures
1. Eligibility criterion (test score, income, or whatever)
vs. Program Enrollment
2. Eligibility criterion vs. Outcome
Figure 1: Participation in PANES and eligibility
Figure 2: Political support for the government and program eligibility
1

.9
.8

.8
.6

.7
.4

.6
.2

.5
0

-.02           -.01               0           .01             .02
-.02            -.01              0            .01        .02                                 standardized SES
standardized SES
So how do we actually do this?
2. Run a simple regression.
(Yes, this is basically all we ever do, and the stats
programs we use can run the calculation in almost any
situation, but before we do it, it’s necessary to make sure
the situation is appropriate and draw the graphs so that we
can have confidence that our estimates are actually
causal.)

Outcome as a function of test score (or whatever), with a
binary (1 if yes, 0 if no) variable for program enrollment.
QuickTime™ and a
decompressor
are needed to see this picture.
Is it really that simple?
Don’t be silly.

You could totally have a situation where the outcome is some
sort of quadratic or cubic or nth polynomial function of
the test score. Try controlling for that. This is going to
depend on the situation and is somewhat arbitrary.

QuickTime™ and a
decompressor
are neede d to see this picture.
Wait, “somewhat arbitrary?”
Yeh, lame, I know. Arbitrary’s what we’re trying to avoid.
But two things aren’t univerally clear:

1. How wide a bandwidth around the cutoff are we looking at?

We’re really only confident in our estimate for people that are
close to the cutoff. This is a LOCAL AVERAGE
TREATMENT EFFECT. We can confidently say that a
school right around the cutoff would improve average test
scores by X if they received the treatment, but we’re not so
confident that already awesome schools would get the same
benefit.
Wait, “somewhat arbitrary?”
2. Without the program, what shaped function
would there be naturally?

What sort of function do we throw in to control
for the fact that even if there was no National
Merit Semifinalist scholarship, smarter kids are
likely to earn more later in life?

You’re Such a Phony.
check is to test for the effects of non-existent programs.

QuickTime™ and a
decompressor
are need ed to see this picture.
You’re Such a Phony.

QuickTime™ and a
decompressor
are need ed to see this picture.
Conclusion
•   Find a threshold
•   Look at people just above and just below
•   Make sure there’s no sorting
•   It’s only a local effect