# RN Diff

Document Sample

```					   Impact Evaluation Methods
Regression Discontinuity Design and
Difference in Differences

Slides by Paul J. Gertler & Sebastian Martinez
Measuring Impact
• Experimental design/randomization
• Quasi-experiments
– Regression Discontinuity
– Double differences (diff in diff)
– Other options

2
Case 4: Regression Discontinuity
• Assignment to treatment is based on a clearly
defined index or parameter with a known
cutoff for eligibility
• RD is possible when units can be ordered
along a quantifiable dimension which is
systematically related to the assignment of
treatment
• The effect is measured at the discontinuity –
estimated impact around the cutoff may not
generalize to entire population
3
Indexes are common in targeting of
social programs
• Anti-poverty programs  targeted to
households below a given poverty index
• Pension programs  targeted to population
above a certain age
• Scholarships  targeted to students with
high scores on standardized test
• CDD Programs  awarded to NGOs that
achieve highest scores

4
Example: Effect of Cash Transfer
on Consumption
• Target transfer to poorest households
• Construct poverty index from 1 to 100 with
pre-intervention characteristics
• Households with a score <=50 are poor
• Households with a score >50 are non-poor
• Cash transfer to poor households
• Measure outcomes (i.e. consumption) before
and after transfer
5
80
75
70
65
60        Regression Discontinuity Design - Baseline

20    30      40       50       60       70       80
Score
6
80
75
70
65        Regression Discontinuity Design - Baseline

Non-Poor

Poor
60

20    30      40          50          60   70     80
Score
7
80
75
70
65        Regression Discontinuity Design - Post Intervention

20        30       40       50       60       70       80
Score
8
80
75        Regression Discontinuity Design - Post Intervention

Treatment Effect
70
65

20        30       40       50       60       70       80
Score
9
Case 4: Regression Discontinuity
on a poverty index

• Where
• Treatment = 1 if score <=750
• Treatment = 0 if score >750

10
Case 4: Regression Discontinuity
Baseline – No
treatment
379.224
Fitted values

153.578
276                                      1294

yi  0  1Treatmenti   ( score)   i

2
11
Case 4: Regression Discontinuity
399.51         Treatment Period

Fitted values

183.647
276                                         1294

Case 4 - Regression Discontinuity
Multivariate Linear Regression
Estimated Impact on CPC                                                         30.58**
(5.93)
** Significant at 1% level

12
• Local average treatment effects – not always
generalizable
• Power: effect is estimated at the discontinuity, so we
generally have fewer observations than in a
randomized experiment with the same sample size
• Specification can be sensitive to functional form:
make sure the relationship between the assignment
variable and the outcome variable is correctly
modeled, including:
– Nonlinear Relationships
– Interactions

13
• RD yields an unbiased estimate of treatment effect
at the discontinuity
• Can many times take advantage of a known rule for
assigning the benefit that are common in the
designs of social policy
– No need to “exclude” a group of eligible
households/individuals from treatment

14
Measuring Impact
• Experimental design/randomization
• Quasi-experiments
– Regression Discontinuity
– Double differences (Diff in diff)
– Other options

15
Case 5: Diff in diff
• Compare change in outcomes between
treatments and non-treatment
– Impact is the difference in the change in
outcomes
• Impact = (Yt1-Yt0) - (Yc1-Yc0)

16
Outcome

Average
Treatment Group                         Treatment
Effect

Control Group

Time
Treatment
17
Outcome
Average
Treatment
Effect                         Estimated
Average
Treatment
Effect
Treatment Group

Control Group

Time
Treatment

18
Diff in Diff
• Fundamental assumption that trends (slopes)
are the same in treatments and controls
• Need a minimum of three points in time to
verify this and estimate treatment (two pre-
intervention)

19
Case 5: Diff in Diff
Case 5 - Diff in Diff
Not Enrolled   Enrolled                                         t-stat
Mean ΔCPC     8.26         35.92                                          10.31

Case 5 - Diff in Diff
Linear Regression        Multivariate Linear Regression
Estimated Impact on CPC        27.66**                         25.53**
(2.68)                          (2.77)
** Significant at 1% level

20
Impact Evaluation Example –
Summary of Results
Case 2 -                            Case 4 -
Case 1 - Before                           Case 3 -                      Case 5 - Diff in
Enrolled/Not                       Regression
and After                             Randomization                        Diff
Enrolled                          Discontinuity
Multivariate                           Multivariate   Multivariate     Multivariate
Linear        Multivariate Linear      Linear         Linear           Linear
Regression         Regression          Regression     Regression       Regression
Estimated Impact
on CPC                        34.28**               -4.15             29.79**        30.58**          25.53**
(2.11)              (4.05)             (3.00)          (5.93)           (2.77)
** Significant at 1% level

21

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 3 posted: 9/17/2012 language: Unknown pages: 21