Magnet Schools and Peers Effects on Student Achievement by zoz11082


									Magnet Schools and Peers: Effects on
       Student Achievement

                Dale Ballou
             Vanderbilt University
              November, 2007

Thanks to Steve Rivkin, Julie Berry Cullen, Adam
Gamoran, Ellen Goldring and Keke Liu.
          Research Questions
• Has attending a magnet school caused an
  increase in mathematics achievement?
• How large is the influence of peers on
  mathematics achievement?
• How much of the magnet school effect
  remains after controlling for the influence
  of peers?
             Study Setting

• Middle Schools in a Large Southern
     1 selective academic magnet
     4 non-selective magnets
     5 student cohorts
     6 years: 1998-99 through 2003-04
     Grades 5 & 6
         Admissions Lotteries
• Oversubscribed magnets conduct lotteries
• Students may enter multiple lotteries
• Students who are not outright winners are
  placed on wait lists
• Wait-listed students accepted until the first
  week of school
       Non-lottery Admissions
• Sibling preferences
• Promotion from a feeder school
• Geographic priority zone

 These students are not included in the
 study sample (though they do enter the
 calculation of peer characteristics).
          Research Design
• Lotteries assign students randomly to
  school type and to peers (magnet school
  peers vs. non-magnet peers).

• Randomized design circumvents biases
  arising from self-selection of schools and
        Limitations of Design
• Results may not generalize beyond lottery

• Effects are relative (magnet schools vs.
  mix of non-magnet schools attended by
  lottery losers).
    Lottery Participation
              Academic Composite
Applicants     2315      2594

Outright        883      1450
Delayed         223       756
Losers,This    1209       388
Losers, All     539       199
          Grade 5 Enrollments
                 Academic   Composite
This Magnet        758        1061

Other Magnets      287         339

Non-Magnets        834         846

Left system or     436         346
Not Tested
           Potential Pitfalls
• Substantial non-compliance, especially
  among winners of non-academic lotteries,
  attenuates estimated treatment & peer
  effects based on comparison of winners
  and losers.

 Remedy: Use lottery outcomes as
 instruments to predict probability of
 attending magnet school, outcomes
 interacted with peer variables at magnet &
 zoned schools as instruments for peer
• High rates of attrition from district can
  introduce systematic differences between
  treatment and control groups.

 Control for student characteristics (race,
 income, ESL, special ed, gender, prior
 Analyze attrition patterns for evidence of
 differences between winners and losers.
• Participating in multiple lotteries increases
  chances of winning. “Multiple participants”
  may differ in ways related to achievement.

  Remedy: Control for the combination of
  lotteries each student entered. Winners
  are compared to losers who entered the
  same combination.
• Lotteries randomly assign students to
  magnet school peers or peers in their
  neighborhood (zoned) school, but lotteries
  do not determine the characteristics of the
  latter—residential decisions do.

 Remedy: Control for characteristics of the
 peers in the zoned school.
        Peer Characteristics
• Percentages black, low income (free &
  reduced-price lunch program), special ed,
  ESL, female
• Absenteeism rate
• Disciplinary incidents (rate per student)
• Intra-year mobility
• Prior achievement in math and reading
         Model (Summary)
• Two treatment variables (academic
  magnet, composite non-academic magnet)
• Variation in peers resulting from lottery
• Other controls (student characteristics,
  peers at the zoned school, lottery
  participation indicators, year by grade
• When model does not include peer
  - Academic magnet, + 18% in grade 5,
  drops to +10% in grade 6 (% of normal
  year growth)
  - Non-academic magnet, no grade 5
  effect, +54% in grade 6
• When models include peer characteristics
  - Reducing percent black from 75% to
  25% increases scores by 60% of normal
  year growth.
  - Effect of percent low income is about half
  that large.
  - Other peer characteristics have no
  statistically significant effect.
- Controlling for either percent black or
percent low income, the effect of the
academic magnet disappears.

- The large 6th grade effect in the non-
academic magnets remains substantially
         Checking Alternative
• Are peers a proxy for heterogeneous
  response to treatment?

 Check: Interact magnet treatment
 indicators with all observed student
 Finding: Peer effects are undiminished.
• Are peers a proxy for teacher quality?

 Check: Control for teacher quality by
 including teacher fixed effects.
 Finding: Peer effects are undiminished.
    Attrition, Academic Magnet
                        Lottery Participants
Left System After Grade: Winners       Losers
4                          13%            21%
5                           8%            14%
6                           9%            11%
7                           6%             9%
         Attrition, Composite
        Non-Academic Magnet
                        Lottery Participants
Left System After Grade: Winners       Losers
4                           8%            12%
5                          12%             9%
6                          10%             4%
7                          12%            16%
     Potential Attrition Biases
• Lottery losers are more likely to leave the
  system than winners.
• Losers are also more likely to leave when
  they can afford private schooling. These
  tend to be higher-achieving students.

 Result: Losers who remain in the system
 have lower achievement than winners who
• Unfavorable peers at zoned school make
  losers more likely to leave system.
• Effect greatest among those who can
  afford private schooling.

 Result: Quality of peers positively
 correlated with losers’ achievement.
 Estimated peer effects appear too strong.
       Checking Attrition Bias
• Are rates of attrition correlated with
  variables that predict individual
  achievement (race, income, prior

• Yes, but not differently for winners and
• For at least some students in some
  places, magnet schools have a positive
  effect on academic achievement.
• There are very strong peer effects on
  middle school achievement. Do not
  appear to operate through behaviors
  readily quantified with administrative data
  (attendance, disruptions, mobility).

To top