Magnet Schools and Peers Effects on Student Achievement by zoz11082

VIEWS: 14 PAGES: 26

									Magnet Schools and Peers: Effects on
       Student Achievement

                Dale Ballou
             Vanderbilt University
              November, 2007

Thanks to Steve Rivkin, Julie Berry Cullen, Adam
Gamoran, Ellen Goldring and Keke Liu.
          Research Questions
• Has attending a magnet school caused an
  increase in mathematics achievement?
• How large is the influence of peers on
  mathematics achievement?
• How much of the magnet school effect
  remains after controlling for the influence
  of peers?
             Study Setting

• Middle Schools in a Large Southern
  District
     1 selective academic magnet
     4 non-selective magnets
     5 student cohorts
     6 years: 1998-99 through 2003-04
     Grades 5 & 6
         Admissions Lotteries
• Oversubscribed magnets conduct lotteries
• Students may enter multiple lotteries
• Students who are not outright winners are
  placed on wait lists
• Wait-listed students accepted until the first
  week of school
       Non-lottery Admissions
• Sibling preferences
• Promotion from a feeder school
• Geographic priority zone

 These students are not included in the
 study sample (though they do enter the
 calculation of peer characteristics).
          Research Design
• Lotteries assign students randomly to
  school type and to peers (magnet school
  peers vs. non-magnet peers).

• Randomized design circumvents biases
  arising from self-selection of schools and
  peers.
        Limitations of Design
• Results may not generalize beyond lottery
  participants.

• Effects are relative (magnet schools vs.
  mix of non-magnet schools attended by
  lottery losers).
    Lottery Participation
              Academic Composite
                       Non-Academic
Applicants     2315      2594

Outright        883      1450
Winners
Delayed         223       756
Winners
Losers,This    1209       388
Lottery
Losers, All     539       199
Lotteries
          Grade 5 Enrollments
                 Academic   Composite
                            Non-Academic
This Magnet        758        1061

Other Magnets      287         339

Non-Magnets        834         846

Left system or     436         346
Not Tested
           Potential Pitfalls
• Substantial non-compliance, especially
  among winners of non-academic lotteries,
  attenuates estimated treatment & peer
  effects based on comparison of winners
  and losers.

 Remedy: Use lottery outcomes as
 instruments to predict probability of
 attending magnet school, outcomes
 interacted with peer variables at magnet &
 zoned schools as instruments for peer
 characteristics.
• High rates of attrition from district can
  introduce systematic differences between
  treatment and control groups.

 Remedies:
 Control for student characteristics (race,
 income, ESL, special ed, gender, prior
 achievement).
 Analyze attrition patterns for evidence of
 differences between winners and losers.
• Participating in multiple lotteries increases
  chances of winning. “Multiple participants”
  may differ in ways related to achievement.

  Remedy: Control for the combination of
  lotteries each student entered. Winners
  are compared to losers who entered the
  same combination.
• Lotteries randomly assign students to
  magnet school peers or peers in their
  neighborhood (zoned) school, but lotteries
  do not determine the characteristics of the
  latter—residential decisions do.

 Remedy: Control for characteristics of the
 peers in the zoned school.
        Peer Characteristics
• Percentages black, low income (free &
  reduced-price lunch program), special ed,
  ESL, female
• Absenteeism rate
• Disciplinary incidents (rate per student)
• Intra-year mobility
• Prior achievement in math and reading
         Model (Summary)
• Two treatment variables (academic
  magnet, composite non-academic magnet)
• Variation in peers resulting from lottery
  outcomes
• Other controls (student characteristics,
  peers at the zoned school, lottery
  participation indicators, year by grade
  effects)
               Findings
• When model does not include peer
  characteristics
  - Academic magnet, + 18% in grade 5,
  drops to +10% in grade 6 (% of normal
  year growth)
  - Non-academic magnet, no grade 5
  effect, +54% in grade 6
• When models include peer characteristics
  - Reducing percent black from 75% to
  25% increases scores by 60% of normal
  year growth.
  - Effect of percent low income is about half
  that large.
  - Other peer characteristics have no
  statistically significant effect.
- Controlling for either percent black or
percent low income, the effect of the
academic magnet disappears.

- The large 6th grade effect in the non-
academic magnets remains substantially
undiminished.
         Checking Alternative
           Interpretations
• Are peers a proxy for heterogeneous
  response to treatment?

 Check: Interact magnet treatment
 indicators with all observed student
 characteristics.
 Finding: Peer effects are undiminished.
• Are peers a proxy for teacher quality?

 Check: Control for teacher quality by
 including teacher fixed effects.
 Finding: Peer effects are undiminished.
    Attrition, Academic Magnet
                        Lottery Participants
Left System After Grade: Winners       Losers
4                          13%            21%
5                           8%            14%
6                           9%            11%
7                           6%             9%
         Attrition, Composite
        Non-Academic Magnet
                        Lottery Participants
Left System After Grade: Winners       Losers
4                           8%            12%
5                          12%             9%
6                          10%             4%
7                          12%            16%
     Potential Attrition Biases
• Lottery losers are more likely to leave the
  system than winners.
• Losers are also more likely to leave when
  they can afford private schooling. These
  tend to be higher-achieving students.

 Result: Losers who remain in the system
 have lower achievement than winners who
 remain.
• Unfavorable peers at zoned school make
  losers more likely to leave system.
• Effect greatest among those who can
  afford private schooling.

 Result: Quality of peers positively
 correlated with losers’ achievement.
 Estimated peer effects appear too strong.
       Checking Attrition Bias
• Are rates of attrition correlated with
  variables that predict individual
  achievement (race, income, prior
  achievement)?

• Yes, but not differently for winners and
  losers.
              Conclusions
• For at least some students in some
  places, magnet schools have a positive
  effect on academic achievement.
• There are very strong peer effects on
  middle school achievement. Do not
  appear to operate through behaviors
  readily quantified with administrative data
  (attendance, disruptions, mobility).

								
To top