The Effects of Teacher Training on Teacher Value-Added

Document Sample
The Effects of Teacher Training on Teacher Value-Added Powered By Docstoc
					Research Using State
Longitudinal Data Systems:
Accomplishments and
Challenges – The Case of Florida

          Tim R. Sass
   The Status of Statewide Longitudinal data
   Projects Underway Using Statewide Data
    from Florida
   Challenges to Using Statewide Longitudinal
    Data for Research
   What Lies Ahead
The Status of State Longitudinal
Data Systems
   Data Quality Campaign’s “10 Essential Elements”
       A unique statewide student identifier that connects student data across
        key databases across years
       Student-level enrollment, demographic and program participation
       The ability to match individual students’ test records from year to year to
        measure academic growth
       Information on untested students and the reasons they were not tested
       A teacher identifier system with the ability to match teachers to students
       Student-level transcript information, including information on courses
        completed and grades earned
       Student-level college readiness test scores
       Student-level graduation and dropout data
       The ability to match student records between the P–12 and higher
        education systems
       A state data audit system assessing data quality, validity and reliability
The Status of State Longitudinal
Data Systems
   States That Can Track Student Gains and Match
    Teachers to Students
       Arkansas, Delaware, Florida, Georgia, Hawaii, Kentucky,
        Louisiana, Mississippi, New Mexico, Ohio, South Carolina,
        Tennessee, Utah, West Virginia, Wyoming
         North Carolina researchers do this indirectly through the
          identity of the teacher administering exams in elementary
         Some states have only recently acquired the ability to
          match students and teachers
             Kentucky (2006-07), Wyoming (2006-07)
The Status of State Longitudinal
Data Systems
Research Projects Using Statewide
Longitudinal Data From Florida
   School Choice
       “Charter Schools and Student Achievement in Florida” (Sass
           Mature charters on par with traditional public schools in math and
            yield higher reading scores than the average traditional public
           Charter school competition associated with small improvement in
            math achievement in traditional public schools
       “Charter Schools and Educational Attainment” (Booker and Sass)
           Will investigate the impact of charter high school attendance on
            high school graduation and college attendance in Florida and in
       “Vouchers and Special Education” (Sass)
           Will analyze the decision to accept a voucher and the effects of
            vouchers on the supply of private schools and the achievement of
            special education students who remain in public schools
Research Projects Using Statewide
Longitudinal Data From Florida
   Peer Effects
       “Classroom Peer Effects” (Burke and Sass)
           Preliminary evidence suggests that peer influences
            are significant and the effects are non-linear
               Low-achieving (lowest quintile) students worse or no better
                off in classes with more high-achieving (top quintile) peers
               Mid-level students generally worse off with more low-
                achieving peers and better off with high-achieving peers
               High-achieving students better off with more high-achieving
                peers and worse off with a greater fraction of low-achieving
Research Projects Using Statewide
Longitudinal Data From Florida
   Teacher Training
       “Teacher Training, Teacher Quality and Student
        Achievement” (Harris and Sass (2007))
         Experience matters, especially over the first five years
         Pedagogical content knowledge matters, both pre-service
          and in-service
         Other forms of preparation do not seem to impact teacher
         No significant correlation between a teacher’s college
          entrance exam scores and later effectiveness as a teacher
   Teacher Preparation Programs
       In conjunction with researchers from New York, Louisiana
        and Ohio, will be analyzing the relative effectiveness of
        teachers who obtain certification through various routes
Research Projects Using Statewide
Longitudinal Data From Florida
   National Board Certification
       “The Effects of NBPTS-Certified Teachers on Student
        Achievement” (Harris and Sass (2007))
         NBPTS certification provides a positive signal of teacher
          effectiveness in some cases, though the ability of NBPTS
          certification to identify high-quality teachers varies
          considerably across grades and subjects
         Results are sensitive to the exam used to measure student
         No evidence that the NBPTS process itself enhances
          teacher quality
         Little of no evidence of positive spillovers from NBPTS
          certified teachers to non-NBPTS-certified teachers
Research Projects Using Statewide
Longitudinal Data From Florida
   Teacher Labor Markets
       “A Longitudinal Analysis of Teacher Mobility: Intra-
        District Moves, Inter-District Moves and Departure
        Decisions” (Feng (2007))
           Exit decisions influenced by teacher salaries relative to
            other occupations and the proportion of minority students in
            their school
           Inter-district moves determined by relative salaries across
            districts and minority enrollment
           Intra-district movement toward schools with higher student
            achievement and fewer minority students
           Minority teachers’ job decisions less effected by minority
            student enrollment than those of their white colleagues
Research Projects Using Statewide
Longitudinal Data From Florida
   Accountability
       “School Accountability and Teacher Job Choice”
        (Feng, Figlio and Sass)
           Will investigate the impact of school accountability
            pressures on teachers decisions to move between schools
            or exit teaching entirely
           Will exploit the fact that Florida changed the mechanism for
            grading schools which lead some schools to face increased
            accountability pressure and lessened the pressure faced by
            other schools
Challenges to the Use of
Statewide Longitudinal Data
   Achievement Tests and the Measurement of
    Student Achievement Gains
       Relatively little known about the effects of alternative
        assessment measures on measured impacts
           Test score scaling
               Validity of using achievement tests to measure intertemporal gains
                in achievement
               Criterion-reference exams versus developmental scale scores
               Norming scores by grade and year
           Choice of test instrument
               Some evidence that results differ across tests (Harris and Sass
Challenges to the Use of
Statewide Longitudinal Data
   Methodology
       How well do “quasi-experimental” methods control for
        selection biases?
           Recent evidence on value-added models versus random
            assignment by Cantrell, Fullerton, Kane and Staiger (2007)
       Inconsistent Estimation Methods and Specification in
        “Value-Added” Achievement Models
           Student and teacher effects versus student and teacher
           Random versus fixed effects models
           Gain scores versus partial persistence models
           Inclusion/exclusion of Peer Effects
Challenges to the Use of
Statewide Longitudinal Data
   Use of Teacher Effects to Measure Teacher
       Not much known about inter-temporal consistency of
        individual teacher effects
           Koedel and Betts (2006)
       Little evidence on sensitivity of teacher effect
        estimates to changes in model specification
       Use in policy research versus use for accountability
        and merit pay
The Future of Research Using
Statewide Longitudinal Data
   Cross-State Analyses
       Due to lack of data, most statewide research so far has been
        limited to Florida, North Carolina and Texas
           Many studies using large urban school districts, such as New
            York, Chicago and Los Angeles
       Data availability in other states will help determine if previous
        findings are robust across geographic areas
           Variability in effectiveness of NBPTS-certified teacher in Florida
            and in North Carolina found by Ladd, Sass and Harris (2007)
       May be able to exploit cross-state variation in policy
           RAND charter school project
The Future of Research Using
Statewide Longitudinal Data
   Linking Value-Added Measures to Qualitative
       Comparison of Teacher Observations to Value-Added
        Measures of Teacher Quality
           Rockoff, et al.
       Comparison of Principal Evaluations of Teachers to
        Value-Added Measures
           Harris and Sass (2007)
   Matching Survey Evidence to Administrative