EDC MPS CCSSO presentation final by g9p551J

VIEWS: 0 PAGES: 29

									The Use of Student Assessments
in Teacher Evaluations Through
the Next Decade
National Conference on Student Assessment
June 27, 2012
Minneapolis
  Agenda

• Introduction:             Julie Kochanek
• Study profile:            Julie Riordan
    – Performance-based teacher evaluation systems in five states
• Study profile:            Karen Shakman
    – Changing culture and building capacity: An exploration of district
      strategies for implementation of teacher evaluation systems
• District profile:         David Heistad
    – The use of value-added in Minneapolis public schools
• Q & A discussion:         Julie Kochanek

  edc.org                                                9/12/2012         2
  An Examination of Performance-based
       Teacher Evaluation Systems

              Karen Shakman
                Julie Riordan
            Maria-Teresa Sanchez
             Kyle DeMeo Cook
             Richard Fournier
                Jessica Brett



edc.org                            9/12/2012   3
Rationale for Study

• Regional need
• Recent research
• Race to the Top
• State policy context




edc.org                  9/12/2012   4
Selection Criteria

•    Required for all practicing general educators
•    Operational statewide in 2010–11 school year
•    Multiple rating categories
•    Multiple measures of teacher effectiveness




edc.org                                  9/12/2012   5
Research Questions

• What are the key characteristics of state-level
  performance-based teacher evaluation systems in the
  study states?

• How do state teacher evaluation measures, the teaching
  standards the evaluations are designed to measure, and
  rating categories differ across states that have
  implemented statewide systems?


edc.org                                    9/12/2012       6
Approach

• Examined 50 SEA websites
• Conducted additional online search
• Reviewed evaluation guides/manuals, rubrics,
  training materials, regulations, program reports
• Contacted state directors




edc.org                                9/12/2012     7
States Included in Study

• Delaware (RTT Phase 1)
• Georgia (RTT Phase 2)
• North Carolina (RTT Phase 2)
• Tennessee (RTT Phase 1)
• Texas (did not apply)




edc.org                          9/12/2012   8
Evaluation Measures

•    Classroom observations
•    Principal evaluations
•    Analysis of classroom artifacts
•    Analysis of teaching portfolios
•    Teacher self-reports of practice
•    Student ratings of teacher performance
•    Value-added (student growth) strategies

                                 (Goe, Bell, and Little, 2008)

edc.org                                        9/12/2012     9
Findings: RQ1 – Key Characteristics




edc.org                       9/12/2012   10
Findings: RQ 2 – Teaching Standards




edc.org                      9/12/2012   11
Findings: RQ 2- Rating Categories




edc.org                       9/12/2012   12
Future Directions

• Further investigation about specific nature of the
  measures in place, including how student data are used in
  the overall assessment of teachers
• Identify the types of knowledge and skills that are
  commonly observed and evaluated in performance-based
  systems
• Investigate the fidelity of implementation of these
  systems


edc.org                                      9/12/2012    13
Changing Culture and Building Capacity:

   An Exploration of District Strategies for
Implementation of Teacher Evaluation Systems

               Karen Shakman
               Nicole Breslow
               Julie Kochanek
                Julie Riordan
                Tom Haeford

edc.org                           9/12/2012    14
Context

• National attention on teacher evaluation

• Focus on districts

• Systems change as framework



edc.org                             9/12/2012   15
Research Questions

1. What challenges are districts facing as they design and
   implement new teacher evaluation systems?

1. What strategies have districts developed to address
   these challenges?




edc.org                                      9/12/2012       16
Approach

• Conducted semi-structured interviews with 16
  district representatives

• Employed an emergent code strategy to transcripts
  and interview notes to identify key themes




edc.org                                 9/12/2012     17
Findings: 2 Major Challenges

1. Changing district and school level culture

1. Building district and school level capacity




edc.org                               9/12/2012   18
Findings: Strategies for Changing Culture


1. Focus on effective teaching
2. Engage stakeholders in the design process
3. Involve the teachers union
4. Align professional development
5. Establish differentiated and direct communication
          strategies

edc.org                                  9/12/2012     19
Findings: Strategies for Building Capacity

1. Invest in training evaluators and teachers

2. Establish district-level, cross-functional teams

3. Identify new roles and fill gaps in expertise

4. Sequence implementation strategies

edc.org                                  9/12/2012    20
Remaining Challenges

• Redefined role for principals


• Use of student achievement measures

• Lack of alignment between local and state level
  reform efforts



edc.org                                9/12/2012    21
Future Directions

• Research
      –Implementation studies
      –Measurement studies
      –Impact studies


•Building a community of practice



edc.org                             9/12/2012   22
edc.org   9/12/2012   23
History of Value-Added Use in Minneapolis
Public Schools

   • Evaluation of the Public School Academy (1992-94) Longitudinal
     Achievement Effects – Rob Meyer and U of W
   • Teachers who Beat the Odds in 2nd Grade Reading (1997-99)
   • Quality Performance Awards (1995–2005) Multiple Measures with
     School Awards
   • Teacher Advancement Program (TAP) School, Grade level and
     individual teacher bonuses (2006-2010)
   • Teachers who Beat the Odds in Kindergarten Literacy (2006-2010)
   • Grade level value-added to all K-8 Principals (2010-11) – Bush
     Foundation and VARC
   • Classroom level value-added to all K-8 Teachers (2012-13) – VARC
     and Dr. Chris Moore
edc.org
 Predictive Validity of the Minneapolis Public Schools
 Beginning of Kindergarten Assessment (BKA)
Measures (MPS Beginning of   9 months later   1.75 years      2.75 years     3.75 years
Kindergarten Assessment)     (EKA Total       later (Gr. 1    later (Gr. 2   later (Gr. 3
N= 1674                      Literacy)        Oral Reading)   MAP Reading)   MCA Reading)
Alphabetic Principle              .74              .71            .64            .58
 - letter names                   .74              .71            .64            .52
 - letter sounds                  .66              .60            .54            .50
Phonological Awareness            .70              .60            .63            .63
 - initial sounds                 .63              .58            .57            .57
 - rhyming                        .65              .53            .59            .58
Vocabulary
 - picture naming                 .61              .47            .54            .52
Listening Comprehension           .63              .52            .65            .67
Concepts of Print                 .55              .49            .55            .57
Total Literacy                    .81              .74            .70            .65

  edc.org                                                                              25
 Value-added Kindergarten Teacher Effects
         “beat the odds teachers”

      • Post-test reading score=
          – Pretest literacy score
          – + Free or reduced price lunch
          – + Racial/ethnic code (4 dummy codes)
          – + Gender
          – + English Language Learner status
          – + Special Education status
          – + Lives with single parent
          – + Teacher effects

edc.org
Follow-up with teachers who beat the odds

• Surveys, observations, video tape
• These teachers can be leads in staff development,
  mentors, etc.
• What about inviting these teachers to work in
  schools with the high concentrations of poverty and
  students of color?
• Videos at:
     http://rea.mpls.k12.mn.us/uploads/teachers_who_beat_the_odds.pdf



edc.org
Current Work with the Bush Foundation and
Value-added Research Center (VARC)

1) Teacher level value-added rolled out K-8 in
   2012-13 (Rubric Scores and Value-added)
2) Graduates from 14 institutions tracked into
   MPS system for feedback to first year
   teachers and the Institutions of Higher
   Education (IHEs)
3) Studies of the human capital teacher pipeline

edc.org      Presentation Title Appears Here   9/12/2012   28
For more information….

•    jkochanek@edc.org
•    jriordan@edc.org
•    kshakman@edc.org
•    dheistad@mpls.k12.mn.us




edc.org        Presentation Title Appears Here   9/12/2012   29

								
To top