Teacher Quality and the Context of Teaching Adam Gamoran University of Wisconsin-Madison Teacher quality as teacher effects Teacher effects are large In the sense that which teacher a student has makes a difference for achievement How large? Sanders & Rivers: 3 years with a highly effective teacher can boost achievement as much as 50 percentile points Rivkin, Hanushek, & Kain: a 1 s.d. increase in teacher quality is worth as much as a 10-student decrease in class size Nye, Konstantopoulos, & Hedges: a 1 s.d. increase in teacher quality is worth a 1/3 s.d. increase in math achievement, less in reading Teacher quality as teacher effects Qualifications What are called “teacher effects” are usually a mix of teacher, peer, and other classroom effects Teachers and classrooms not distinguished The mechanisms through which teacher effects occur have not been well identified The role of the context in which teachers work has not been investigated Teacher effects v. teacher quality A puzzle: if teacher effects are so large, why have the mechanisms not been identified? Inadequate theory Inadequate data Inadequate measurement tools How can we solve this puzzle? Sources of teacher/classroom effects Teacher characteristics Subject matter knowledge Experience Certification status Classroom characteristics Track/ability group Class size Class composition Sources of teacher/classroom effects Classroom activities Not what teachers do, but what teachers and students do together So teaching may be endogenous to students Measurement Relatively simple in early elementary grades More complex as students move up grade levels Sources of teacher/classroom effects Measures of class activities that predict achievement Coverage of content Teacher logs Survey of enacted curriculum Effects of content coverage Figure 2. Estimated Learning Gains Le arning G a ins b y Co urs e Ty pe 2 .5 2 R e g e n ts 1 .5 A l g e b ra S tr e tch Re g e n ts 1 M a th A /B /UC S MP G e n M t h /P re A lg 0 .5 0 b a Achievement Growt h, c ont rolling f or content Achievement Growt h a N otes: A djuste d for stude nt initial sc ore, sex, rac e/ethnicity, previous grade , cla ss socio-econom ic status. b Simulated to ha ve con tent covera ge like that of Re gents classe s. Source: White e t al. (1997). Sources of teacher/classroom effects Measures of class activities that predict achievement Instructional strategies Reading wars: phonics vs. “whole language” Math wars: drill vs. “teaching for understanding” In reading, both have proven important The same will likely emerge in math Effects of kindergarten reading instruction (teacher reported) Class-Level Equation: Effects on Adjusted Mean Achievement Between Classes Model 1 Model 2 Coefficient St. Coefficient St. Error Error Class percent black -0.02 *** 0.003 -0.02 *** 0.003 Class percent Hispanic -0.01 0.004 -0.01 0.004 “Whole Language” 0.08 *** 0.02 Instruction “Phonics” Instruction 0.20 *** 0.03 Whole Class Instruction 0.28 ** 0.10 Small Group Instruction 0.32 ** 0.10 Individual Instruction -0.02 0.13 Class Behavior 0.16 0.10 * P>|t| .05 ** P>|t| .01 *** P>|t| .001 Source: Milesi & Gamoran, 2006. Effects of kindergarten reading instruction (teacher reported) Class-Level Equation: Effects on Adjusted Mean Achievement Between Classes Model 1 Model 2 Coefficient St. Coefficient St. Error Error Class percent black -0.02 *** 0.003 -0.02 *** 0.003 Class percent Hispanic -0.01 0.004 -0.01 0.004 “Whole Language” 0.08 *** 0.02 Instruction “Phonics” Instruction 0.20 *** 0.03 Whole Class Instruction 0.28 ** 0.10 Small Group Instruction 0.32 ** 0.10 Individual Instruction -0.02 0.13 Class Behavior 0.16 0.10 * P>|t| .05 ** P>|t| .01 *** P>|t| .001 Source: Milesi & Gamoran, 2006. Effects of kindergarten math instruction (teacher reported) Class-Level Equation: Effects on Adjusted Mean Achievement Between Classes Model 1 Model 2 Coefficient St. Coefficient St. Error Error Class percent black -0.02 *** 0.003 -0.02 *** 0.003 Class percent Hispanic -0.01 0.003 0.01 * 0.003 “Teaching for 0.07 *** 0.02 understanding” Instruction “Drill” Instruction 0.09 *** 0.02 Whole Class Instruction 0.07 0.08 Small Group Instruction 0.15 0.08 Individual Instruction 0.06 0.09 Class Behavior 0.11 0.08 * P>|t| .05 ** P>|t| .01 *** P>|t| .001 Source: Milesi & Gamoran, 2006. The context of teacher effects Earlier view: school characteristics could leverage classroom events “Restructuring” studies showed this is not correct Changing practice reflects teacher learning, not school structure However, school characteristics may enhance or inhibit the relation between teacher learning and teaching practice The context of teacher effects Schools with greater capacity for change are better able to support improved practice By capacity, we mean: Material resources such as time, money Human resources: teacher knowledge, skills, and dispositions Social resources: relation of trust and shared values that allow teachers to discuss new ideas and provide a safe environment for change A study of teacher effects in context The “new education sciences” calls for rigorous designs that permit causal inference Can we combine this with: Conceptions that draw on disciplinary knowledge? Opening the “black box” to examine events inside classrooms? These qualities are needed to identify mechanisms of teacher effects A study of teacher effects in context The case of elementary science in Los Angeles, CA Los Angeles: lowest average score in CA on 4th grade NAEP science CA: lowest average scores in the U.S. Improvement strategy: District centralized curriculum A study of teacher effects in context Immersion teaching in K-8 science Extended, inquiry-based curricular units Rigorous content based on California state science standards Grade 4: Cycling of matter and transfer of energy Grade 5: Weather forces and prediction A study of teacher effects in context Efficacy trials of immersion teaching Positive effects of immersion teaching in a quasi- experiment, particularly for African American students Positive effects of professional development for immersion teaching on teacher knowledge in pre- post studies Implementation challenges Simply distributing the curriculum did not change practice Response: Intensive professional development A study of teacher effects in context Scaling up immersion teaching 40 treatment schools, 40 control schools Power > .80 with n=100 students per school, d = .20, covariate correlation = .80 Treatment = intensive professional development for immersion teaching All teachers can get the curriculum, but only those who get the pd are likely to use it A study of teacher effects in context Allowing for context Teacher survey indicators of material, human, social resources Models will test school-level interaction of treatment x capacity Expectation is that treatment effects will be larger in schools with greater capacity A study of teacher effects in context Strengths of this approach Randomized treatment allows for stronger causal inference Evidence on mechanisms (observations of instruction) Evidence on context (school capacity) Limitation: generalizability Evidence on context helps, but does not fully resolve this limitation Conclusions Teacher/classroom effects are large, but too little is known about how these effects occur We need to get inside classrooms to find out Focus on implementation and context will be essential to understanding teacher/classroom effects References Gamoran, A., and S. Kelly. 2003. Tracking, instruction, and unequal literacy in secondary school English. Pp. 109-126 in M. T. Hallinan, A. Gamoran, W. Kubitschek, and T. Loveless (Eds.), Stability and change in American education: Structure, process, and outcomes. Clinton Corners, NY: Eliot Werner Publications. Milesi, C., and A. Gamoran. 2006. Effects of class size and instruction on kindergarten achievement. Educational Evaluation and Policy Analysis, 28, 287- 313. Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26, 237-257. Rivkin, S.G., Hanushek, E.A. & Kain, J.F. (2005). Teachers, schools and academic achievement. Econometrica, 73, 417-458. Rowan, B., E. Camburn, and R. Correnti. 2004. Using teacher logs to measure the enacted curriculum in large-scale surveys: Insights from the Study of Instructional Improvement. Elementary School Journal, 105, 75-102. Sanders, W. & Rivers, J. (1996). Cumulative and residual effects of teachers on future student academic achievement. Knoxville, TN: University of Tennessee. White, P. A., A. C. Porter, A. Gamoran, and J. Smithson. 1996. Upgrading high school math: A look at three transition courses. NASSP Bulletin, 81 (Fall 1997), 72-83.