Recent Research On Alternative Teacher Certification

							Research On Alternative Teacher Certification




1/14/04

Michael Podgursky
Department of Economics
University of Missouri - Columbia

Prepared for “Alternative Routes into Teaching: Defining Challenges, Devising
Strategies,” First Annual Conference of the National Center for Alternative Certification,
San Antonio, Texas Feb. 1-3, 2004.
Introduction: What is “Alternative Certification?”

         Alternative certification (AC) typically refers to programs that permit career
changers or non-traditional teaching candidates to earn teaching licenses. The traditional
route to teaching, which still accounts for the majority of new teachers, is for students to
earn undergraduate degrees in education from a state-approved teacher training program.
However, in response to concern about teacher quality, difficulties in recruiting qualified
minority teachers, and shortages of teachers in academic fields such as science or math,
many states have created alternative programs for entering the teaching profession.
These programs are designed for individuals who already have baccalaureate degrees in
areas other than education, and who often have experience in careers other than teaching.
Such individuals are generally unwilling to make major investments of time and
resources in returning to a school of education for a year or two of coursework required
for a traditional license before they earn a paycheck as a teacher. Thus, common
features of these programs are:

      Minimum pre-service training
      On-going professional development coursework and mentoring while on the job,
       leading to
      Regular teaching license after a probationary period of 2-3 years

Although 43 states have some sort of AC program in place, AC route teachers account for
a substantial share of teaching recruits in only a few states (e.g., New Jersey, Texas, and
California).

         A standard reference on state programs is the annual volume by Feistritzer and
Chesser (2002). A recent survey of the literature by Mathematica Policy Research
(MPR) provides a useful taxonomy and overview of state programs (Mayer, et.al. , 2003).
The MPR survey also makes the point that there is considerable variation even within
states in the character of AC programs, which mean that evaluations for any particular
program may not generalize across a state, much less between states. For this reason, an
MPR evaluation currently under way is focusing on particular features of AC programs
that more readily generalize.

Alternative Certification and Student Achievement

      How does the performance of alternative versus traditionally certified teachers
compare? Do AC teachers produce larger or smaller student achievement gains as
compared to traditionally-trained teachers?

       Unfortunately we do not at present have research that can answer that question.
While there have been many articles published about AC, few meet the standards of




                                                                                           1
scientific rigor that would permit us to draw conclusions about the effect of AC on
student achievement.

         Scientific evaluation of the effect of educational policies, including teacher
preparation, on student achievement requires either: a) randomized experimental study
design, or b) non-experimental longitudinal data on student achievement. Unfortunately,
little research on teacher testing or licensing meets either standard and the research that
does is tentative and inconclusive.

        Randomized experimental design usually provides the best data for education
policy research. With respect to alternative certification, this would involve estimating
the effect of teachers with AC and traditional preparation on student achievement through
random assignment of students to classrooms with the two types of teachers, otherwise
comparable (e.g., similar experience), within a school. Unfortunately, at present there is
no research on teacher credentials or training that meets this standard, although the
Institute for Education Sciences of U.S. Department of Education is promoting such
studies (Moesteller and Boruch, 2002; U.S. Department of Education, undated).
Mathematica Policy Research (MPR) has two random assignment studies of teachers
under way. The first compares the performance of Teach for America teachers to
conventionally prepared teachers ((http://www.mathematica-
mpr.com/3rdLevel/teach4amer.htm). A second study compares AC with traditionally
trained teachers at several sites across the nation (Mayer, et.al. 2003).1

         If randomization is not feasible, and often it is not, then one must rely on non-
experimental data to evaluate education policy. If we are to measure the contribution of a
classroom teacher to student achievement, it is necessary to control for prior achievement
of the student before he or she enters the classroom. Ideally, researchers would pretest
the students in the fall and test them again in the spring. The difference in these scores,
averaged over the classroom, would be a measure of a teacher’s “value-added.” If
students are not pretested in the fall, then it is also possible to use test scores the previous
spring, or for more than one previous year (longitudinal achievement data). Large
longitudinal data files have formed the basis for the most sophisticated current research
on teachers and teacher effects on student achievement (Sanders and Horn, 1994;
Hanushek and Rivkin, 2004; Rivkin, Hanushek, and Kain, 2001; Aronson, Barnow, and
Sanders, 2003; Betts, Zau, and Rice, 2003).

       Studies that do not have a rigorous study design, i.e., with randomization or
controls for prior student achievement, are likely to produce seriously biased estimates of
1
 Unfortunately, in both cases, only elementary teachers are assessed, which may limit the
generalizability of the study, since AC programs are often focused on recruiting
secondary school teachers with strong content knowledge in subject areas. For example,
McKibbon (2002) notes that one of the original motivations for the California Internship
program was to develop a fast track entry program for professionals laid off in the
aerospace industry.




                                                                                               2
the effect of teacher certification or other teacher characteristics on student achievement.
The reason is that they do not adequately control for the socioeconomic background of
students in classrooms and these omitted student SES factors are correlated with teacher
credentials. In the language of econometrics, we say that these cross-section studies of
teacher credentials suffer from “omitted variable bias.”2


         The number of studies of teacher certification that meet these minimum
methodological standards outlined above is very small. A survey of the literature in the
Spring 2003 Review of Education Research (Wayne and Youngs, 2003) found only two
studies of teacher certification that were peer-reviewed, used longitudinal student-level
achievement data, and controlled for student SES. The results of these studies (both by
Goldhaber and Brewer and both using the National Longitudinal Educational Survey of
1988) had mixed results. They did find a small positive effect of math teacher
certification on math achievement, but no statistically-significant effect of science teacher
certification on science achievement. Recent surveys of the literature by Hanushek and
Rivkin (2003) focusing on “high quality” studies that meet the standards described above
find little evidence linking teacher credentials to student achievement.

        Three recent surveys of teacher quality research take up the question of the
teaching performance of AC teachers (Allen, 2003; Wilson, Floden, and Ferrini-Mundy,
2001, Mayer, et.al. 2003). All three set a lower standard for inclusion of studies that
what I have argued for above. Specifically, they include in their surveys descriptive and
correlational studies that do not control for prior student achievement. Allen (2003)
concludes that literature to date provides “limited” support for the proposition that AC
2
  A recent study by Hoxby (2001) highlights the importance of these socioeconomic
variables and their potential for producing bias in teacher effects research. Hoxby
analyzed the effect of family, neighborhood, and school input variables on student
achievement and educational attainment using two large nationally representative
longitudinal studies of students (the National Educational Longitudinal Survey, NELS88,
and the National Longitudinal Survey of Youth, which began in 1979). The list of
variables included in each of the areas is extensive. Family variables include parent's
education, family income, student race and ethnicity, books at home, etc. The school
input variables include per-pupil spending, average class size, average teacher salary,
maximum teacher salary, percent of teachers with MA's, average experience of the
teacher, teacher certification status, and other information on school resources.
Community variables include income and demographic data on households in the school
district and city. Hoxby compared the percent of the variation in student achievement on
various field tests (math, reading) explained by each of these sets of factors. For every
test, the percent of the variation explained by the family variables far exceeded the school
input variables. The family variables explained from 34 to 105 times as much variation in
student achievement test scores as the school input variables. She also examined years of
schooling completed at age 33. Family variables explained 19 times as much variation in
student educational attainment as did school inputs.




                                                                                               3
programs can produce teachers who are as effective as traditionally-trained teachers.
Wilson, Floden, and Ferrini-Mundy, 2001 take up a slightly different question and ask
“what are the components and characteristics of high-quality alternative certification
programs?” They conclude that the existing literature is “… limited and has produced
decidedly mixed findings.” (Wilson, Floden, and Ferrini-Mundy, 2001). Of course,
answering the question they pose for AC programs is a tall order since they also note that
the existing literature cannot answer these same question for traditional programs either.
Unlike the earlier papers, which survey the more general literature on teacher quality,
Decker, et. al. 2003 focus only on the question of AC and student achievement. They
conclude that the exiting literature is not sufficiently rigorous (for reasons I have noted)
to draw any conclusions about the AC and student achievement.

Academic Ability and Content Knowledge

        Since there is no direct evidence on the relative performance of AC teachers, it is
important to gather indirect evidence on their quality. The slender research that does
exist on teacher quality suggests that teachers with better general academic skills, and, for
teachers in specialty areas such as science and math, better specific content knowledge or
coursework are associated with larger student achievement gains (Whitehurst, 2003).
Thus it is useful to have studies that measure the academic skills of AC and traditional
teachers. While states have published anecdotal or fragmentary information on the
academic skills of teachers, there has been little systematic compilation of such data.
Particularly useful are pass rates or scores on teacher licensing examinations, college
GPA’s and coursework in teaching fields, and the selectivity of colleges attended by AC
and traditional route candidates.

Retention of AC Teachers

         How does the turnover of AC and traditionally-prepared teachers compare? A
simple comparison of turnover rates by years of experience of ACP and traditionally
trained teachers may be a misleading indicator of AC teacher’s commitment to teaching.
It is well established in the research literature that schools with high concentrations of
minority and poor students have higher teacher turnover rates. Now suppose the
propensity to quit is the same for traditional and AC teachers in similar circumstances but
AC teachers are disproportionately concentrated in poorer or high poverty schools. Then
we will tend to observe higher turnover rates of AC teachers, but this is a biased estimate
of the true difference in turnover propensity of the two types of teachers.

        I am aware of no careful statistical study of teacher turnover comparing the two
types of teachers. The Texas State Board for Educator Certification (SBEC) reports
turnover rates of teachers with different types of certification for high and low poverty
school districts. These SBEC data are posted on their web site (www.sbec.state.tx.us)
and provides interesting comparisons of AC and traditional teaching candidates. An
example is in Figure 1 below. However, a multivariate statistical analysis would provide
a more convincing demonstration of differences in the propensity to quit for teachers with
different types of certificates. In this regard, it is important to have good controls for



                                                                                            4
working conditions in schools as well as teacher compensation for AC and traditionally
trained teachers.

        The survey discussed above by Allen (2003) also addresses the question of
whether the retention of AC teachers matches that of traditionally trained teachers. He
finds “limited” support for the proposition that the short-term turnover of AC teachers is
comparable to that of traditionally prepared teachers, but “inconclusive” evidence about
the long-term differences. However, this literature is very thin, and largely focuses on
small samples of teachers in particular AC programs. It is possible that these results
would not generalize to a larger universe of AC programs.




                                                                                             5
Figure 1.




High Poverty / High Minority Middle Schools

Source: Texas State Board for Educator Certification (SBEC)

http://www.sbec.state.tx.us/SBECOnline/reprtdatarsrch/tchrattremploy/Attrition%20of%
20Teachers%20in%20High-Poverty,%20High-
Minority%20Schools%20by%20Cert.%20Route%20(Class%20of%201995).pdf

Cost-Effectiveness of AC

        Suppose that AC and traditional teachers are, on average, equally effective,
however, suppose that the turnover rate of AC teachers is higher. Does this mean that the
benefit-cost ratio of the former is actually lower? In fact it does not. In order to establish
that, we need to compare the training costs of the two types of teachers. Here is where
AC may have a considerable advantage. The reason is that AC training, by its very
nature, is targeted to a person filling an actual teaching vacancy, whereas traditional
university-based programs are not. University-based programs have two problems. First,
there is a persistent mismatch between the degrees awarded by field and vacancies.
University programs typically produce large numbers of elementary education majors –
far in excess of need – whereas they produce relatively few graduates in areas such as



                                                                                             6
math and science education, or special education. Table 1 illustrates this problem in
Missouri. The number of newly certified elementary school teachers (predominantly
graduates recommended for certification by schools of education) far exceeds the number
of elementary school vacancies. By contrast the number of new certs in math was is well
below the number of vacancies.


Table 1: Missouri First-Time Teaching Certificates Issued and Vacancies, 1997-98

                  First-Time         Public School      Ratio (1)/(2)
                  Certs              Vacancies
Elementary        2000               1396               1.43
Education
Science           300                262                1.15
Math              193                270                .72

Source: Podgursky, et. al. 1999.
First-time time certs = Did not teach in a Missouri public school and did not previously
hold a Missouri teaching certificate in another field.


This suggests that $10,000 spent on producing an additional education major is less likely
to meet a public school vacancy than $10,000 spent on an AC candidate, who is actually
filling a vacancy.

        In addition to an imbalance between supply and demand by field, a second
problem is that many education school graduates never teach in a public school, or do so
only briefly. In this case, the public investment in their specialized pedagogical training
has been wasted. In study of Missouri public higher education graduates cited above, we
found that of 2239 1994-95 graduates with baccalaureate degrees in education, only 55
percent were teaching in a Missouri public school in 1998-99. (Podgursky, et. al. 1999).
Again, because AC training is targeted to an existing vacancy, much less training is
wasted.

        Return to the Texas statistics in Figure 1 above. Let’s suppose that the cost of
training for an ACP and a traditional teacher are $10,000 but only 55 percent of
traditional candidates teach in a public school classroom as compared to 100 percent of
ACP candidates. Then after six years, on average only $3476 of the training for a
traditional teacher remains in a public school classroom as compared to $6910 for an AC
candidate.

         These are back-of-the-envelope calculations. However, a comparison of cost-
efficiency between AC and traditional teachers must take account not only of differences
in attrition after initial teaching employment, but also between the delivery of training
and initial employment. I have seen no careful study making such cost comparisons.




                                                                                            7
Conclusion

        At present there is little reliable research to inform most policy with regard to
teacher training and licensing. However, there is now a strong emphasis at the new
Institute for Education Sciences in developing such a research base in teaching. In
addition, researchers are beginning to exploit large data files linking student achievement
records longitudinally to examine how teacher characteristics affect student achievement
gains. These data files hold great promise for providing data sets to monitor student
achievement gains in schools and classrooms.

        To date, studies that have estimated teacher effects on student achievement gains
using these large longitudinal data files find that teacher performance is highly
idiosyncratic, in that there is considerable variation in teacher performance across
classrooms but these differences are not explained by differences in measured teacher
characteristics such as type of license, experience, MA degrees, test scores, etc
(Goldhaber, 2003; Podgursky, 2004). This suggests that AC programs that combine a
search over a larger applicant pool with careful screening and mentoring hold promise
has a mean to identify and cultivate superior teachers. States should continue their
experimentation with AC programs, but make better use of their testing and
administrative data to monitor their effectiveness.




                                                                                            8
References

Allen, Michael B. 2003. Eight Questions on Teacher Preparation: What Does the
Research Say? Denver: Education Commission of the States. (July)
http://www.ecs.org/ecsmain.asp?page=/html/publications/home_publications.asp?am=5

Aaronson, Daniel, Lisa Barrow, and William Sander. 2003. “Teachers and Student
Achievement in the Chicago Public High Schools” Working Paper. Research
Department. Federal Reserve Bank of Chicago.

Betts, Julian R., Andrew C. Zau, and Lorien A. Rice. 2003. Determinants of Student
Achievement: New Evidence from San Diego. Sacramento, CA: Public Policy Institute
of California. http://www.ppic.org/main/publication.asp?i=321

Feistritzer, Emily C and David C. Chester. 2002. Alternative Teacher Certification: A
State-by-State Analysis: 2002. Washington DC: National Center for Education
Information.

Goldhaber, Dan. 2002. “The Mystery of Good Teaching.” Education Next Vol. 2
(Spring). pp. 50-55. www.educationnext.net

Hanushek, Eric. A. 2003. “The Failure of Input-Based Resource Policies.” The
Economic Journal. Vol. 113 No. 485 (February). pp. F64-F98.

Hanushek, Erik A. and Steven G. Rivkin. 2004. “How to Improve the Supply of High
Quality Teachers” Brookings Papers in Education Policy: 2004. Washington, DC:
Brookings Institution. Forthcoming.

Hoxby, Caroline. 2001. “If Families Matter Most, Where Do Schools Come In?” in
Terry M. Moe (ed.) A Primer on America’s Schools. Stanford University: Hoover
Institute Press.

Mayer, Daniel P., Paul T. Decker, Steven Glazerman, Timothy W. Silva. 2003.
“Identifying Alternative Certification Program for an Impact Evaluation of Teacher
Preparation.” 8940-400. Washington, DC: Mathematica Policy Research, Inc. (April).
http://www.mathematica-mpr.com/PDFs/identify.pdf

Michael McKibbon. 2002. “Implementing Alternaitve Routes to Teacher Preparation
and Certification in California.” Presentation to the California School Boards
Association. (November).

Mosteller, Frederick and Robert Boruch (eds.) 2002. Randomized Trials in Education
Research. Washington DC: Brookings Institution.




                                                                                        9
Podgursky, Michael, Donald Watson, Mark Ehlert, Michael Walker, William Foster.
1999. A Statistical Analysis of the Labor Market for Missouri Public School Teachers,
1994-95 to 1998-99. http://web.missouri.edu/~econ4mp/Downloadable_Papers.htm

Podgursky, Michael. 2004. “Improving Academic Performance in U.S. Public Schools:
Why Teacher Licensing is (Almost) Irrelevant.” in Richard Hess, Andrew Rotterham, and
Kate Walsh (eds.) A Qualified Teacher in Every Classroom? Appraising Old Answers
and New Ideas. Cambridge, MA: Harvard Education Press.

Rivkin, Steven G., Eric A. Hanushek, and John F. Kain. 2001. “Teachers, Schools, and
Academic Achievement.” Cambridge, MA: National Bureau of Economic Research.

Sanders, William L. and Sandra P. Horn. 1994. “The Tennessee Value-Added
Assessment System (TVAAS): Mixed Model Methodology in Educational Assessment.”
Journal of Personnel Evaluation in Education. Vol. 8, pp. 299-311.

U.S. Department of Education. Institute of Education Sciences. Undated. Random
Assignment in Program Evaluation and Intervention Research: Questions and Answers
http://www.mathematica-mpr.com/PDFs/randomassign.pdf

Wayne, Andrew J. and Peter Youngs. 2003 “Teacher Characteristics and Student
Achievement Gains: A Review.” Review of Education Research. Vol. 73 No. 1
(Spring) pp. 89-122.

Whitehurst, Grover. 2003. “Scientifically Based Research on Teacher Quality: Research
on Teacher Preparation and Professional Development.” in U.S. Department of
Education. Meeting the Highly Qualified Teachers Challenge: The Secretary’s Second
Annual Report on Teacher Quality. Washington D.C.
http://www.ed.gov/offices/OPE/News/teacherprep/index.html

Wilson, Suzanne M., Robert E. Floden, Joan Ferrini-Mundy. 2001. Teacher Preparation
Research: Current Knowledge, Gaps, and Recommendations. A Research Report
Prepared for the U.S. Department of Education. Seattle, WA: Center for the Study of
Teaching and Policy. (February).




                                                                                        10