What Are We Trying to Measure and How Should by xyd32971

VIEWS: 16 PAGES: 54

									What Are We Trying to Measure and How Should We Measure It? Conceptual and
Technical Issues in the Construction of High School Dropout and Completion Rates




                              John Robert Warren

                           Department of Sociology
                          Minnesota Population Center

                             University of Minnesota


                                October 9, 2008
   What Are We Trying to Measure and How Should We Measure It? Conceptual and
   Technical Issues in the Construction of High School Dropout and Completion Rates


This chapter focuses on the conceptual and technical differences between measures used to

characterize high school dropout and completion rates in the United States.          It includes

discussions of the strengths and weaknesses of each measure, as well as the strengths and

weaknesses of the data that underlie them. It concludes with a discussion of which sorts of

measures should be used for given purposes, and makes recommendations for changes to how

dropout and completion data are collected and how measures are constructed.

WHY ARE HIGH SCHOOL COMPLETION AND DROPOUT RATES SO INCONSISTENT?

       Most observers begin with a sense that it should be straightforward to quantify the U.S.

high school dropout rate (or else the high school completion rate, which many take to be one

minus the high school dropout rate). Intuitively, one need only divide the number of dropouts or

completers by the number of people at risk of dropping out or completing.

       However, confusion sets in when those observers are confronted with widely publicized

high school completion and dropout rates. What was the high school completion rate in 2005?

According to the U.S. Department of Education’s Digest of Education Statistics 2007, “74.7

percent of public high school students graduated on time [in 2004-05]” (Snyder, Dillow, and

Hoffman 2008: 3). Similarly, “about 71 percent of 9th graders make it to graduation four years

later [in 2005]” according to Diploma Counts 2008 (Editorial Projects in Education 2008).

However, the Digest also reported a dropout rate for 16 to 24 year olds of only 9 percent in 2005

(Table 105) and the Annie E. Casey Foundation’s Kid’s Count 2008 Data Book reported a

dropout rate for 16 to 19 year olds of only 7 percent in that year (2008). How should we

understand a 70 to 75 percent completion rate in light of a dropout rate below 10 percent?



                                               1
       The situation gets more confusing at the state level. In Texas in 2005, for example, the

percentage of 9th graders who completed high school was either 74 percent or 69 percent,

according to the Digest of Education Statistics and Diploma Counts 2008, respectively.

However, the Texas Education Agency (TEA) reported that the “overall graduation rate for the

[Texas] class of 2005 Grade 9 cohort was 84.0 percent” (Ramirez, McMillion, and Roska 2006:

viii). In the same report, the TEA noted that the “longitudinal dropout rate for the class of 2005

Grade 9 cohort” was only about 4 percent. The Kid’s Count 2008 Data Book reported a dropout

rate for Texas 16 to 19 year olds of about 8 percent in 2005. How should Texans make sense of

high school completion rates that range from 69 to 84 percent in 2005, especially in light of

dropout rates below 10 for that year?

       The picture is no clearer at the district level.     For example, in the Madison (WI)

Metropolitan School District in 2005 the percentage of 9th graders who completed high school

was about 69 percent according to Diploma Counts 2008 but about 80 percent according to the

Wisconsin Department of Public Instruction.

       Why do highly publicized estimates of the high school completion and dropout rate differ

so much from one another? The answer has to do with differences in what various estimates are

designed to accomplish; differences across estimates in the conceptual and technical definition of

both the numerator and denominator of the rates; and differences in the accuracy of the data used

to produce these estimates.

Estimates Differ Because They Are Intended To Measure Difference Concepts

The first reason that dropout and completion rate estimates differ so much from one another is

that they differ with respect to what they are trying to measure. Analysts operationalize high

school completion and dropout rates in different ways because they have different conceptual or




                                                2
practical reasons for making those measurements. There are at least three main reasons for

measuring high school completion and dropout rates.

       One reason to measure high school completion and dropout rates is to describe the

amount (or lack) of human capital in a population. Motivated by this purpose, the objective is to

quantify the share of people who have not completed high school and who are no longer

enrolled.   A second reason to measure high school completion and dropout rates is to

characterize schools’ “holding power” (Hartzell, McKay, and Frymier 1992). How well do

schools move young people from the first day of high school to successful high school

completion?   A third reason to measure these rates is to characterize students’ success at

successfully navigating high school from beginning to end. In contrast to measures of “holding

power” --- which are intended to describe an attribute of schools --- these measures are designed

to describe an attribute of students. How well do students manage to move themselves from the

first day of high school to successful high school completion?

       For the purposes of describing amounts of human capital in a population, the timing of

high school completion --- how long ago or at what age people completed high school --- is not

important. Nor does it matter exactly how young people complete high school --- that is, by

obtaining a diploma, a GED, obtaining a certificate of completion, completing an adult education

program, or some other way. Any young person who has completed high school is considered to

have surpassed a necessary threshold, regardless of their age at the time of surpassing that

threshold. For the latter two purposes, however, both the timing of high school completion and

the manner in which young people complete high school can be important. For example, schools

may only be deemed successful at moving a young person through to completion if those young

people obtained regular diplomas “on time,” typically within four years.




                                                3
       Given these differences in intended purpose, it becomes less puzzling to read in the

Digest of Education Statistics that “74.7 percent of public high school students graduated on

time” despite the fact that only 9 percent of 16 to 24 year olds were dropouts in 2005 (Snyder,

Dillow, and Hoffman 2008). The former estimate is explicitly intended to describe the share of

young people who complete high school on time and by obtaining a diploma---an attribute of

schools. The latter estimate is clearly intended to describe the share of young people who lack

the human capital associated with high school completion. Presumably many of the 100 – 74.7 =

25.1 percent of the 9th graders in the fall of 2001 who did not go on to graduate from high school

with a diploma by spring 2005 were still enrolled or will compete high school later, via a GED or

another alternative credential.    Given the 9 percent dropout rate, we might presume that

eventually about 100 – 9 = 91 percent of these young people will eventually complete high

school one way or another.

Estimates Differ By Construction

The second reason that high school dropout and completion rates differ so much from one

another is that there are technical differences in how various measures are constructed. This is

true even among measures intended to quantify the same concept. All high school completion

and dropout rates boil down to a ratio: In the numerator is the number of high school completers

or dropouts and in the denominator is the number of people at risk of completing or dropping out

of high school. However, even when analysts are trying to measure the same concept, they

frequently differ with respect to who counts as a “completer” or “dropout” in the numerator and

who is considered “at risk” in the denominator.

       If we are measuring high school completion rates, who should be counted as a “success”

in the numerator? Those who obtain high school diplomas surely count. But what about GED




                                                  4
recipients? Or those who obtain certificates of completion? And should we count a student as a

“success” if they do not complete high school in a timely manner? If we are measuring high

school dropout rates, who should be counted as a “failure?” Everyone agrees that students who

leave school, never return, and never obtain any high school credential should count as dropouts.

But what about students who drop out and then obtain a GED or complete a state-certified adult

education program? Or students who drop out for a while and then re-enroll in high school?

Even when analysts agree on the numerator and can quantify “success” or “failure,” they

frequently differ in their definition of the denominator --- or those at risk of “success” or

“failure.” How should the denominator account for geographic migration into or out of a

population of interest? How should students who are expelled or otherwise “pushed” out of high

school be counted in the denominator?

Estimates Differ Because of Variability in the Underlying Data Used to Construct Them

The third reason that high school dropout and completion rate estimates differ so much from one

another is that they differ with respect to the accuracy of the data used in their construction. As a

result, even when measures are intended to quantify the same concept and agree on the technical

definition of the numerator and denominator, their estimates may differ quite a bit. There has

been considerable debate in recent years, for example, about the accuracy of measures that rely

on individuals’ self-reports of their high school completion status (Mishel and Roy 2006; Warren

and Halpern-Manners 2007; Heckman and LaFontaine 2007). Another prominent example is the

recent debate about how to accurately measure the number of 9th graders who begin high school

for the first time in a particular year (e.g., Warren 2005). These and other accuracy issues are

described in more detail below.




                                                 5
A CATALOGUE OF HIGH SCHOOL COMPLETION AND DROPOUT RATES

This section includes definitions of status rates, event rates, cohort rates, and synthetic cohort

rates and reviews of their intended purposes, technical definitions, and data sources. Subsequent

sections include discussions of issues of accuracy and bias, both in the technical definition of

particular measures and in the underlying data typically used to produce them.

Status Rates

A status rate reports the fraction of a population that falls into a population sub-category at a

given point in time. The most common and visible example is the status dropout rate --- the

share of people in a population who are high school dropouts --- but status enrollment rates and

status completion rates are occasionally reported as well. For example, in Dropout Rates in the

United States, 2006 the U.S. Department of Education reported that 9.3 percent of 16 to 24 year

olds were not enrolled in school and did not have any high school credential in October of that

year (Laird et al. 2008). In that same month, 87.8 percent of 16 to 24 year olds were status

completers --- that is, they were not enrolled in high school and held some sort of high school

credential. These measures’ focus on 16 to 24 year olds --- as opposed to the population of all

adults --- is designed to offer cohort-specific information. Some analysts use slightly different

age ranges. For example, the Annie E. Casey Foundation reports status dropout rates for 16 to

19 year olds in its annual reports (e.g., 2008).

       The numerator of the status “dropout” rate reflects the number of people who have not

obtained any high school credential and are not working toward one. The fact that many

dropouts subsequently re-enroll in high school, obtain GEDs, or earn high school credentials in

other ways is immaterial; so is the age at which young people complete high school.




                                                   6
       Status dropout and completion rates are usually calculated using cross-sectional data on

individuals in the target population. All that is required is information about individuals’ age,

enrollment status, and high school completion status.              Status dropout (and occasionally

completion) rates are routinely calculated using data from the decennial U.S. Census, the Current

Population Survey (CPS), and the American Community Survey (ACS). The Census and (in

some years) ACS data contain a sufficient number of observations to report status dropout and

completion rates at the national, state, and sub-state levels.

       All status dropout and completion rates are measures of the amount (or lack) of human

capital in a population. They are very poor measures of schools’ holding power or of young

people’s success at navigating the secondary school system and persisting in school. This is

because such measures do not differentiate between those who obtain high school credentials by

graduating with a diploma and those who obtain GEDs or other alternative credentials, and do

not consider whether credentials were obtain after the typical ages of high school completion. A

very low status dropout rate may reflect very high holding power of schools, or it may belie a

situation in which schools have very low holding power and many young people obtain alternate

credentials in their late teens or early twenties. What is more, status rates do not consider the

location of high school dropout or completion. A geographic area with a very low status dropout

rate may have schools with very high holding power, or it may attract many in-migrants who

either are enrolled or have high school credentials.             For example, counties with higher-

technology industries or large post-secondary institutions tend to have relatively lower status

dropout rates. This likely says more about the human capital of people who move to those

counties and less about those counties’ schools’ holding power.




                                                  7
Event Rates

An event rate reports the fraction of a population that experiences a particular event over a given

time interval; by definition, everyone in the population is at risk of experiencing that event

during the period. The most frequently reported example is the event dropout rate --- the

proportion of students who exit school during an academic year without completing high school

(Komiski 1990). Between October 2005 and October 2006, 3.8 percent of 15 to 24 year olds

who began that year in grades 10, 11, or 12 left school without obtaining a diploma or an

alternate credential (Laird et al. 2008). Some states report variants of this measure among their

students. For example, North Carolina (2004) eliminates the age restriction and reports these

rates for 9th through 12th graders and separately for 7th through 12th graders.

        It is intuitively tempting to use a series of grade-level-specific event dropout rates to infer

an event completion rate that describes completion across the four years of high school. Imagine

four separate event dropout rates, focusing on 9th, 10th, 11th, and 12th graders in academic years

X, X+1, X+2, and X+3, respectively --- call them E9th, X, E10th, X+1, E11th, X+2, and E12th, X+3. One

might take (1- E9th, X) × (1- E10th, X+1) × (1- E11th, X+2) × (1- E12th, X+3) = G to represent the fraction

of incoming 9th graders who persist across the four years of high school, but this would be

inaccurate. First, the tendency for students to repeat grades downwardly can bias these grade-

specific event dropout rates; E9th, X, in particular, will be downwardly biased in most populations

and thus G will be upwardly biased. Second, students who drop out of two different grades

upwardly bias these grade-specific event dropout rates and downwardly bias G.

        Event dropout and completion rates can be calculated using either cross-sectional data or

longitudinal data on individuals in the target population. All that is required is information about

individuals’ enrollment status in two consecutive academic years, their completion status in the




                                                    8
second of those years, and (under some formulations) age.           Enrollment status is typically

measured at the beginning of each academic year so that event dropout measures can more

clearly represent the dropout during well-defined academic years. Using longitudinal data,

current enrollment status is measured in each of two consecutive academic years. This is, for

example, how North Carolina estimates its event dropout rate for 7th to 12th graders (2004) using

longitudinal administrative data. Using cross-sectional data, enrollment status in the previous

year is measured retrospectively and current enrollment status is measured contemporaneously.

For example, the October supplement to the CPS asks about enrollment in both the current and

the previous October. The event dropout rate is routinely calculated using data from the October

supplement to the CPS. Because of sample size restrictions, it is sufficiently reliable only when

calculated for the nation as a whole and for a few larger states (Winglee et al. 2000).

       Because they are measures of the share of a population that experiences a particular event

--- typically high school dropout --- over the course of a specific time interval, event dropout and

completion rates can be used either to describe schools’ holding power or else to describe young

people’s ability to successfully navigate the school system. Whether an event dropout rate fairly

characterizes the former or the latter is a matter of (a) how “success” and “failure” are defined in

the numerator and (b) how the population is defined in the denominator. If the goal is to

measure something about schools --- their “holding power” --- then the numerator is determined

by how schools define “success” (e.g., they are explicit about whether GEDs and other

alternative credentials are treated as equivalent to regular diplomas) and the denominator is

restricted to those continuously residing in a well-defined geographic area (typically school

districts or states). The resulting event dropout rate should thus speak to the experiences of only

those students for whom the school district or state is formally responsible.      If the goal is to




                                                 9
measure something about students --- the rate at which they succeed in navigating the secondary

school system --- then the denominator need not be restricted to those who continuously reside in

a particular geographic area.

Cohort Rates

Cohort rate are derived from longitudinal data on individuals, all of whom share a common

characteristic at one point in time (e.g., they are all students entering high school). Cohort rates

report the fraction of individuals who transition into a particular status by a subsequent point in

time. In the case of dropout and completion rates, students are usually followed from the

beginning of high school until at least the normal time of high school completion. Cohort rates

typically report the fraction of students who ended the period as high school dropouts or

completers. For example, Mishel and Roy (2006) use data on students who were 8th graders in

the spring of 1988, and report that 12 years later 83.0 percent had obtained regular diplomas, 7.7

percent had earned GEDs, and 9.3 percent had obtained neither credential. Likewise, the Texas

Education Administration used a cohort rate to determine that “[f]or the class of 2007 Grade 9

cohort, 78.0 percent of students graduated, and 2.0 percent received GEDs” (McMillion,

Ramirez, and Roska 2008).

       The longitudinal data used to compute cohort dropout and completion rates has typically

come from one of two places. The first is state- or district-level administrative data on students

who are under the supervision of schools. Here, analysts typically select students who were first-

time 9th graders in the fall of one academic year (constituting the denominator), and then count

the number of students who obtain diplomas (for cohort graduation rates), obtain any secondary

credential (for cohort completion rates), or leave school without obtaining any credential (for

cohort dropout rates). As with event dropout rates, states and districts differ with respect to what




                                                10
counts as “success” and “failure” in the numerator of cohort dropout rates computed using

administrative data. Some agencies count only regular diplomas as “successes,” while other also

count GEDs and alternative credentials.

       The second source of longitudinal data used for computing cohort dropout and

completion rates is the series of major longitudinal studies undertaken by the National Center for

Education Statistics (NCES).      These include the 1972 sample of seniors in the National

Longitudinal Study of the High School Class of 1972 (NLS); the 1980 and 1982 samples of

sophomores and seniors in High School & Beyond (HS&B); the sample of 8th graders in the

1988 National Educational Longitudinal Study (NELS); and the sample of sophomores in the

2002 Educational Longitudinal Study (ELS). In each case, NCES began by selecting stratified,

nationally representative samples of students in the focal grade(s) in the base year. Those

students were then followed periodically, allowing for the computation of cohort dropout and

completion rates. Here, the denominator of cohort rates consists of all sampled students who

were in the same grade at the same point in time. The numerator can be defined by the analysts,

as each data set includes information about whether, when, and how students completed school.

       Although similar in many ways, cohort dropout and completion rates computing using

longitudinal administrative data differ in a few ways from those computed using longitudinal

data (from NCES or elsewhere) on samples of students. Most importantly, because the former

are constructed in order to characterize dropout and completion rates in a particular state or

school district, the denominator of these rates must be adjusted to account for entry and exit into

the population of interest. Most prominently, the denominators of such rates are frequently

adjusted to account for migration into and out of the jurisdiction in question and for transfers to

other educational settings (like GED or adult education programs). Likewise, some states or




                                                11
localities adjust the denominator to account for students who are incarcerated or expelled or who

die. Conceptually, these students are no longer considered to be at risk of dropping out from or

completing high school. Cohort dropout and completion rates computing using NCES or other

longitudinal survey samples are typically computed for the nation as a whole, and so issues of

migration into and out of the population of interest are less of a concern.

       Cohort dropout and completion rates computed using longitudinal administrative data

from schools or districts are typically intended to characterize schools --- that is, to describe their

holding power or their success in graduating students.            Spurred by the language in the

Elementary and Secondary Education (“No Child Left Behind,” or NCLB) Act of 2001, all 50

states have entered into the National Governors Association’s (NGA) Compact on State High

School Graduation Data. Among other things, the compact stipulates that all states will construct

a “standard, four-year adjusted cohort graduation rate” which is calculated by “dividing the

number of on-time graduates in a given year by the number of first-time entering ninth graders

four years earlier.”   In the numerator, graduates are those receiving a regular high school

diploma in four academic years. In the denominator, adjustments are made to account for (among

other things) in- and out-migration.

       Cohort dropout and completion rates computing using longitudinal data on samples of

students are typically intended to characterize students --- that is, to describe their success in

navigating through post-secondary educational institutions. The difference in purpose is partly

attributable to the nature of the longitudinal cohort samples themselves. The NCES cohorts, for

example, are infrequently conducted and are not sufficiently large to characterize school districts

or even states; they are thus not useful for NCLB or other school accountability purposes.

Because student-level microdata from NCES and other sample surveys are publicly available,




                                                  12
analysts are generally more focused on understanding the individual student-level correlates of

high school completion or dropout. Researchers are also at liberty to construct cohort dropout

and completion rates that suit their own purposes and that differ with respect to the technical

definitions of both the numerator and denominator. Finally, because NCES and similar surveys

include students in both public and private schools, researchers are in a better position to

generalize their estimates to all students.

Synthetic Cohort Rates

Synthetic cohort rates are designed to approximate true cohort rates in situations in which

longitudinal data are not available. Beginning with a count of the number if individuals who

share a common characteristic at one point in time (e.g., students entering high school), synthetic

cohort rates estimate the fraction of individuals who transition into a new status (e.g., high school

completion) by a subsequent point in time. In the case of dropout and completion rates, this

frequently takes the form of a rate with the number of dropouts or completers in a cohort in the

numerator and an estimate of the number of people at risk of dropping out or completing in the

denominator. For example, the NCES Averaged Freshman Graduation Rate “estimates the

proportion of public high school freshmen who graduate with a regular diploma 4 years after

starting 9th grade” (Laird et al. 2008: 2).

       Synthetic cohort dropout and completion rates are primarily based on administrative data

collected from schools.     All that is required is information about the number of students

completing and/or dropping out at a point in time and the estimated number of students at risk of

doing so; this information is usually tied to specific cohorts of incoming students.            The

denominator usually takes the form of an estimate of the number of first time 9th graders in a

particular cohort, which can be calculated in various ways (described below). Many states, for




                                                 13
example, have computed synthetic cohort completion rates by dividing the number of completers

in the spring of academic year X by the sum of the number of completers in academic year X and

the number of dropouts in academic years X, X-1, X-2, and X-3.

       In recent years, the most widely publicized synthetic cohort graduation rates have been

computed by dividing the numbers of regular diploma recipients in academic year X by an

estimate of the number of first-time 9th graders in academic year X-3. Prominent examples

include Swanson’s Cumulative Promotion Index (Editorial Projects in Education 2008; Swanson

and Chaplin 2003); the NCES Averaged Freshman Graduation Rate (Seastrom, Chapman et al.

2006; Seastrom, Hoffman et al. 2006); Greene’s Adjusted Completion Rate (Greene and Forster

2003; Greene and Winters 2006); and Warren’s Estimated Completion Rate (Warren 2005;

Warren and Halpern-Manners Forthcoming). These measures are all based on data from the

State Nonfiscal Survey of the NCES’ Common Core of Data (CCD). These CCD data --- which

are described in more detail below --- include annual counts of numbers of graduates and

numbers of students enrolled in each grade for all states and school districts in the United States.

All of these measures also make some effort to adjust the denominator --- which is based on the

total number of 9th graders --- to account for migration into and out of the “at risk” population

and to account for bias introduced by the fact that some 9th graders are not first-time 9th graders.

       As described above, true cohort dropout and completion are employed either to describe

schools’ holding power (when the rates are based on longitudinal administrative data) or to

describe students’ success at persisting in school (when the rates are based on longitudinal

samples of students). As approximations of true cohort rates, synthetic cohort rates have serious

limitations for both purposes. Synthetic cohort rates are frequently used to characterize the

holding power of states, districts, and schools’ --- as exemplified by the publicity surrounding




                                                 14
the Editorial Projects in Educations’ Diploma Counts (e.g., 2008) reports. However, they are

conceptually imperfect for this purpose because of their inability to distinguish “on time”

graduates from other graduates in the numerator --- all that is known is the total number of

graduates. As a result, they are less useful for accountability purposes because they do not meet

the graduation rate definition spelled out by NCLB or by the NGA Compact. When used to

characterize students’ success at persisting in school, this issue about numbers of “on time”

graduates is less consequential. However, synthetic cohort rates based on data from public

schools say nothing about the experiences of those students who attend private schools, and so

they are less useful as descriptors of students’ experiences. For all purposes, the inability to

adjust synthetic cohort rates’ denominators properly to account for migration and grade retention

leads to systematic bias under many conditions (as described in detail below).

       Although there are inherent weaknesses in synthetic cohort rates for either of their main

purposes, they do have one major advantage: They can be computed for every state and every

local education agency in the country in a technically consistent manner, and they are available

annually going back for many years. This allows for meaningful comparisons over time and

across locales.   The same cannot be said of true cohort rates: When they are based on

longitudinal administrative data, they are generally not computed in a consistent manner across

locales (although the NGA Compact may change that) or over time. When they are based on

longitudinal survey data on students, they cannot be generalized to districts or even (in many

cases) states. For any analyses of change in completion or dropout rates over time and/or across

locales, synthetic cohort rates are all that are available. This reality makes understanding their

conceptual and technical problems (described below) all the more important.




                                               15
ACCURACY AND BIAS IN DROPOUT AND COMPLETION RATES

The most widely used of the status, event, cohort, and synthetic cohort rates described above

come from one of three sources of data: Cross-sectional sample surveys (e.g., the CPS and the

Census), longitudinal sample surveys (e.g., NELS and ELS), and administrative data (e.g., CCD

and state agency data). Observers have frequently claimed that discrepancies between various

high school dropout and completion rates are due to biases or other limitations inherent in the

data that underlie these rates. One example is the 2006 debate about the national dropout rate

that took place in the pages of Education Week (Greene, Winters, and Swanson 2006; Mishel

2006); this debate largely amounted to disagreement about whether the CPS or CCD data are

better suited to measuring high school dropout rates at the national and state levels. Another

example is controversy that arises any time a district or state is accused of “cooking the books”

in order to have their official dropout rates appear very low. These controversies boil down to

disagreements about how to enumerate dropouts and completers in administrative data.

       How valid are claims of bias or flaws in the major sources of data used for constructing

the most widely used dropout and completion rates? Each of the status, event, cohort, and

synthetic cohort rates described above can be decomposed into two component parts: Their

counts of the number of people at risk of completing or dropping out (in their denominators) and

their counts of actual completers or dropouts (in their numerators). The sections that follow

describe these major data resources and discuss their biases and other flaws.

Overview of Main Data Resources

This section includes descriptions of the design and weaknesses of the major data resources used

to compute high school dropout and completion rates. These include cross-sectional sample

surveys (using the CPS as the main example); longitudinal sample surveys (using NELS as the




                                                16
main example); cross-sectional administrative data (using the CCD as the main example); and

longitudinal administrative data.

Cross-Sectional Sample Surveys

The most widely used cross-sectional sample surveys for the purposes of measuring high school

dropout and completion are the CPS, the decennial US Census, and (in more recent years) the

ACS. Because the decennial Census is not conducted more frequently, and because the ACS is a

relatively new resource, the CPS has served as the central cross-sectional data resource for

decades.

       The CPS is a monthly survey of more than 50,000 households and is conducted by the

Bureau of the Census for the Bureau of Labor Statistics. Households are selected in such a way

that it is possible to generalize to the nation as a whole and, in recent years, to individual states

and other specific geographic areas. Individuals in the CPS are broadly representative of the

civilian, non-institutionalized population of the United States. In addition to the basic

demographic and labor force questions that are included in each monthly CPS survey, questions

on selected topics are included in most months. Since 1968 the October CPS has obtained basic

monthly data as well as information about school enrollment—including current enrollment

status, public versus private school enrollment, grade attending if enrolled, most recent year of

enrollment, enrollment status in the preceding October, grade of enrollment in the preceding

October, and high school completion status. In recent years, the October CPS has also

ascertained whether high school completers earned diplomas or GED certificates.

       There are a number of conceptual and technical problems with CPS-derived measures of

high school dropout and completion, particularly when computed at the state level. First and

foremost, the sample sizes are not large enough to produce reliable estimates of rates of high




                                                 17
school completion or dropout at the state or sub-state levels (Kaufman 2001; Winglee et al.

2000). Even when data are aggregated across years—for example, in the Annie E. Casey

Foundation’s Kids Count (2008) measure—the standard errors of estimates for some states are

frequently so large that it is difficult to make meaningful comparisons across states or over time.

What is more, by aggregating across years the resulting measure no longer pertains to specific

cohorts of incoming students; as a result, CPS-based measures are not useful for assessing

schools’ holding power or for describing the dropout or completion rates of specific cohorts of

young people.

       Second, until 1987 it was not possible to distinguish high school completers from GED

recipients in the CPS; since 1988 October CPS respondents who recently completed high school

have been asked whether they obtained a diploma or GED, but there are serious concerns about

the quality of the resulting data (Chaplin 2002; Kaufman 2001). Third, as noted by Greene and

Winters (2002: 7), “[status] dropout statistics derived from the Current Population Survey are

based on young people who live in an area but who may not have gone to high school in that

area.” To the extent that young people move from state to state after age 18, CPS-based state-

level high school dropout rates—particularly status dropout rates based on 16 to 24 year olds—

may be of questionable validity (see also Kaufman, McMillen, and Bradby 1992). Fourth, some

observers have expressed concern about coverage bias in the CPS, particularly for race/ethnic

minorities. The CPS is representative of the civilian, non-institutionalized population of

household residents in the United States, and so young people who are incarcerated, in the

military, or homeless are not represented. To the extent that these populations differ from the rest

of the population with respect to frequency and method of high school completion, there is the

potential for bias in estimates. Finally, substantial changes over time in CPS questionnaire




                                                18
design, administration, and survey items have made year-to-year comparisons difficult (Hauser

1997; Kaufman 2001).

       It is possible to overcome some, but not all, of these limitations of the CPS by using data

from the ACS or the decennial Census. Sample sizes are larger, enhancing the reliability of

state- and urban-level estimates. Both the ACS and the decennial Census include individuals

who are institutionalized or in the military, and so generalizability is enhanced. However, it is

still not clear how accurately ACS respondents report whether they obtained GEDs or regular

high school diplomas. In addition, the ACS and Census share with the CPS the limitation that

sampled young people may not be living in the state in which they attended high school. As a

result, the ACS and the Census are useful for constructing status (but not event) dropout and

completion rates that describe the human capital of populations. Measures derived from the

CPS, the ACS, and the Census are not well suited to describing schools’ holding power or to

describing young people’s success at navigating the secondary school system.

Longitudinal Sample Surveys

The most widely used longitudinal sample surveys are those produce periodically by the NCES:

the 1972 sample of seniors in the NLS; the 1980 and 1982 samples of sophomores and seniors in

HS&B; the sample of 8th graders in the 1988 NELS; and the sample of sophomores in the 2002

ELS. This discussion focuses on NELS because it has been the center of so much research and

debate on the measurement of high school dropout and completion rates in recent years (Greene,

Winters, and Swanson 2006; Kaufman 2004; Mishel 2006; Mishel and Roy 2006).

       NELS is a longitudinal survey of the 8th grade student cohort of 1988. In the base year,

the sample included approximately 25,000 randomly selected students in 1,000 public and

private schools across the United States. In addition to the data collected from student interviews,




                                                19
NELS contains information from parents, school administrators, teachers, and student transcripts.

The initial student cohort has been followed-up on four occasions, in 1990, 1992, 1994, and

2000. Students who dropped out of school between survey waves were also interviewed, and in

early follow-ups, the sample was “freshened” with new sample members in order to make the

first and second follow-ups cross-sectionally representative of 1990 sophomores and 1992

seniors, respectively. The content of the surveys include students’ school, work, and home

experiences; educational resources and support; parental and peer influences; educational and

occupational plans and aspirations; delinquency; and many others (Curtin et al. 2002).

       For the purposes of measuring high school dropout and completion rates, the key feature

of NELS (and other longitudinal sample surveys) is that it includes information about whether

and when cohort members dropped out of school and whether and how they obtained secondary

school credentials. A key design feature of NELS is the availability of transcript data on high

school enrollment, dropout, and completion.          In the absence of coverage bias and non-

participation, NELS data would provide very accurate estimates of high school dropout and

completion rates --- albeit for a single cohort of young people.

       A number of technical issues raise questions about the accuracy of dropout and

completion rates based on NELS (Kaufman 2004); these issues also arise in the context of other

longitudinal sample surveys. First, the base year NELS sample excluded many students with

limited English proficiency or mental or physical disabilities; NCES gathered supplementary

information from these students later, but it is not clear how often this supplemental information

is used in calculating NELS-based dropout and completion rates. Second, as noted by Kaufman

(2004: 119), “[s]ince NELS is a sample survey, it is subject to the same potential for bias due to

non-response and undercoverage that CPS has.”             Third, transcripts data are frequently




                                                20
unavailable for dropouts or alternative completers; this is due in part to the logistical difficulties

inherent in collecting such data and in part to non-response by schools (Ingels et al. 1995). Some

of these problems are overcome by the use of sample weights in the NELS, but in the end NELS

--- like all longitudinal sample surveys --- has a difficult time retaining difficult to follow

populations like high school dropouts.

Administrative Data

Each state maintains its own system for counting the numbers of students who are enrolled in

each grade (usually at the beginning of each academic year) and the numbers of students who

obtain regular diplomas and other high school completion credentials. These counts are usually

aggregated up from the schoolhouse level, and are increasingly linked to longitudinal data

systems. At the national level, cross-sectional administrative data on enrollments and numbers

of completers are compiled as part of the CCD. The CCD is described in more detail below;

subsequent sections include comparisons of enrollment counts, numbers of completers, and

dropout and completion rates as computed using both CCD data and states’ administrative data.

        Compiled by NCES, the CCD is the federal government’s primary database on public

elementary and secondary education. Each year the CCD survey collects information about all

public elementary and secondary schools from local and state education agencies. One

component of the CCD—the State Nonfiscal Survey—provides basic, annual information on

public elementary and secondary school students and staff for each state and the District of

Columbia. CCD data from the State Nonfiscal Survey includes counts of the number of students

enrolled in each grade in the fall of each academic year and the number of students who earned

regular diplomas, who earned other diplomas, and who completed high school in some other

manner in the spring of each academic year. Although the State Nonfiscal Survey has collected




                                                 21
counts of public school dropouts since the 1991-1992 academic year, many states have not

provided this information or have provided it in a manner inconsistent with the standard CCD

definition of dropout (Winglee et al. 2000; Young and Hoffman 2002).

         Few observers have raised concerns with CCD enrollment counts. As shown below,

these counts closely correspond to states’ own reports of enrollment numbers (for good reason)

and to CPS estimates of the number of public school students enrolled in grades 8 and 9; as

discussed below, discrepancies in enrollment counts between thr CCD and the CPS are likely

attributable to biases in the CPS. Some observers have raised concerns about CCD counts of

high school completers; at issue is whether diplomas issued by community colleges or adult

education programs are included among the CCD counts of regular diplomas or alternative

credentials (Mishel and Roy 2006). It is not clear how substantial these biases might be.

         One obvious limitation of CCD data --- and indeed all state administrative data --- is that

it pertains exclusively to public school students. When high school dropout and completion rates

are used for the purposes of describing levels of human capital in a population or for describing

young people’s success at navigating the secondary education system, this limitation is

important. In the fall of 2003, 8.3 percent of secondary school students were enrolled in private

schools (Snyder, Dillow, and Hoffman 2008: Table 52).

         Beyond these cross-sectional, state-produced enrollment counts, states also frequently

make use of longitudinal administrative data to produce high school dropout and completion

rates.   Each state utilizes somewhat different data collecting, reporting, and aggregation

procedures, but in general there have been few concerns about states’ reports of the numbers of

students in each grade or of the numbers of students obtaining regular diplomas. The most

prominent controversies surrounding the use of longitudinal administrative data concern the use




                                                 22
(or misuse) of those data. While there is little concern about states’ abilities to accurately count

the number of students who begin high school as 9th graders (for example), there is frequent

concern about how states account for factors like migration, incarceration, expulsion, and

enrollment in alternative educational programs as they manipulate their 9th grade enrollment

counts to cause them to reflect the number of students at risk of obtaining high school diplomas

some time later.     For example, in 2004 and 2005 the Houston public schools were sharply

criticized for reclassifying nearly all dropouts as either enrollees in alternative educational

programs or out-migrants --- thus removing them from the denominators of dropout rate.

The Denominators: Differences in Estimates of the Size of the At-Risk Population

The denominators of high school dropout and completion rates represent the number if students

at risk of having dropped out or completed high school at a particular point in time. This section

describes differences between cross-sectional sample surveys, longitudinal surveys, and

administrative data with respect to their estimates of the size of the at risk population.

       According to NCES reports, the NELS cohort of 8th graders in spring of 1988 generalizes

to a population of approximately 2.46 million public school 8th graders during the 1987-88

academic year (Hafner et al. 1990; Kaufman, McMillen, and Bradby 1992). The 1987 CCD

State Nonfiscal Survey indicates that there were about 2.84 million public school 8th graders in

the fall of 1987; there were about 2.85 million public school 8th graders in that year according to

the 1987 October CPS. The relatively low numbers of NELS students in 1988 may be due to the

base year exclusions of sampled students with English language problems or physical or mental

disabilities. Without these exclusions, the NELS base year sample would have generalized to a

population of about 2.60 million public school 8th graders (Spencer et al. 1990). Given the vast




                                                 23
differences between these sources of data with respect to how this information was gathered, it

may be surprising that their level of agreement is so high.

       A similar story holds among 10th graders using data that are more recent. In fall 2002,

there were 3.65 million public school 10th graders according to the October CPS and 3.58 million

public school 10th graders according to the CCD. In the ELS cohort, there were roughly 3.15

million public school 10th graders --- again, this NCES survey excluded young people with

language problems English language problems or physical or mental disabilities.

       Although the information above suggests that the CPS and CCD data generally agree

about the number of enrolled public school students, there are also important discrepancies

between these sources of data. Figure 1 reports the ratio of the number of enrolled public school

8th to 12th graders in the October CPS to parallel figures in the CCD for the years 1969 through

2002. This figure, and the discussion that follows, is adapted from Warren and Halpern-Manners

(2007). For 8th and 9th graders, these ratios are around 1.00 in each year. This level of

agreement between the CPS and the CCD makes it difficult to conclude that the CPS severely

under-represents certain population groups or that the CCD contains systematic reporting biases.

       However, the situation is quite different among 10th to 12th graders. The ratio of the

number of enrolled public school 10th graders in the CPS to the same number in the CCD is 1.00

in 1969 and then grows modestly over time until it reaches a maximum of 1.09 in 1994. Among

11th graders, that ratio hangs around 1.03 in the early 1970s and then rises as high as 1.17 in

1997. In other words, in 1997 the CPS estimate of the number of public school 11th graders was

17 percent higher than in the CCD. The situation for 12th graders is more extreme. The ratio of

the number of enrolled public school 12th graders in the CPS to the equivalent number in the

CCD is at a minimum of 1.08 in the mid-1970s but rises to as high as 1.34 in 1995. Why are the




                                                24
estimated numbers of 10th, 11, and 12th graders higher in the CPS than in the CCD? Why has

this discrepancy grown over time?

           One logical interpretation of ratios greater than 1.00 in Figure 1 is that the CCD simply

undercounts the numbers of 10th, 11th, and 12th graders and that the CPS data are basically

accurate (although subject to random error). This seems implausible for two reasons. First, it is

not clear why the CCD would be increasingly likely to undercount 10th, 11th, and 12th graders

over time. The structure of the CCD data collection operation has remained essentially the same

over this period. Second, it is not at all clear why the CCD would undercount the number of

10th, 11th, and 12th graders while quite accurately reporting the numbers of 8th and 9th graders

(again, assuming that the CPS data are accurate). All of this information is collected on the same

form and from the same sources.

           A second interpretation of ratios greater than 1.00 for 10th, 11th, and 12th graders in Figure



                Figure 1. Ratio of the Number of Enrolled Public School Students in the October CPS
                                      to those in the CCD, by Grade, 1969-2002


  1.5
  1.4
  1.3
  1.2
  1.1
   1
  0.9
  0.8
  0.7
  0.6
  0.5
        1969

               1971

                      1973

                             1975

                                    1977

                                           1979

                                                     1981

                                                              1983

                                                                      1985

                                                                             1987

                                                                                    1989

                                                                                              1991

                                                                                                     1993

                                                                                                            1995

                                                                                                                   1997

                                                                                                                          1999

                                                                                                                                 2001




                             8th grade            9th grade          10th grade            11th grade       12th grade



                                                                     25
1 is that CPS respondents misstate the grade in which their children are enrolled.            CPS

respondents may think of their children’s “grade” as equivalent to the number of years in which

they have been enrolled in high school, such that first-year high school students tend to be

referred to as freshman or 9th graders, second-year students tend to be referred to as sophomores

or 10th graders, and so forth. These self-reports may not necessarily correspond to students’

actual administrative progress through high school with respect to Carnegie units accumulated,

courses successfully completed, or other administratively-important milestones that officially

determine students’ grade of enrollment. It may be that some young people who are labeled 10th

graders in the CPS—because they are attending their second year of high school—are classified

by school districts (and then states) in the CCD as 9th graders because they have not passed

required courses, accumulated sufficient course credits, or otherwise failed to achieve official

designation as 10th graders. It is easy to imagine that this would be even more common for 11th

and 12th graders. Given movements in recent years toward reducing social promotion and to

increasing accountability in public education, it is also not difficult to imagine why these

discrepancies would become more common over time.

       This interpretation assumes that the CCD data are accurate (subject to random error), and

that the CPS data are systematic biased by CPS respondents’ misstatements of their children’s

grade of enrollment. However, this explanation for the ratios greater than 1.00 in Figure 1 is also

not entirely satisfactory from an empirical point of view. If there are large numbers of students

who are claimed in the CPS to be 10th graders, for example, but who are administratively

classified as 9th graders in the CCD, then we should also observe relatively fewer 9th graders in

the CPS. As shown in Figure 1, this is not the case. The core problem is that the total number of

enrolled public school 9th through 12th graders in the CPS is consistently higher than the




                                                26
corresponding number of enrolled public school 9th though 12th graders in the CCD, ranging

from a low of 1.03 in the 1970s to a high of 1.09 in the late 1990s. This suggests that some CPS

respondents may simply be incorrectly reporting that their high school age children are enrolled

in 10th, 11th, or 12th grade when they are not officially enrolled at all.

        Thus a third explanation for ratios greater than 1.00 for 10th, 11th, and 12th graders in

Figure 1 is that some young people who are claimed by CPS respondents to be enrolled in 10th,

11th, and particularly 12th grade are in fact not officially enrolled in high school at all. This may

come about through CPS respondents’ unwillingness to report that their high school-aged

children have left high school (a social desirability issue), or it may arise because of honest

confusion about young people’s official enrollment status.

        This possibility seems plausible for four reasons. First, in many districts, students who

are truant for a certain number of days are administratively dropped from the enrollment lists,

regardless of whether the student considers himself or herself to have dropped out and regardless

of whether the student ultimately returns to school. These students are not officially enrolled in

school, but their parents may well report them as being enrolled in school in the CPS. Second,

students who drop out of high school often intend to return the following year. It may be that

many CPS respondents claim that their dropout children are enrolled because their children have

only temporarily stopped attending classes. Third, there is evidence that some CPS respondents

describe their children as enrolled in the latter high school grades when they are in fact enrolled

in GED programs (Rust 2004). Fourth, there is evidence from NELS that parents of high school

dropouts frequently report that their children are still enrolled in school, typically in the latter

grades of high school (Warren and Halpern-Manners 2007).




                                                   27
       In larger states, it is possible to compare enrollment counts from the CPS, the CCD, and

states’ own internal administrative data. For example, Figure 2 depicts the total number of

public school 9th through 12th graders as reported by the California Department of Education

(CDE) and as reflected in the CCD and October CPS data; parallel figures for other large states

are similar. The CDE and CCD are virtually identical in each year. This should come as no

surprise --- both are derived from states’ own accounting of enrollments, aggregated up from the

schoolhouse level. In contrast, the CPS data are much more variable, sometimes yielding higher

counts than the CDE or CCD and sometimes yielding lower counts. These discrepancies are due

in part to the sorts of biases described above (at the national level) and in part to sampling

variability. There are only about 50 California high school students represented in the CPS

sample each October. These CPS sample size issues are one reason that status dropout and

completion rates generally include a wide range of ages (i.e., 16 to 24 year olds) as opposed to

focusing more narrowly on specific cohorts of students.


                Figure 2. Number of High School Students in California Public Schools, 1993 to 2005


 2,500,000



 2,000,000



 1,500,000



 1,000,000



   500,000



        0
             1993   1994   1995     1996    1997    1998    1999     2000    2001    2002    2003     2004   2005

                                              CA Dept. of Ed.   CCD     Oct. CPS



                                                       28
The Denominators: Differences in the Estimated Number of First Time 9th Graders

Cohort measures are usually intended to characterize rates of dropout or completion among a

group of young people who began high school at the same time.                 Using longitudinal

administrative or sample survey data, this is straightforward. Longitudinal administrative data

can be limited to individuals beginning high school in a particular academic year; longitudinal

sample surveys begin with individuals in the same grade at a given point in time, and include

information about whether they are enrolled in that early grade for the first time. Unfortunately,

large-scale longitudinal sample surveys are infrequently conducted (and have their own

problems, as described above), and longitudinal administrative data are neither strictly

comparable across states nor consistently collected over more than a few years. Researchers

seeking to describe differences between states and/or over time in cohort dropout rates thus lack

data. In recent years, many researchers have turned to cross-sectional administrative data –

principally the CCD --- in an effort to construct synthetic cohort measures that approximate the

results that they would obtain if they had comparable cohort data for several years and/or large

numbers of states. For instance, Warren, Jenkins, and Kulick (2006) used CCD-based synthetic

cohort completion rates to model the effects of states’ high school exit examination policies on

completion rates in each state for the graduating classes of 1975 through 2002 . Chaplin, Turner,

and Pape (2003) used a conceptually-similar CCD-based measure to model the effects of states’

minimum wage rates on high school dropout in each state between 1990 and 1997.

       Cohort and synthetic cohort rates both utilize counts of completers or dropouts in the

numerator. However, whereas true cohort rates typically use actual counts of the number of

students at risk of dropping or completing in the denominator, synthetic cohort rates --- based

centrally on cross-sectional data on enrollments and completions --- must approximate the




                                               29
number of students at risk. CCD-based measures, for example, typically begin with the number

of 9th graders in a cohort in the denominator, and then adjust that number to account for various

factors that lead the number of enrolled 9th graders at the beginning of one academic year to

differ from the true number of students at risk of graduating or dropping out three academic

years later.

        Many observers have noted that synthetic cohort rates based on CCD or similar cross-

sectional administrative data may be seriously biased by migration, grade retention, mortality,

enrollment in ungraded programs, and other factors that lead to biases in the estimated number of

students at risk of completing or dropping out (e.g.,Pallas 1990). The most widely used CCD-

based synthetic cohort rates have all sought to adjust the denominator (which begins with the

total number of 9th graders enrolled in an academic year) to account for migration, grade

retention, and (indirectly) mortality. Given the high visibility of these measures --- which

include Swanson’s Cumulative Promotion Index (Editorial Projects in Education 2008; Swanson

and Chaplin 2003); the NCES Averaged Freshman Graduation Rate (Seastrom, Chapman et al.

2006; Seastrom, Hoffman et al. 2006); Greene’s Adjusted Completion Rate (Greene and Forster

2003; Greene and Winters 2006); and Warren’s Estimated Completion Rate (Warren 2005;

Warren and Halpern-Manners Forthcoming) --- it is worth asking about the accuracy of these

adjustments to the denominator.

        Define “bias” in a CCD-based synthetic cohort graduation rate as the difference between

the true cohort rate and the rate reported by a particular measure. To demonstrate the degree to

which particular measures are biased, as well as the specific factors that give rise to these biases,

this section includes a series of simulations in Tables 1 through 3. These simulations are based

on enrollment counts and counts of the number of regular diploma recipients in one geographic




                                                 30
area over five academic years. For demonstration purposes, the first two simulations (in Tables 1

and 2) stipulate that every single student obtains a regular high school diploma; in Table 3, the

simulation stipulates that 10 percent of 9th graders drop out of school during the academic year.

By design, then, an unbiased measure of the high school completion rate should equal 100

percent for every academic year in the simulations in Tables 1 and 2, and 90 percent in Table 3.

The simulations in Tables 1 through 3 differ only with respect to assumptions about net

migration rates and 9th grade retention rates. All four simulations begin with 100 students

entering the 8th grade for the first time in one academic year and follow that and subsequent

cohorts of students over successive academic years under a variety of assumptions about net

migration, grade retention, and dropout rates.

        The CCD data include counts of the number of public school students who are enrolled in

each grade in October of each academic year and counts of the number of regular diplomas that

are issued each spring. Using these two sets of figures, it is intuitively appealing to compute a

Basic Completion Rate (BCR) by comparing the number of enrolled public school 9th graders in

the fall of one academic year to the number of regular public high school diploma recipients

three academic years later, when that cohort of 9th graders should have graduated:

        High School CompletersSpring of Year X
BCR =                                                                                         (1)
                    E Grade 9Year X -3
                      Fall of



where E GradeX9 denotes the number of enrolled of students enrolled in the 9th grade in academic
        Year


year X. Indeed the BCR, so defined, has been used in work on the impact of state policies on

state-level public high school graduation rates (Fitzpatrick and Yoels 1992; Haney 2000).




                                                 31
            Table 1. Enrollments by Grade at the Beginning and End of Five Academic Years: No Dropout, No Grade Retention, +5% Net Migration Each Year

                         Academic Year #1             Academic Year #2                  Academic Year #3          Academic Year #4             Academic Year #5

                       Start           End          Start           End             Start           End          Start           End          Start           End
Enrollment             of the Change of the         of the Change of the            of the Change of the         of the Change of the         of the Change of the
                       Year           Year          Year           Year             Year           Year          Year           Year          Year           Year


8th Grade         0      100     5     105      0     100     5     105       0         100    5     105    0     100     5     105      0     100       5   105

                 10                           10                             10                            10                           10
                   5                            5                              5                             5                            5

9th Grade         0      105     5     110      0     105     5     110       0         105    5     110    0     105     5     110      0     105       5   110

                 110                          110                            110                           110                          110

10th Grade        0      110     6     116      0     110     6     116       0         110    6     116    0     110     6     116      0     110       6   116

                 116                          116                            116                           116                          116

11th Grade        0      116     6     122      0     116     6     122       0         116    6     122    0     116     6     122      0     116       6   122

                 122                          122                            122                           122                          122

12th Grade        0      122     6     128      0     122     6     128       0         122    6     128    0     122     6     128      0     122       6   128


                                       128                          128                              128                        128                          128


Regular Diploma Recipients             128                          128                              128                        128                          128


                          Basic Completion Rate (Eq. 1) = 122%                        Greene's Method (Eq. 4) = 100%             Warren's Method (Eq. 7) = 100%
              Adjusted Freshman Graduation Rate (Eq. 2) = 121%             Cumulative Promotion Index (Eq. 6) = 122%




             Table 2. Enrollments by Grade at the Beginning and End of Five Academic Years: No Dropout, No Net Migration, 10% 9th Grade Retention Rate

                         Academic Year #1             Academic Year #2                  Academic Year #3          Academic Year #4             Academic Year #5

                       Start           End          Start           End             Start           End          Start           End          Start           End
Enrollment             of the Change of the         of the Change of the            of the Change of the         of the Change of the         of the Change of the
                       Year           Year          Year           Year             Year           Year          Year           Year          Year           Year


8th Grade         0      100     0     100      0     100     0     100       0         100    0     100    0     100     0     100      0     100       0   100

                 10                           10                             10                            10                           10
                   0                            0                              0                             0                            0

9th Grade         11     111     0     111     11     111     0     111       11        111    0     111   11     111     0     111     11     111       0   111

                 100                          100                            100                           100                          100

10th Grade        0      100     0     100      0     100     0     100       0         100    0     100    0     100     0     100      0     100       0   100

                 100                          100                            100                           100                          100

11th Grade        0      100     0     100      0     100     0     100       0         100    0     100    0     100     0     100      0     100       0   100

                 100                          100                            100                           100                          100

12th Grade        0      100     0     100      0     100     0     100       0         100    0     100    0     100     0     100      0     100       0   100


                                       100                          100                              100                        100                          100


Regular Diploma Recipients             100                          100                              100                        100                          100


                          Basic Completion Rate (Eq. 1) = 90%                         Greene's Method (Eq. 4) = 96%              Warren's Method (Eq. 7) = 100%
              Adjusted Freshman Graduation Rate (Eq. 2) = 96%              Cumulative Promotion Index (Eq. 6) = 90%




                                                                                   32
       The first problem with the BCR has to do with migration. Students who appear as 9th

graders in a state in the fall of academic year X may move to another state before the spring of

academic year X+3; they may be replaced by (a smaller or larger number of) students who are

counted among the number of high school graduates in the spring of academic year X+3 but who

lived in another state in the fall of academic year X. As shown in Table 1, a positive net

migration rate biases the BCR upward—with a positive net migration rate of 5% in each grade,

the BCR is upwardly biased by 22 percentage points. Conversely, a negative net migration rate

downwardly biases this measure. A second problem has to do with grade retention. Given that

the denominator intends to represent the number of incoming public school 9th graders, measures

like the BCR are problematic to the extent that the denominator includes 9th graders who are

enrolled in the 9th grade in more than one year. Such measures count retained 9th graders in the

denominator for more than one year but in the numerator (a maximum of) one time. As shown in

Table 2, 9th grade retention downwardly biases the BCR. If 10% of 9th graders are retained each

year, the BCR is downwardly biased by 10 percentage points. In more recent work, Haney and

colleagues (2004) have tried to overcome the grade retention problem by using the number of

public school 8th graders enrolled in academic year X-4 as the denominator of Equation 1;

Heckman and LaFontaine (2007) also follow this strategy. Since far fewer students repeat the 8th

grade than repeat the 9th grade, this partially alleviates the grade retention bias; however, the

longer time horizon can exacerbate the migration bias.

       Numerous observers have recognized the consequences of migration and grade retention

for CCD-based synthetic cohort rates like the BCR. In recent years NCES has produced the

Averaged Freshman Graduation Rate (AFGR) “based on a technical review and analysis of a set

of alternative estimates” (Seastrom, Hoffman et al. 2006: 1). The AFGR is computed as




                                               33
                Regular Diploma Recipients Spring of Year X
AFGR =                                                                              ,                                                                        (2)
                              Smoothed E Grade 9Year X -3
                                         Fall of


where

                                           (E Grade 8Year X -4 + E Grade 9Year X-3 + E Grade 10 X-2 )
Smoothed E Grade 9Year X -3 =
           Fall of
                                              Fall of              Fall of             Fall of Year
                                                                                                      .                                                      (3)
                                                                          3

             The smoothing in the denominator is “intended to account for higher grade retentions in

the ninth grade” (Seastrom, Hoffman et al. 2006: 1). However, the AFGR does not accomplish

that goal. Like the BCR, the AFGR is downwardly biased when there is any 9th grade retention

(Table 2); unlike the BCR, the AFGR is also downwardly biased by 8th and 10th grade retention.

The “smoothing” of the denominator does nothing to adjust for net migration, and so the AFGR

is also biased by migration (Table 1). Furthermore, as shown in Table 3, the AFGR is biased

even in the absence of grade retention and migration. Assuming no net migration and no grade

retention, if 10% of 9th graders drop out (such that the true cohort graduation rate equals 90%),


        Table 3. Enrollments by Grade at the Beginning and End of Five Academic Years: No Net Migration, No Grade Retention, 10% of 9th Graders Drop Out

                        Academic Year #1             Academic Year #2                   Academic Year #3          Academic Year #4             Academic Year #5

                      Start           End           Start           End             Start           End          Start           End          Start           End
Enrollment            of the Change of the          of the Change of the            of the Change of the         of the Change of the         of the Change of the
                      Year           Year           Year           Year             Year           Year          Year           Year          Year           Year


8th Grade        0      100     0     100      0     100     0     100        0         100    0     100    0     100     0     100      0     100     0     100

                10                            10                             10                            10                           10
                  0                             0                              0                             0                            0

9th Grade        0      100    -10    90       0     100    -10     90        0         100   -10    90     0     100    -10     90      0     100    -10     90

                90                            90                             90                            90                           90

10th Grade       0      90      0     90       0     90      0      90        0         90     0     90     0     90      0      90      0     90      0      90

                90                            90                             90                            90                           90

11th Grade       0      90      0     90       0     90      0      90        0         90     0     90     0     90      0      90      0     90      0      90

                90                            90                             90                            90                           90

12th Grade       0      90      0     90       0     90      0      90        0         90     0     90     0     90      0      90      0     90      0      90


                                      90                            90                               90                          90                           90


Regular Diploma Recipients            90                            90                               90                          90                           90


                         Basic Completion Rate (Eq. 1) = 90%                          Greene's Method (Eq. 4) = 93%              Warren's Method (Eq. 7) = 90%
             Adjusted Freshman Graduation Rate (Eq. 2) = 93%               Cumulative Promotion Index (Eq. 6) = 90%




                                                                                   34
the AFGR equals 93%. Nonetheless, the AFGR is routinely reported by NCES, including in the

annual Digest of Education Statistics (Snyder, Dillow, and Hoffman 2008).

        Greene and Winters’ (2005) Adjusted Completion Rate (ACR) differs from the AFGR by

including a migration adjustment in the denominator:

            Regular Diploma Recipients Spring of Year X
ACR =                                                         ,                                  (4)
        Smoothed E Grade 9Year X -3 ∗ Migration Adjustment
                   Fall of


where smoothed 9th grade enrollment is the same as for the AFGR (Equation 3) and where

                           ⎡ (17 Year Olds Year X-1 − 14 Year Olds Year X-4 ) ⎤
Migration Adjustment = 1 + ⎢                                                  ⎥.                 (5)
                           ⎣           14 Year Olds Year X-4                  ⎦

        The simulation in Table 1 demonstrates that the migration adjustment in Equation 5

produces its desired effect: The ACR is unbiased by positive or negative net migration.

However, the ACR is biased by 9th grade retention (Table 2); it is also biased by 8th and 10th

grade retention. Like the AFGR, the ACR is biased even when there is no grade retention or net

migration: As shown in Table 3, if 10% of 9th graders drop out, the ACR equals 93%.

        Swanson (Swanson 2003: 14) proposed an innovative CCD-based synthetic cohort

graduation rate measure that “approximates the probability that a student entering the 9th grade

will complete high school on time with a regular diploma. It does this by representing high

school graduation rate [sic] as a stepwise process composed of three grade-to-grade promotion

transitions (9 to 10, 10 to 11, and 11 to 12) in addition to the ultimate high school graduation

event (grade 12 to diploma).” Specifically, the Cumulative Promotion Index (CPI) is:

      ⎛ Regular Diploma Recipients Year X    ⎞ ⎛ E GradeX +1 ⎞ ⎛ E GradeX +1 ⎞ ⎛ E GradeX +1 ⎞
                                                        12              11              10
CPI = ⎜
      ⎜                                      ⎟ × ⎜ Year 11 ⎟ × ⎜ Year 10 ⎟ × ⎜ Year 9 ⎟ ,
                                             ⎟ ⎜ E Grade ⎟ ⎜ E Grade ⎟ ⎜ E Grade ⎟               (6)
      ⎝
                           12
                    E GradeX
                      Year                   ⎠ ⎝ Year X ⎠ ⎝ Year X ⎠ ⎝ Year X ⎠

where, again, E Grade12 equals the number of 12th graders enrolled in the fall of academic year X.
                Year X


The author notes that this approach “estimates the likelihood of a 9th grader from a particular


                                                    35
district completing high school with a regular diploma in four years given the conditions in that

district during the [given] school year” (2003: 15; emphasis in original). Swanson (2003) argues

that this measure has the virtues of being timely and reflective of current education system

performance because it requires data from only two academic years. This measure has formed

the basis of several widely publicized reports, including the Diploma Count series by Editorial

Projects in Education (2008).

        As shown in the simulation in Table 1, however, the CPI is systematically biased by net

migration. Given that the CPI is designed to measure on-time graduation rates, the fact that the

CPI equals 90% when 10% of 9th graders are retained is not evidence of bias (Table 2).

However, the CPI is not affected by 10th, 11th, or 12th grade retention (simulations not shown).

That is, when there is no dropout, no net migration, and no 9th grade retention, the CPI equals

100% regardless of how many 10th, 11th, or 12th grade students are retained.

        Warren’s (2005) Estimated Completion Rate (ECR) conceptually represents the ratio of

the number of regular public school diplomas that are issued in a particular year to the estimated

number of first-time 9th graders three academic years earlier (after accounting for migration):

                             Regular Diploma Recipients Spring of Year X
ECR =                                                                                          ,   (7)
        Estimated Number of First Time 9 th Graders Fall of Year X -3 × Migration Adjustment


where the estimate of the number of first-time public school 9th graders in a particular academic

year is simply the number of public school 8th graders in the preceding academic year. As shown

by Warren (2005), 8th grade retention rates are very low even in states with relatively high 9th

grade retention rates. As a result, the number of 8th graders in one year closely approximates the

number of first-time 9th graders the following year (even before adjusting for migration). For

example, Table 4 shows the number of first-time 9th graders in North Carolina public schools in



                                                  36
            Table 4. Estimates of the Number of First Time 9th Graders in North Carolina
                                  Public Schools, 2000-01 to 2004-05

  120,000
  115,000
  110,000
  105,000
  100,000
   95,000
   90,000
   85,000
                 2000-01           2001-02            2002-03           2003-04          2004-05

                  New 9th Graders in Academic Year X         8th Graders in Academic Year X-1
                  "Smoothed" Estimate of New 9th Graders

each fall between 2000 and 2004 (North Carolina Department of Public Instruction 2006), the

number of 8th graders in the fall of the preceding academic year, and the number of students

derived from the “smoothed” estimate (Equation 3) from the denominator of the ACR and

AFGR. In each year, the number of 8th graders in the preceding year closely approximates the

number of first-time 9th graders; the “smoothed” estimate is consistently higher.

       The ECR’s migration adjustment is:

                           ⎡ (17 Year Olds Year X-1 − 13 Year Olds Year X -5 )⎤
Migration Adjustment = 1 + ⎢                                                  ⎥.                   (8)
                           ⎣           13 Year Olds Year X -5                 ⎦

This adjustment is the same as Equation 5, except that it is extended to 13 year olds in order to

account for migration between grades 8 and 9. The numbers of 17 and 13 year olds for Equation

8 are derived from annual state-level population estimates by single year of age produced by the

U.S. Bureau of the Census.         As the simulations in Tables 1 through 3 indicate, the ECR

overcomes the two most serious forms of systematic error in CCD-based synthetic cohort



                                                   37
measures by accounting for 9th grade retention and interstate migration in an empirically sound

manner. Unlike the AFGR or the CPI, however, the ECR cannot be calculated at the sub-state

level. Furthermore, Warren (2005) notes that the ECR is biased by 8th grade retention (the rates

of which are generally very low even in states with high 9th grade retention rates); by high rates

of movement from private to public schools (or vice versa) between the 8th and 9th grades; and by

the international in-migration of young people who never enroll in US schools.

       To summarize, efforts to adjust the denominators of CCD-based synthetic cohort rates to

better approximate true cohort rates have largely been unsuccessful. Because of problems in

their denominators, widely-publicized CCD-based synthetic cohort rates like the AFGR, the CPI,

the ACR, and the ECR are biased. These are systematic errors, not random errors. But what are

the magnitude of these errors?          Figure 3 depicts CCD-based synthetic cohort completion rates

for the US, using each measure. Despite the discussion above, these rates generally fall within a

few percentage points of one another. Even in states or smaller geographic areas with fairly high


                 Figure 3. CCD-Based Synthetic Cohort Graduation Rates for the US



   100%
    90%
    80%
    70%
    60%
    50%
    40%
    30%
    20%
    10%
     0%
          1975

                   1977

                          1979

                                 1981

                                        1983

                                               1985

                                                      1987

                                                             1989

                                                                    1991

                                                                           1993

                                                                                  1995

                                                                                          1997

                                                                                                 1999

                                                                                                        2001




                                    BCR        ACR           AFGR          CPI           ECR


                                                        38
grade retention or net migration rates, the full range of these estimates is about 10 percentage

points. One lesson from all of this is that we should not make too much of year-to-year or state-

to-state differences of a few percentage points.

The Numerators: Estimating the Numbers of Completers and Dropouts

The preceding sections described issues in quantifying the number of students “at risk” of

graduating from or dropping out of high school in the denominators of status, event, cohort, and

synthetic cohort dropout and completion rates. This section describes differences between

estimates based on cross-sectional sample surveys, longitudinal surveys, and administrative data

with respect to their numerators, or their counts of the number of people who complete or drop

out of high school.

       Cross-sectional sample surveys --- most notably the CPS, the decennial Census, and the

ACS --- allow respondents to classify themselves as either high school completers or non-

completers.    The status “high school dropout” is an inferred residual category, assigned to

respondents who have not obtained a secondary school credential and who are not enrolled in

school. In recent years these sample surveys have included additional questions that seek to

differentiate high school graduates from those who obtain GEDs. A number of observers have

raised questions about the validity of these self-reports of how respondents complete high school

(Chaplin 2002; Warren and Halpern-Manners 2007). What is more, no effort is made in these

cross-sectional sample surveys to differentiate individuals who obtained regular high school

diplomas from those who earned certificates of completion or who completed high school by

completing community college or adult education programs. In the end, cross-sectional sample

surveys do a relatively poor job of representing the number of regular diploma recipients.




                                                   39
Finally, these sample surveys include no information about when respondents completed high

school, and so it is not possible to measure on-time high school completion.

       Longitudinal sample surveys --- like NELS, ELS, and their predecessors --- typically

supplement self-reports of high school completion and dropout with transcript data on those

events. It is thus possible to construct status, event, and cohort dropout or completion rates. In

combination with information about the timing of dropout and completion events, it is also

possible to differentiate on-time completion from later completion. Although the transcript data

frequently available in these longitudinal sample surveys includes detail about when respondents

completed high school and what sort of public school credential they obtained, there is less (and

less well-validated) information about credentials obtained from GED, community college, and

adult education programs. An exception is the ELS data, which includes detailed administrative

data from the GED Testing Service.

       States’ longitudinal administrative data include information about enrollment in and exits

from public schools. Most students exit the school system by obtaining a regular diploma or

alternative credential, and states’ longitudinal data systems generally count these things in a

consistent manner. Where they differ, though, is in their accounting of other exits from the

system. Districts and states vary with respect to how and how well they track and account for

students who transfer to schools in other districts or states; who transfer to private schools,

correctional facilities, or other institutional settings; or who are expelled, who die, or who exit

the system in other ways.       This means that states’ longitudinal data generally preclude

enumerating students who obtain credentials from GED programs or private schools. What is

more, exiting the system by dropping out is rarely directly observed. Students are presumed to

have dropped out if they do not exit the system by completing high school, transferring, or




                                                40
through some other quantifiable transition.   It is difficult to validate such indirect techniques for

counting dropouts.

       Cross-sectional state administrative data --- for example, as compiled in the CCD ---

generally include only annual counts of the numbers of regular high school diplomas, GEDs, and

other credentials issued by schools, districts, and states. As such, it is impossible to distinguish

on-time completers from other completers. Some observers have raised concerns about whether

alternative credentials are consistently distinguished from regular diplomas in the CCD (Warren

2005), and GED data are frequently missing in the CCD. As with longitudinal administrative

data, dropout is a residual outcome, and is not directly observed in cross-sectional state

administrative data. Because the CCD and similar data are usually compiled from data produced

by public school agencies, very little information is available about private high school

completions.    Consequently, analysts sometimes supplement CCD data on public school

completions with similar data on private schools from the bi-annual NCES Private School

Survey (Tourkin et al. 2008).

       To summarize, the type of data used to construct dropout or completion rates determines

what can be counted in the numerator. Status dropout and completion rates are typically based

on cross-sectional sample survey data, and thus they do less well at differentiating regular

diploma recipients from other high school completers and they do not distinguish on-time

completers from later completers. Event dropout and completion rates can be constructed from

cross-sectional survey data, longitudinal survey data, or longitudinal administrative data.

Depending on which type of data is used, event rates may or may not distinguish between types

of completions; may or may not include private school completers; may or may not directly

enumerate dropouts; and may or may not differentiate on-time completers from other completers.




                                                41
Cohort dropout and completion rates can be constructed from longitudinal sample survey data or

longitudinal administrative data, and this choice determines analysts’ ability to observe dropout

events and to include private school completers.

DISCUSSION

There are three important reasons to measure high school dropout and completion rates: To

assess the level of human capital in a population, to characterize schools’ “holding power,” and

to describe students’ success at navigating the secondary school system. For each purpose, what

is the best available technique for measuring rates of high school dropout and completion? What

are the weaknesses in those “best techniques,” and how might they be improved?

Describing Levels of Human Capital

Status dropout or completion rates computed from cross-sectional sample surveys are best suited

to describing levels of human capital in a population. Because the goal is to describe the share of

all individuals who have obtained a high school credential, it is important to utilize data that

include people who may have obtained those credentials from public schools, private schools,

GED programs, community colleges, adult education programs, or elsewhere.             This is not

possible with state administrative data, because it only pertains to people who went to school in

that state (as opposed to the target population, which includes everyone in an age range who

lives in that state) and because those data generally do not include those who complete high

school outside of the public school system.

       Status completion and dropout rates have traditionally been based on CPS or decennial

Census data. The CPS has known flaws for this purpose --- it excludes the non-civilian and

institutionalized populations and includes too few respondents to generalize reliably to the state

or substance levels --- and the long-form of the Census has been phased out. Since the ACS




                                                42
sample is about 16 times larger than the CPS, and that it generalizes to the entire population of

US households, status completion rates should more frequently be constructed using these data.

       For the purposes of describing levels of human capital, status completion or dropout rates

are imperfect in a couple of respects. One key weakness is their treatment of all high school

credentials as equivalent. Economists and others have long questioned the labor market value of

the GED (e.g., Cameron and Heckman 1993); little is known about the labor market value of

other alternative credentials.   The 2008 ACS was the first to distinguish regular diploma

recipients from GED recipients. It would be useful for the ACS to add a third category in order

to distinguish regular diploma recipients from those who obtain certificates of completion or who

complete high school via community college or adult education programs. As described below,

there is also potential payoff to adding just a few questions about the year and state/country in

which individuals completed high school. A second key weakness of status completion rates for

describing levels of human capital in a population concerns the validity and reliability of

respondents’ reports of whether (and how) they completed high school. This issue could be

better understood, and perhaps put to rest, with a validation study in which high school and GED

records are matched to a random subset of ACS records.             This would require obtaining

information from the selected respondents about where, when, and how they obtained their high

school credentials.

Describing Schools’ Holding Power

       For purposes of describing schools’ holding power, dropout and completion rates must

directly and reliably pertain to specific locales and to specific cohorts of students, and they must

be available annually. This implies the use of cohort rates based on longitudinal administrative

data. As described above, the biggest drawback to using cohort rates based on these data is that




                                                43
different states and locales have historically defined the numerators and denominators

differently. In the numerator, most states have only counted regular diploma recipients as

“successes,” both some have been less restrictive in their definition of “success.”           In the

denominator, states have typically begin with counts of first time 9th graders, but they have

differed in how they adjust that figure to account for migration and other exits and entries in

order to have the denominator reflect the number of students at risk of graduating or dropping

out. As a result, the public has sometimes been distrustful of these rates, trend analyses have

been hampered, and cross-state comparisons have been next to impossible. In this respect, the

movement toward universal adoption of the standards laid out in the National Governors

Association Compact (2008) is a great step forward.           If states faithfully and consistently

implement the standards laid out in this agreement --- which call for restricting the numerator of

cohort rates to regular diploma recipients and which spell out standards for deriving the

denominator --- then we will eventually reach a point at which these cohort rates can be

compared over time and across states.

       In the meantime, however, many situations call for the use of synthetic cohort completion

rates based on cross-sectional administrative data like the CCD. This includes any situation in

which the goal is to describe trends over time and/or differences across states in rates of high

school dropout or completion. Policy makers or the members of the public may wish to consider

changes over several years in their state or district’s dropout rate. Or they may wish to compare

dropout rates in their state or district to those in neighboring states or districts. Researchers may

want to model state- or district-level dropout or completion rates in order to understand the

factors that influence schools’ holding power or to understand the consequences of schools’

holding power for any number of social, economic, or other outcomes. All of these situations




                                                 44
require measures of dropout or completion that are consistent over time and/or place. Until

consistently define cohort rates based on longitudinal administrative data have been around for a

number of years, synthetic cohort rates based on CCD or similar data will have to suffice.

Unfortunately, data limitations preclude the construction of synthetic cohort dropout rates.

       Since there will likely be important roles for synthetic cohort measures based on CCD or

similar data for foreseeable future, it is important to recognize weaknesses in those measures. As

described at length above, the most widely publicized variants of these measures are

demonstrably biased. The ECR is currently the least biased of these measures, and should be

used whenever synthetic cohort completion rates are computed at the national or state levels.

There is currently no trustworthy CCD-based synthetic cohort completion measure than can be

constructed for use at the level of cities or districts. The reason that the ECR has not been

applied to the sub-state level is because its migration adjustment cannot be reliably computed.

This is also the reason that the measure has not been applied to sociodemographic subgroups of

students (but see Warren and Halpern-Manners Forthcoming). It is conceivable that one could

adjust for migration using different data or different algebra than Warren (2005), and that this

would allow for these applications of the ECR.

Describing Students’ Progress through the Secondary School System

       As described above, measures of students’ progress through the system of secondary

education are conceptually and technically distinct from measures of schools’ holding power.

Measures that describe students should be based on data that include private school students and

that give analysts control over how to define “success” and “failure” in the numerator. They

should also allow for an understanding of how dropout and completion vary by students’ social,

demographic, and other characteristics. For these reasons, longitudinal administrative data from




                                                 45
schools are not appropriate. Neither are cross-sectional administrative or sample survey data

(because they do not allow analysts to follow students over the course of high school).

        For these reasons, cohort rates base don longitudinal sample survey data (like NELS or

ELS) are best suited for describing students’ progress through the secondary school system.

From a practical standpoint, however, the reliance on such surveys is limiting.           They are

relatively expensive and infrequently conducted. They suffer from problems of coverage bias

and sample attrition (as described above). They cannot be used to describe the experiences of

students in particular states, cities, or districts. So while it is certainly worth investing in

longitudinal sample surveys (for these and many other purposes), it is worth considering some

alternatives.

        For example, it would be relatively easy to construct a cohort measure of students’

success at completing their secondary schooling by adding just one item to the ACS: “In what

year did you first enroll in the 9th grade?” In the absence of any other information, this would

allow for the construction of an event completion rate (which is essentially the same as a cohort

rate when the time lapse is the standard length of high school). This cohort measure could be

made to apply to particular states --- and with much greater statistical power --- by adding

another item that asks about where individuals first enrolled in the 9th grade, and it could become

an on-time cohort completion rate by adding a question about the year in which individuals

obtained their high school credential. To be confident of the validity of these rates, however, it

would first be wise to validate reports of whether, when, and how individuals completed high

school against high school and GED Testing Service records (as described above).




                                                46
Conclusion

       A large share of the confusion about how often young people drop out of or complete

high school stems from mismatches between the purpose of quantifying rates of school dropout

and completion and the specific measures and data used to do it. There are many reasons to

describe schools’ holding power --- especially in this age of school accountability --- but CPS-

based status dropout rates and cohort dropout rates derived from longitudinal NCES surveys are

very dull instruments for achieving this end. There are probably as many good reasons to

describe students’ success at progressing through America’s system of secondary schools, but

this end is not achievable with CCD-based synthetic cohort completion rates or even cohort rates

based on states’ longitudinal data systems. Likewise, there are good reasons to describe the

human capital in a population, but it is important to recognize that none of the measures

developed in the last several decades take us much further toward that goal that the status

dropout rates that were in use in the 1970s. There are ways to better achieve all three of these

goals, but the biggest step we can take toward clarifying our understanding of the high school

dropout/completion situation in the United States would is to be consistently clear about why we

are measuring high school dropout/completion, what measures best address this motivation, and

what data are best suited to that purpose.




                                              47
REFERENCES

Annie E. Casey Foundation. 2008. Kids Count 2008 Data Book. Baltimore, MD: Annie E. Casey

       Foundation.

Cameron, Stephen V., and James J. Heckman. 1993. The Nonequivalence of High School

       Equivalents. Journal of Labor Economics 11 (1):1-47.

Chaplin, Duncan. 2002. Tassels on the Cheap. Education Next 2:24-29.

Chaplin, Duncan D., Mark D. Turner, and Andreas D. Pape. 2003. Minimum Wages and School

       Enrollment of Teenagers: A Look at the 1990’s. Economics of Education Review 22:11-

       21.

Curtin, Thomas R., Steven J. Ingels, Shiying Wu, and Ruth Heuer. 2002. User's Manual:

       NELS:88 Base-Year to Fourth Follow-up: Student Component Data File. NCES

       2002323. Washington, DC: National Center for Education Statistics, US Department of

       Education.

Editorial Projects in Education. 2008. Diplomas Count 2008: School to College: Can State P–16

       Councils Ease the Transition? Vol. 27 (special issue), no. 40, June 5, 2008, Education

       Week.

Fitzpatrick, Kevin, and William Yoels. 1992. Policy, School Structure, and Sociodemographic

       Effects on Statewide High School Dropout Rates. Sociology of Education 65:76-93.

Greene, Jay P., and Greg Forster. 2003. Public High School Graduation and College Readiness

       Rates in the United States. New York: Center for Civic Innovation, Manhattan Institute.

Greene, Jay P., and Marcus A. Winters. 2002. High School Graduation Rates in the United

       States. New York: Center for Civic Innovation, Manhattan Institute.




                                              48
———. 2005. Public High School Graduation and College-Readiness Rates: 1991-2002. New

      York: Center for Civic Innovation, Manhattan Institute.

———. 2006. Leaving Boys Behind: Public High School Graduation Rates. New York: Center

      for Civic Innovation, Manhattan Institute.

Greene, Jay P., Marcus A. Winters, and Christopher B. Swanson. 2006. Missing the Mark on

      Graduation Rates: A Response to 'The Exaggerated Dropout Crisis'. Education Week,

      March 29, 39, 42.

Hafner, Anne, Steven Ingels, Barbara Schneider, and David Stevenson. 1990. A Profile of the

      American Eighth Grader: NELS:88 Student Descriptive Summary. NCES 90458.

      Washington, DC: Natioinal Center for Education Statistics, US Department of Education.

Haney, Walt. 2000. The Myth of the Texas Miracle in Education. Education Policy Analysis

      Archives 8:11 (http://epaa.asu.edu/epaa/v8n41/, retrieved 7/24/2002).

Haney, Walt, George Madaus, Lisa Abrams, Anne Wheelock, Jing Miao, and Ileana Gruia. 2004.

      The Education Pipeline in the United States, 1970–2000. Chestnut Hill, MA: National

      Board on Educational Testing and Public Policy, Boston College.

Hartzell, Gary, Jack McKay, and Jack Frymier. 1992. Calculating Dropout Rates Locally and

      Nationally with the Holding Power Index. ERIC Document Reproduction Service No.

      ED 343 953.

Hauser, Robert M. 1997. Indicators of High School Completion and Dropout. In Indicators of

      Children's Well-Being, edited by R. M. Hauser, B. V. Brown and W. R. Prosser. New

      York: Russell Sage Foundation.

Heckman, James J., and Paul A. LaFontaine. 2007. The American High School Graduation Rate:

      Trends and Levels: Retreived from http://ftp.iza.org/dp3216.pdf on April 21, 2008.




                                              49
       Discussion Paper No. 3216, December 2007. Bonn, Germany: Institute for the Study of

       Labor.

Ingels, Steven J., Kathryn L. Dowd, John R. Taylor, Virginia H. Bartot, Martin R. Frankel, Paul

       A. Pulliam, and Peggy Quinn. 1995. User's Manual: NELS:88 Second Follow-Up:

       Transcript Component Data File. NCES 95377. Washington, DC: National Center for

       Education Statistics, US Department of Education.

Kaufman, Phillip. 2001. The National Dropout Data Collection System: Assessing Consistency.

       Paper presented at the Harvard Civil Rights Project, January 13, 2001. Available on the

       web at: http://www.law.harvard.edu/civilrights/publications/dropout/kaufman.html.

———. 2004. The National Dropout Data Collection System: History and Search for

       Consistency. In Dropouts in America: Confronting the Graduation Rate Crisis, edited by

       G. Orfield. Cambridge, MA: Harvard Education Press.

Kaufman, Phillip, Marylin M. McMillen, and Denise Bradby. 1992. Dropout Rates in the United

       States: 1991. NCES 92129. Washington, DC: Natioinal Center for Education Statistics,

       US Department of Education.

Komiski, Robert. 1990. Estimating the National High School Dropout Rate. Demography

       27:303-311.

Laird, Jennifer , Emily Forrest Cataldi, Angelina KewalRamani, and Chris Chapman. 2008.

       Dropout and Completion Rates in the United States: 2006. NCES 2008053. Washington,

       DC: US Department of Education, National Center for Education Statistics.

McMillion, Robin, Jason Ramirez, and Linda Roska. 2008. Secondary School Completion and

       Dropouts      in     Texas      Public        Schools,   2006-07.      Accessed       at

       http://www.tea.state.tx.us/research/pdfs/dropcomp_2006-07.pdf on 9/23/2008. Austin,




                                                50
        TX: Department of Assessment, Accountability, and Data Quality, Division of

        Accountability Research, Texas Education Administration.

Mishel, Lawrence. 2006. The Exaggerated Dropout Crisis. Education Week, March 8, 40.

Mishel, Lawrence, and Joydeep Roy. 2006. Rethinking High School Graduation Rates and

        Trends. Washington, D.C.: Economic Policy Institute.

National Governors Association. 2008. Implementing Graduation Counts: State Progress to

        Date, 2008. Washington, D.C.: National Governors Association Center for Best

        Practices.

North Carolina Department of Public Instruction. 2006. North Carolina Public Schools 2005

        Statistical Profile

Retrieved from http://www.ncpublicschools.org/docs/fbs/resources/data/statisticalprofile/2005profile.pdf on

        10/8/08. Raleigh:: North Carolina Department of Public Instruction.

North Carolina State Board of Education. 2004. North Carolina Public Schools: Statistical

        Profile 2004. Raleigh, NC: North Carolina State Board of Education.

Pallas, Aaron M. 1990. Conceptual and Measurement Issues in the Study of School Dropouts. In

        Research in Sociology of Education and Socialization, edited by R. G. Corwin and K.

        Namboodiri. Stamford, CT: JAI Press.

Ramirez, Jason, Robin McMillion, and Linda Roska. 2006. Secondary School Completion and

        Dropouts        in      Texas       Public        Schools,       2004-05.       Accessed         at

        http://www.tea.state.tx.us/research/pdfs/dropcomp_2004-05.pdf on 9/18/2008. Austin,

        TX: Department of Assessment, Accountability, and Data Quality, Division of

        Accountability Research, Texas Education Administration.




                                                     51
Rust, Keith. 2004. Sampling and Field Operations at Westat, 1983-2001. In The Nation's Report

       Card: Evolution and Perspectives, edited by L. V. Jones and I. Olkin. Bloomington: Phi

       Delta Kappa Education Foundation.

Seastrom, Marilyn, Chris Chapman, Robert Stillwell, Daniel McGrath, Pia Peltola, Rachel

       Dinkes, and Zeyu Xu. 2006. A Review and Analysis of Alternative High School

       Graduation Rates, Volume 1. Users Guide to Computing High School Graduation Rates,

       Volume 2, An Analysis of Alternative High School Graduation rates. NCES 2006–604

       and 605. Washington, D.C.: National Center for Education Statistics, U.S. Department of

       Education.

Seastrom, Marilyn, Lee Hoffman, Chris Chapman, and Robert Stillwell. 2006. The Averaged

       Freshman Graduation Rate for Public High Schools From the Common Core of Data:

       School Years 2002–03 and 2003-04. NCES 2006-606rev. Washington, D.C.: National

       Center for Education Statistics, U.S. Department of Education.

Snyder, Thomas D., Sally A. Dillow, and Charlene M. Hoffman. 2008. Digest of Education

       Statistics, 2007. NCES #2008-022. Washington, D.C.: US Government Printing Office.

Spencer, Bruce D., Martin R. Frankel, Steven J. Ingels, Kenneth A. Rasinski, and Roger

       Tourangeau. 1990. NELS:88 Base Year Sample Design Report. NCES 90463.

       Washington, DC: National Center for Education Statistics, US Department of Education.

Swanson, Christopher B. 2003. Keeping Count and Losing Count: Calculating Graduation Rates

       for All Students under NCLB Accountability. Washington, D.C.: The Urban Institute,

       Education Policy Center.

Swanson, Christopher B., and Duncan Chaplin. 2003. Counting High School Graduates When

       Graduates Count. Washington, D.C.: Education Policy Center, Urban Institute.




                                              52
Tourkin, Steven, Nancy Swaim, Xiaodong Guan, Jennifer Peterson, Dennis Schwanz, Toni

      Warner, Randall Parmer, Meagan Wilson, Stephen Broughman, and Elizabeth Walter.

      2008. 2003-2004 Private School Universe Survey (PSS) Data File User's Manual and

      Survey Documentation. NCES 2008314. Washington, DC: National Center for Education

      Statistics, US Department of Education.

Warren, John Robert. 2005. State-Level High School Completion Rates: Concepts, Measures,

      and Trends. Education Policy Analysis Archives (Retrieved March 13, 2006 from

       http://epaa.asu.edu/epaa/v13n51/). 13 (51).

Warren, John Robert, and Andrew Halpern-Manners. 2007. Is the Glass Emptying or Filling Up?

      Reconciling Divergent Trends in High School Completion and Dropout. Educational

      Researcher 36:335-343.

———. Forthcoming. Measuring High School Graduation Rates at the State Level: What

      Difference Does Methodology Make? Sociological Methods & Research.

Warren, John Robert, Krista N. Jenkins, and Rachael B. Kulick. 2006. High School Exit

       Examinations and State-Level Completion and GED Rates, 1975-2002. Educational

      Evaluation and Policy Analysis 28:131-152.

Winglee, Marianne, David Marker, Allison Henderson, Beth Aronstamm Young, and Lee

       Hoffman. 2000. A Recommended Approach to Providing High School Dropout and

      Completion Rates at the State Level. NCES2000305. Washington, DC: National Center

      for Education Statistics, US Department of Education.

Young, Beth, and Lee Hoffman. 2002. Public High School Dropouts and Completers From the

      Common Core of Data: School Years 1991-92 through 1997-98. NCES 2002382.

      Washington, DC: National Center for Education Statistics, US Department of Education.




                                               53

								
To top