Class-Size-Reduction-in-Wisconsin-A-Fresh-Look-at-the-Data

Document Sample
Class-Size-Reduction-in-Wisconsin-A-Fresh-Look-at-the-Data Powered By Docstoc
					Class Size Reduction in Wisconsin: A Fresh Look at the Data
By Phil Smith, University of Wisconsin-Milwaukee Alex Molnar, Arizona State University John Zahorik, University of Wisconsin-Milwaukee

Education Policy Research Unit (EPRU) Education Policy Studies Laboratory College of Education Division of Educational Leadership and Policy Studies Box 872411 Arizona State University Tempe, AZ 85287-2411

September 2003

EPSL | EDUCATION POLICY STUDIES LABORATORY Education Policy Research Unit
EPSL-0309-110-EPRU
http://edpolicylab.org

Education Policy Studies Laboratory Division of Educational Leadership and Policy Studies College of Education, Arizona State University P.O. Box 872411, Tempe, AZ 85287-2411 Telephone: (480) 965-1886 Fax: (480) 965-0303 E-mail: epsl@asu.edu http://edpolicylab.org

Class Size Reduction in Wisconsin: A Fresh Look at the Data
Philip Smith, University of Wisconsin at Milwaukee Alex Molnar, Arizona State University John A. Zahorik, University of Wisconsin at Milwaukee

Introduction
The Student Achievement Guarantee in Education (SAGE) evaluation was conducted between 1996-97 – 2000-01 by the School of Education at the University of Wisconsin–Milwaukee under contract to the Wisconsin Department of Public Instruction. The purpose of the SAGE evaluation was to determine the effectiveness of the Student Achievement Guarantee in Education (SAGE) program in promoting academic achievement of students in kindergarten through third-grade classrooms in schools serving low-income children. The 1995 SAGE statute [Wisconsin s. 118.43] required participating schools to (1) reduce class size to 15 in kindergarten and grade one in 1996–97, grades kindergarten through two in 1997–98, and grades kindergarten through three in 1998–99 to 2000–01; (2) stay open from early in the morning to late in the day and collaborate with community organizations to provide educational, recreational, community, and social services (i.e., the "lighted schoolhouse"); (3) provide a rigorous academic curriculum to improve academic achievement; and (4) establish staff development and accountability mechanisms. The SAGE evaluation involved the 44-47 schools in the 21 school districts that initially participated in the program. During the 1996-97 school year SAGE was implemented in kindergarten and first grade. Second grade was added in 1997-98, and

third grade in 1998-99. The SAGE evaluation compared 30 SAGE schools to a group of 14-17 non-SAGE Comparison schools located in SAGE districts.

Overview of SAGE Evaluation Findings
Achievement Outcome Findings 1996-2000 To measure academic achievement, first-grade students in SAGE schools and in a group of Comparison schools were tested in October of their first grade placement, and again the following May, using the Comprehensive Test of Basic Skills (CTBS) Complete Battery, Terra Nova edition, Levels 10 and 11 respectively. After one academic (October to May) year, students in SAGE first-grade classrooms gained more on the CTBS tests than first-grade students in Comparison schools. As a group, SAGE students scored significantly higher on the post-test in reading, language arts, and mathematics sub-tests of the CTBS. The total score of SAGE students was also significantly higher than the total score of comparison group students. The achievement advantage associated with participation in the SAGE program was revealed both in the analysis of individual student scores and in the analysis of averaged classroom scores. At the individual level of analysis, after controlling for pre-test (October) scores, socioeconomic status (SES) as defined by eligibility for subsidized lunch, absenteeism, and race and ethnicity, SAGE first-grade students scored higher than Comparison school first-grade students on the CTBS post-test in reading, language arts, mathematics and total score. The adjusted differences were in the 4-6 scale score points range. Most of these differences were found to be statistically significant. Both SES and absenteeism were significant factors in each of these analyses as well.

2 of 34

In general, the classroom level data on the averaged performance of first-grade students in SAGE classrooms suggested that the lower student-teacher ratio in SAGE classrooms mitigated the negative achievement consequences of poverty. SAGE classrooms achieved at a higher level than Comparison school classrooms despite the fact that, as a group, SAGE classrooms enrolled more students who were eligible for subsidized lunch. Furthermore, after adjusting for individual pre-test results and SES as defined by lunch status and student attendance, the post-test scale score advantage of SAGE was in the 6-12 scale score point range across subtests. These results were all statistically significant. Students in the SAGE and Comparison groups in all cohorts were followed through second and third grade where they were tested again during the spring semester in each of these years. Test results suggest that the statistically significant positive effects of SAGE, which occurred in first grade, were maintained (i.e., further increases or decreases were generally not observed) in second and third grade. When using the first-grade post-test as the baseline, African American SAGE students made larger gains than students in Comparison schools on every test except in reading, but the gains were not statistically significant. While African American students, as a group, scored significantly lower than White students in both SAGE and Comparison schools, the gap between African American and White students was larger in Comparison schools. Gains made by African American versus White students were significantly better in SAGE schools from the beginning of first grade to the end of third grade. The opposite pattern was observed in Comparison schools.

3 of 34

Overall, analysis of test results at the class level suggest that students in smaller classrooms tend to score significantly higher in language arts, mathematics, and reading as well as total score after adjusting for individual pre-test results, socioeconomic status, and attendance. In other words, classrooms with fewer students are more likely to have higher class average achievement scores and are more likely to contribute to closing the achievement gap between African American and White students than classrooms with a higher number of students. Students In 2000-2001, the SAGE evaluation involved a total of 2,474 students in third grade. The characteristics of students in SAGE and Comparison schools are displayed in Table 1.

Table 1: Characteristics of SAGE and Comparison Students 1996-97, 1997-98, 199899, 1999-00, and 2000-01 Characteristic Percent of Students* Percent of Students* SAGE Comparison 96-97 97-98 98-99 99-00 00-01 96-97 Gender Female Male Race/Ethnicity African American Asian Hispanic Native American White Other Subsidized Lunch Eligibility Free Reduced Not Eligible Repeating Grade 9798 48.5 51.5 24.7 5.6 10.0 1.5 52.2 2.3 98-99 99-00 00-01

48.6 51.4 24.8 5.7 6.6 11.7 48.8 1.6

48.4 51.6 26.3 5.2 6.5 10.3 43.8 2.0

48.6 51.4 22.4 4.8 6.4 10.9 44.2 1.8

48.6 51.3 25.3 5.2 7.8 10.4 46.9 1.4

48.5 49.3 24.9 5.5 7.9 10.9 46.6 1.6

49.4 50.6 32.9 5.5 8.0 1.4 49.0 2.7

48.7 51.3 19.7 5.9 9.5 1.5 53.4 2.3

48.2 51.8 27.4 6.5 12.5 1.3 48.5 2.4

45.8 52.6 24.9 5.9 15.8 1.2 46.3 2.7

57.7 10.9 31.4 3.2

54.0 10.6 35.4 2.7

52.7 11.5 35.8 2.0

53.1 12.3 31.6 1.6

49.7 15.0 32.0 0.8

49.4 9.9 40.7 2.6

43.4 8.9 47.7 2.0

40.7 10.4 48.8 1.5

48.4 11.2 38.6 1.0

40.1 19.1 37.7 0.0

4 of 34

English as a Second Language Referred to MTeam Exceptional Education Need

8.2

7.9

7.5

7.0

6.6

4.9

6.4

6.7

9.2

9.5

13.6 13.1

9.6 10.0

12.7 12.7

13.2 13.7

11.4 11.3

9.2 9.7

6.8 7.1

9.1 1.3

11.3 11.1

10.2 10.4

(*Percentages may not always total to 100% due to incomplete reports submitted by some schools.)

During the course of the 2000-01 school year, records were compiled on 2,474 students. Some students withdrew from SAGE and Comparison schools during the year, while others enrolled as new students. The number of students in SAGE and Comparison schools by grade and school year can be seen in Table 2.

Table 2: Number of Students in SAGE and Comparison Schools by Grade and School Year SAGE COMPARISON* Kindergarten First Grade Second Grade Third grade Totals
199697 1494 1723 NA NA 3217 199798 1524 1567 1541 NA 4632 199899 1416 1525 1446 1531 5918 199900 NA NA 1636 1611 3247 200001 NA NA NA 1542 1542 199697 820 1001 NA NA 1821 199798 676 985 868 NA 2529 199899 887 983 1047 1041 3958 199900 NA NA 991 1045 2036 200001 NA NA NA 932 932

*The number of Comparison schools participating in the study since 1996 has fluctuated from 14 to 17. Student numbers for Comparison schools reflect this fluctuation.

Table 3 illustrates the stability of student enrollment for SAGE and Comparison schools by school year. The data was obtained from student profiles completed by the schools. SAGE classrooms and Comparison classrooms are naturally occurring classrooms. This means that class composition is not held stable; new students enter classes and others withdraw from school during the course of the school year. Some students move from a SAGE School to a Comparison School and vice versa. Table 3: Enrollment Changes in Sage and Comparison Schools by School Year (Number of Students) 5 of 34

1996 -97 Fall Enrollment New Students Withdrawals From SAGE to Comparison From Comparison to SAGE Spring Enrollment 3271 249 362 7 6

SAGE 1997 1998 1999 -98 -99 -00 4544 294 522 11 8 6107 400 484 7 7 3005 242 222 1 4

2000 -01 1503 133 95 1 2

1996 -97 1884 103 204 7 6

Comparison 1997 1998 1999 -98 -99 -00 2522 172 714 11 8 4128 321 420 7 7 1873 162 133 1 4

2000 -01 883 88 38 1 2

3157

4313

6023

3028

1542

1784

1983

4029

1899

932

A New Look At Some Important Questions
The results of the SAGE study have been used to fortify arguments for increasing support for reduced class size programs in Wisconsin and elsewhere. The results have also been attacked on both methodological and ideological grounds by those who feel the program is too expensive and the effects not strong enough to warrant the financial and other systemic costs associated with it. In Wisconsin, and elsewhere, there has been considerable debate on the meaning and results of the SAGE evaluation findings, especially as they relate to the academic achievement variables. At least four persistent issues continue to arise:

1. How to best describe the size and practical significance of the SAGE effect on academic achievement.

6 of 34

2. Whether or not SAGE results persist after first grade. 3. Whether or nor SAGE reduces the African-American/White achievement gap? 4. The impact SAGE has on the achievement of children living in poverty and whether the program mitigates the impact of poor attendance.

This report addresses each of these issues in ways that they have not been specifically addressed in the 1996-2000 annual SAGE evaluation reports.

Data Collection Instruments
To provide information about the processes and outcomes of the SAGE program a number of instruments were used as part of the evaluation. Data reported here relate only to the 1997-98 first grade cohort because the achievement data for this cohort represent the “cleanest” of the three SAGE cohort’s longitudinal achievement record.1 A description of the test and non-test instruments used in 1997-98, 1998-99, and 1999-00 follows. 1. Comprehensive Test of Basic Skills (CTBS) (1997-98, 1998-99, 1999-00). In October 1997, the Comprehensive Test of Basic Skills (CTBS) complete Battery, Terra Nova edition, Level 10 was administered to first-grade students and the Level 11 again in May of 1998. Level 12 was administered to these students in May of their second-grade year (1998-99), and Level 13 in May of their thirdgrade year (1999-00). The purpose of the first grade October administration of the CTBS was to obtain baseline measures of achievement for SAGE schools and Comparison schools. The complete battery includes sub-tests in reading,

7 of 34

language arts, and mathematics. The analyses reported here are for the reading and mathematics sub-test scores. The CTBS was chosen as an achievement measure because it is derived from an Item Response Theory (IRT) model that allows comparison of performance across time. Moreover, it is one of a few instruments that attempts to minimize items biased against minorities and educationally disadvantaged students. 2. Student Profiles (1997-98, 1998-99, 1999-00). This instrument, completed in October and February, provided demographic and other data on each SAGE school and Comparison school student. Specifically related to the results reported here, information regarding student “free-lunch” participation and attendance data were collected from teachers each semester to augment the student profile data.

Persisting Students
As in the annual SAGE reports, we draw a distinction between “participating” students and “persisting” students. Persisting students are those with scores available for both the 1st pre and 3rd post testing who persisted in the same school (and thus the study) through all three grades. During the course of the data collection, a number of students dropped out while new students replaced them. Participating students are those students who were in one of the participating classrooms at the time of testing (this includes persisting students). Because student mobility is generally related to academic performance, the test score averages for persisting students are often larger than those for participating students. General trends in the data were the same for all analyses conducted using the two groups, however. The analysis reported here pertains to

8 of 34

persisting students and therefore the actual mean scores, and possibly the gains, reported probably have a slight positive bias when compared to comparable statistics for all participating students.

Measuring the size of the SAGE effect on achievement
In the social and behavioral sciences, statistical significance is often the stalwart indicator that important results have been found. However, for a number of reasons, the use of statistical significance as an indicator of important effects can be misleading. For example, in our case, with the very large sample sizes involved, relatively small group differences may be found to be statistically significant, possibly leaving the impression that these small differences have practical importance (which may or may not be the case). As a supplement to statistical significance indicators, various “effect size” measures are often reported to help with the interpretation of differences when they are found. Depending on the analytical procedures used, a variety of statistics have been recommended for use as “effect size” indicators (e.g., omega-squared, eta-squared, r squared). All of these indicators express either “variance accounted for” or group differences in terms of standard deviation units. While these indicators do provide appropriate perspective to a technical audience, their interpretation often depends on a firm understanding of the statistical techniques used. Often, what might be considered to be a large or small effect must be considered in the context of the research problem. As with statistical significance, these indicators can be confusing, especially to a lay audience.

9 of 34

In this report, an alternative “effect size” indicator will be used to provide perspective on the size and practical significance of the SAGE effect. Average Growth Curves (AGC) as defined by the average performance of the test norm group across time can be considered an appropriate standard with which to base conclusions regarding program success. Essentially, AGCs provide an indication of growth that might be expected of any group across time and thus provides a basis for comparison in evaluating educational effects. One potential problem with AGCs is that they are based upon cross sectional data, so when comparing these to longitudinal data, we must assume that cohort differences are negligible. Another potential problem with AGCs is that these curves are probably not parallel for low, average, and high scoring cohorts. Test scores tend to produce a “fan spread” pattern across time with initially low scoring students not gaining as much and initially high scoring students making greater gains. Thus, AGCs might represent somewhat ambitious standards for low scoring students and somewhat modest standards for high scoring students. Table 4 shows the published norms for the Reading and Mathematics sub-tests of the Comprehensive Test of Basic Skills (CTBS) complete Battery, Terra Nova edition (ref). The Terra Nova Technical Report also includes means for African-American students included in the norming sample. Figures 1 and 2 show the AGCs for the reading and mathematics scores respectively.

Table 4: Means for Norm Group and African-Americans Included in the Norm Group CTB Test Level Level 10 Level 11 Level 12 Level 13 1st pre 1st post 2nd 3rd Grade 4623 8958 10491 9047 Norm N

10 of 34

Reading NORM Math NORM Reading AA - N Reading AA Math AA - N Math AA

543 506 370 533.3 363 488.4

576 535 256 561.1 252 518.2

608 571 268 591.8 254 551.4

629 606 217 623.4 209 592.4

The data shown in Table 4 and Figures 1 and 2 can be used to form general achievement “expectations” for a general group of students. If students are progressing as the norm group, their growth curve will parallel that of the norm group. A steeper curve indicates accelerated growth beyond what might be expected and a milder curve indicates performance that lags behind that of the norm group. Additionally, it can be seen that the norm group gained an average of 23, 32, and 21 scale score points in reading and 29, 36, and 35 scale score points in math for the 1 st grade pretest to 1st grade post test, 1st grade post test to 2 nd grade post test, and 2nd grade post test to 3rd grade post test intervals respectively. These are average gains experienced by the norm group for each year and can be used to provide a benchmark when evaluating the effects of SAGE. These tables and figures form the basis for the comparisons drawn in the remainder of this report.

11 of 34

Figure 1: AGC for Reading Subscore
650

620

590

Reading NORM Reading AA

560

530 1st pre 1st post 2nd 3rd

Figure 2: AGC for Mathematics Subscore
630

600

570 Math NORM 540 Math AA

510

480 1st pre 1st post 2nd 3rd

12 of 34

Do SAGE “Effects” Persist After First Grade?
A common observation throughout the SAGE evaluation findings is that statistically significant differences between SAGE and Comparison students are observed in first grade, but few additional differences are found following first grade. This issue is important since it has been advanced as a finding that would not support reduced class sizes after first grade. While in general no significant gain differences between SAGE and Comparison students were seen from first to second grade and from second to third, the separation that was observed in first grade was essentially maintained over this period. Since the evaluation effort could not track drop-outs, it is impossible to say whether this advantage would have been maintained had the SAGE student returned to larger classroom settings. Tables 5 and 6 show the SAGE test score averages for SAGE and comparison students across testing periods in reading and mathematics respectively. The panels on the left of these tables show the results for all students, and those on the right show the averages for persisting students (those who remained in the program for all three years of the evaluation period). Figures 3 and 4 show the averages for persisting students in both groups and the AGCs for Reading and Mathematics respectively. Note first that the norm group would suggest that students would gain, on average, 33 points in reading and 29 points in mathematics from fall to spring of the first grade year. In reading, persisting SAGE students gained 47 points in reading and 42 points in mathematics while the comparison students gained 38 and 35 points in reading and math respectively. While both groups outperformed the expected gain, the SAGE students had a nine-point gain advantage in reading and a seven-point gain advantage in

13 of 34

mathematics. Based upon the norm group performance this difference translates into somewhere close to 25-30% of a year’s growth. In reading, after first grade, this difference remains fairly constant. In mathematics, the SAGE group did not gain on the norm group, but did show gains relative to the comparison group that fell significantly off the norm group performance standard. Second grade to third grade changes for both SAGE and the comparison students are comparable to the norm standard for both reading and mathematics.

All 1st Group pre 543 Norm SAGE 533 COMP 535 1st post 576 581 572 2nd 608 608 603

Table 5: Reading All 3rd 629 630 628 Group Norm SAGE COMP

PERSIST 1st pre 543 539 539 1st post 576 586 577 2nd 608 612 605 3rd 629 633 625

ALL Group Norm SAGE COMP 1st pre 506 492 493 1st post 535 539 527 2nd 571 571 556

Table 6: Mathematics ALL 3rd 606 606 596 Group Norm SAGE COMP 1st pre 506 502 501

PERSIST 1st post 2nd 535 571 544 578 536 558

3rd 606 608 592

14 of 34

Figure 3: Reading
640

610 Norm 580 SAGE COMP 550

520 1st pre 1st post 2nd 3rd

Figure 4: Mathematics
630

600

570

Norm SAGE COMP

540

510

480 1st pre 1st post 2nd 3rd

15 of 34

Does SAGE Reduce the AA-White Achievement Gap?
While the above results indicate that participation in the SAGE program leads to greater achievement gains than those not participating, the results of the SAGE evaluation consistently found that the benefits seemed to be greater for AfricanAmerican students than for White students. Thus, the claim is made that SAGE helps reduce the racial achievement gap that is observed in many educational settings. Tables 7 and 8 separate the SAGE and comparison achievement data by race (White/AfricanAmerican) for reading and mathematics respectively. Once again, attention will focus on persisting students although the trends are generally the same for non-persisting students.

AfricanAmericans 1st Group pre 543 Norm SAGE 529 COMP 529

Table 7: Reading by Race AfricanAmericans 1st post 576 574 571 2nd 608 606 603 3rd 629 626 622 Group Norm SAGE COMP AA Norm White 1st pre 543 526 534 533.3

PERSIST 1st post 2nd 576 608 577 601 563 594 561.1

3rd 629 618 611

591.8 623.4

White Group Norm SAGE COMP 1st pre 543 542 543 1st post 576 590 583 2nd 608 619 614 3rd 629 640 638

Group Norm SAGE COMP

1st pre 543 546 547

PERSIST 1st post 2nd 576 608 594 621 588 614 PERSIST 1st post 2nd 576 608 594 621 577 601

3rd 629 642 638

SAGE ONLY 1st Group pre 543 Norm White 542 Af-Am 529

SAGE ONLY 1st post 576 590 574 2nd 608 619 606 3rd 629 640 626 Group Norm White Af-Am 1st pre 543 546 526

3rd 629 642 618

16 of 34

African-Americans 1st 1st post Group pre 506 535 Norm SAGE 472 523 COMP 482 508

Table 8: Mathematics by Race AfricanAmericans 2nd 571 552 534 3rd 606 590 572 Group Norm SAGE COMP AA Norm White 1st pre 506 479 488 488.4

Math

PERSIST 1st post 2nd 535 571 528 556 518 536 518.2

3rd 606 592 572

551.4 592.4

White Group Norm SAGE COMP 1st pre 506 508 504 1st post 535 551 540 2nd 571 584 569 3rd 606 615 609

Math

Group Norm SAGE COMP

1st pre 506 514 513

PERSIST 1st post 2nd 535 571 556 588 550 578 PERSIST 1st post 2nd 535 571 556 588 528 556

3rd 606 616 608

SAGE ONLY 1st Group pre 506 Norm White 508 Af-Am 472

SAGE ONLY 1st post 535 551 523 2nd 571 584 552 3rd 606 615 590 Group Norm White Af-Am 1st pre 506 514 479

Math

3rd 606 616 592

In reading, while the African-Americans in the SAGE program had a pretest score average six points below that of their comparison counterparts, by the end of the first grade they outscored the comparison students by 14 points. This represents a full 20 point gain over the comparison students which, when compared to the AGC represents two-thirds of a year’s worth of growth. By any standard, this is a sizable effect. The advantage gained in the first grade was maintained through the second grade, but diminished somewhat through third grade. The first-to-third grade gain was a net 13 points higher for African-American students in SAGE schools versus those in comparison schools. This represents nearly one-half year’s growth when compared to the AGCs.

17 of 34

White SAGE students also showed a gain advantage over their comparison counterparts (a seven point difference in gain). The same general trend observed for African-American students was also observed for White students: The first grade advantage persisted through second grade and diminished somewhat through third grade. The African-American versus White achievement difference grew only slightly (4 points) for SAGE students but grew over three times as much, 14 points (almost half an academic year’s growth), for non-SAGE students. Figures 5 through 7 show these data for reading.

Figure 5: Reading Scores for African-American Students
640

610 Norm 580 SAGE COMP 550

520 1st pre 1st post 2nd 3rd

18 of 34

Figure 6: Reading Scores for White Students

630

600

Norm SAGE COMP

570

540 1st pre 1st post 2nd 3rd

Figure 7: Reading Scores for SAGE Students by Race

640

610 Norm 580 White Af-Am 550

520 1st pre 1st post 2nd 3rd

The same general trends found for reading were also observed for mathematics. However, the relative advantage of SAGE participation afforded to African-American students appears to be greater in mathematics than in reading. In mathematics, again the African-Americans in the SAGE program had a pretest score average below (nine 19 of 34

points) that of their comparison counterparts, but by the end of the first grade they outscored the comparison students by 10 points. This represents a 19 point gain over the comparison students which, when compared to the AGC again represents two-thirds of a year’s worth of growth. The advantage gained by African-American SAGE students in the first grade grew by another 10 points through the second grade, with the overall 29 point gain maintained through the third grade. This represents nearly one year’s growth advantage when compared to the AGCs. White SAGE students also showed a gain advantage over their comparison counterparts (a five point difference in gain). The same general trend observed for African-American students was also observed for White students: The first grade advantage grew to 10 points through second grade and diminished slightly to eight points through third grade. Overall this represented about a one-forth grade advantage for White students. Over the three years covered by the SAGE evaluation of the 1997-98 first grade cohort, the African-American versus White achievement difference narrowed somewhat (11 points) for SAGE students but grew by 11 points (almost half an academic year’s growth), for non-SAGE students. Compared to the AGCs, this represents an achievement gap narrowing of about two-thirds of an academic year over the three-year period of the study. Figures 8 through 10 show these data for mathematics.

20 of 34

Figure 8: Math for African-Americans
620

590

560

Norm SAGE COMP

530

500

470 1st pre 1st post 2nd 3rd

Figure 9: Math for White
630

600

570

Norm SAGE COMP

540

510

480 1st pre 1st post 2nd 3rd

21 of 34

Figure 10: Math for SAGE
620

590

560

Norm White Af-Am

530

500

470 1st pre 1st post 2nd 3rd

How Does SAGE Effect Children Living in Poverty and Those With Poor Attendance Records?
Children living in poverty Since the SAGE program targeted low achieving schools to begin with, and generally these schools are dominated by students of poverty, it has been claimed that the effects of SAGE are limited to students of poverty. Fortunately, the data collected in the SAGE evaluation allows us to examine this in more detail. In the SAGE student profile, participation in the federally subsidized school lunch program was monitored. Students were classified as low SES (full Federal subsidy), moderate SES (partial Federal subsidy), and high SES (no Federal subsidy). The shortcomings of using the Federal lunch subsidy as a surrogate for SES are obvious and have been thoroughly discussed elsewhere. However, use of this index should provide insight into the differential effects of SAGE among levels of poverty.

22 of 34

For these analyses, interest focuses on only those students participating in SAGE. Tables 9 and 10 show the score averages for the three SES groups broken down by race (AA / White) for reading and mathematics respectively.

All SAGE Students 1st pre 543 538 535 525

Norm SES-1 SES-2 SES-3

Table 9: Reading Scores by SES and ethnicity All SAGE Students 1st 1st post 2nd 3rd pre 576 608 629 543 Norm 585 612 634 READ SES-1 541 580 608 629 539 SES-2 573 600 623 532 SES-3

Persist 1st post 576 588 586 580

2nd 608 616 613 603

3rd 629 637 631 623

All SAGE Students - AfricanAmerican 1st pre 1st post 2nd 543 576 608 Norm 525 572 600 SES-1 520 571 590 SES-2 515 573 594 SES-3

All SAGE Students 3rd 629 622 613 615 1st pre 543 525 522 531 1st post 576 573 574 591 2nd 608 604 597 601 3rd 629 622 614 612

Norm READ SES-1 SES-2 SES-3

All SAGE Students White 1st pre 1st post 543 576 Norm 544 593 SES-1 544 590 SES-2 538 580 SES-3

All SAGE Students 2nd 608 620 620 615 3rd 629 642 639 636 1st pre 543 547 547 538 1st post 576 596 595 583 2nd 608 622 621 615 3rd 629 645 641 637

Norm READ SES-1 SES-2 SES-3

23 of 34

Table 10: Mathematics scores by SES and ethnicity All SAGE Students All SAGE Students 1st 1st 1st 1st pre post 2nd 3rd pre post 506 535 571 606 506 535 Norm Norm 500 546 576 610 505 550 SES-1 MATH SES-1 493 537 571 603 499 543 SES-2 SES-2 481 523 561 596 494 527 SES-3 SES-3

2nd 571 582 576 565

3rd 606 614 604 594

All SAGE Students - AfricanAmerican 1st 1st pre post 2nd 506 535 571 Norm 479 527 556 SES-1 472 520 543 SES-2 462 515 552 SES-3

3rd 606 596 583 584

All SAGE Students 1st 1st pre post 506 535 Norm 476 527 MATH SES-1 474 528 SES-2 483 525 SES-3

2nd 571 560 546 552

3rd 606 598 584 583

All SAGE Students White 1st 1st pre post 506 535 Norm 513 556 SES-1 508 549 SES-2 500 537 SES-3

2nd 571 587 583 572

3rd 606 618 612 603

All SAGE Students 1st 1st pre post 506 535 Norm 517 560 MATH SES-1 513 553 SES-2 504 537 SES-3

2nd 571 591 586 571

3rd 606 621 612 599

Examination of the data in Tables 9 and 10 shows an expected pattern: In general, those in lower SES categories (3) score lower than those in higher categories (1) across the board. While these averages are based on relatively small sample sizes, it also appears that the gap between high and low SES levels seems to widen across time. This is more true in math than in reading. It also appears to be more the case for AfricanAmerican students than for White students. For example in mathematics, the SES gap for White students is 13 points at the beginning of first grade and grows to 22 points by the end of third grade. For African-American students, low SES students actually 24 of 34

outscore the high SES group by seven points at the beginning of first grade but fell behind the high group by 15 points by the end of third grade; a 22 point swing. Coupled with the “achievement gap” results presented above, it would appear that the SAGE program has differential effects for varying levels of SES with the relatively higher SES African-American students profiting the most. Figures 11 through 14 show these results graphically. Figures 12 and 14 in particular show the apparent differential SES effect compared across race.

Figure 11: Reading all SES

640

610 Norm SES-1 580 SES-2 SES-3 550

520 1st pre 1st post 2nd 3rd

25 of 34

Figure 12: Reading SES by ethnicity

640

610

Norm SES-1(AA) SES-1(W) SES-3(AA) SES-3(W)

580

550

520 1st pre 1st post 2nd 3rd

Figure 13: Mathematics by SES
630

600 Norm SES-1 SES-2 540 SES-3

570

510

480 1st pre 1st post 2nd 3rd

26 of 34

Figure 14: Mathematics by SES and Race
620 590 Norm 560 530 500 470 1st pre 1st post 2nd 3rd SES-1(AA) SES-1(W) SES-3(AA) SES-3(W)

Children with poor attendance The SAGE evaluation project collected a variety of data regarding individual participants. Among these data were attendance data for the students in SAGE and Comparisons schools. An examination of the achievement trends for various attendance patterns among these students is revealing. For this analysis, students were divided into three groups: 1) high attenders who were reported to have missed 5 or fewer days of school per year, 2) moderate attenders who missed between 5 and 15 days per year and 3) low attenders who missed more than 15 days of school per year. Table 11 shows a summary of the achievement score averages for each of these three groups in the SAGE and comparison schools along with the norm group for both mathematics and reading.

27 of 34

SAGE

COMP

Table 11: Achievement Score Averages Math 1st pre 1st post 2nd 506 535 571 Norm 505 550 582 S High Attend 499 543 576 S Mid Attend 494 527 565 S Low Attend 503 541 563 C High Attend 500 533 553 C Mid Attend 498 524 551 C Low Attend Math Norm S Low Attend C High Attend Reading Norm S High Attend S Mid Attend S Low Attend C High Attend C Mid Attend C Low Attend 1st pre 506 494 503 1st pre 543 540 539 531 542 536 544 1st post 535 527 541 1st post 576 588 585 580 580 578 570 2nd 571 565 563 2nd 608 616 613 602 606 608 598

3rd 606 614 604 594 597 590 576 3rd 606 594 597 3rd 629 637 631 624 630 623 621

SAGE

COMP

In general, the achievement scores for both the SAGE students and the comparison students are ordered as expected: High attending students have the highest score average and low attending students have the lowest average. In math, it can be seen that the trends for both low and high attending students are roughly the same. Figures 15 and 16 show these trends for the low and high attending students in math. In both cases the SAGE group “separates” from the comparison group during first grade and maintains the advantage through second and third grade. This indicates that the general achievement patterns that held in the full group also hold for the groups when split by attendance. Figure 17 shows the SAGE low attending group and the comparison high attending group. Note that the achievement pattern is nearly identical (although the

28 of 34

SAGE students do gain slightly in early years) indicating that the SAGE “effect” appears to mitigate the devastating effects of low attendance that is observed in many schools with a high poverty rate and generally low achievement.

Figure 15: Math High Attend

610

580 Norm 550 S High Attend C High Attend

520

490 1st pre 1st post 2nd 3rd

Figure 16: Math Low Attend

610

580 Norm 550 S Low Attend C Low Attend

520

490 1st pre 1st post 2nd 3rd

29 of 34

Figure 17: Math High Comparison Versus low SAGE

610

580 Norm 550 S Low Attend C High Attend 520

490 1st pre 1st post 2nd 3rd

The same general patterns are found in reading. Figures 18 and 19 show the reading score patterns for high and low attending students respectively. Figure 20 shows the SAGE low attending students and the comparison high-attending students. Note that the same pattern observed for math is seen in the reading scores.

Figure 18: Reading High Attend
640 610 580 550 520 490 1st pre 1st post 2nd 3rd Norm S High Attend C High Attend

30 of 34

Figure 19: Reading Low Attend
640

610

580

Norm S Low Attend C Low Attend

550

520

490 1st pre 1st post 2nd 3rd

Figure 20: Reading SAGE Low Attend Comparison High Attend
640

610

580

Norm S Low Attend C High Attend

550

520

490 1st pre 1st post 2nd 3rd

31 of 34

Discussion
This analysis offers a different interpretive perspective on the 1996-2000 SAGE evaluation data, and uses AGCs as a benchmark for interpreting the relative academic performance of reduced size classes and larger classes. It is hoped that this interpretive perspective will help decision makers evaluate the academic impact of SAGE reduced size classes in a realistic context. Based on results reported here, the following conclusions seem reasonable:

1. The “size” of the SAGE effect varies by subject matter, but represents 1/3 to 1/2 of one school year’s worth of growth when compared to the norm group averages. By any standard, this is a significant gain especially when one considers that most of the gain occurred during one school year.

2. The greatest separation between the achievement of SAGE and comparison students occurs in first grade. This separation persists through third grade. From the data available, it is not possible to determine what might have happened had the program ended following first grade, forcing students to return to regular sized classrooms. It appears, however, that the gains made in first grade are sustainable, and in some instances increased upon, if similar classroom conditions exist in subsequent grades.

3. African-American students seem to profit more from the SAGE experience than White students when compared to non-SAGE students. Within SAGE classrooms

32 of 34

the greatest effect for SAGE students appears to be in the highest SES groups participating in the program.

4. The SAGE program narrows the achievement gap between African-American and White Students in first grade and prevents it from widening in second and third grades. In larger class size comparison classrooms the achievement gap between African-American and White students widens each year.

5. The SAGE “effect” may help compensate for attendance problems as it was generally observed that those low attending students in SAGE perform comparably to the best attending non-SAGE students. Increased attendance could enhance the SAGE effect by as much as 50%.

33 of 34

Notes

For the 1996-97 cohort, the level 10 test was used as both the 1 st grade pre and post test. This resulted in many of the students in the SAGE and comparison schools hitting the ceiling on the post test. For the third grade year of the 1998-99 1st grade cohort, the Milwaukee Public Schools adopted the Basic Multiple Assessments Plus test to test all third-grade students in the district. To avoid compromising the testing for both the SAGE Evaluation Project and the Milwaukee Public Schools, an agreement was reached to have the third-grade SAGE students in the Milwaukee Public Schools take the Basic Multiple Assessments Plus test during the SAGE testing window. The Basic Multiple Assessments Plus test contains more subtests than the Comprehensive Test of Basic Skills (CTBS) used in the SAGE Evaluation; however, both tests are Level 13 Form A and are on the same scale. Because the test was given earlier than normal, the third grade results might be compromised.

1

34 of 34


				
DOCUMENT INFO
Shared By:
Tags: Class, -Size
Stats:
views:20
posted:11/27/2009
language:English
pages:35
Description: Class-Size-Reduction-in-Wisconsin-A-Fresh-Look-at-the-Data