VIEWS: 19 PAGES: 34 POSTED ON: 10/5/2011 Public Domain
California Standards Test and CAHSEE Correlation Use of Student Data for Targeted Preemptive Intervention November 1, 2006 Dr. Janis Fries-Martinez, Principal Gerardo Loera, Assistant Principal Polytechnic High School Los Angeles Unified School District 2 In This Presentation You Will Learn: How to accurately identify 10th grade students who need intervention for the CAHSEE before they take it in the spring How to identify a target group of students to strategically raise the percentage of students scoring Proficient or Above on the CAHSEE in order to meet AYP requirements How to predict, with significant amount of certainty, which 10th grade students will pass and fail the CAHSEE on their initial try No Child Left Behind and State Accountability Criteria - CAHSEE AYP and API are dependent in large part to CAHSEE Graduation scores AYP: Annual Measurable Objectives are measured mainly by success of 10th grade students taking the CAHSEE the 1st time CAHSEE Problem: High failure rate AYP API on CAHSEE Problem: Not enough students scoring Proficient or Above on CAHSEE Guiding Questions How do we effectively target students for pre- emptive intervention for success on the CAHSEE before they initially take it? How do we proactively identify students that need additional support? Do we know which students are likely to pass the test or get a proficient score ? What are Passing and Proficient Scores? A passing score on the CAHSEE is: 350 or greater on the English Language Arts portion of the test AND 350 or greater on the Mathematics portion of the test A proficient score is: 380 or greater on the English portion of the test 380 or greater on the Mathematic portion of the test Some Possible Attributes Contributing To Success On The CAHSEE? Grades/Marks Demographics Socioeconomic status Learning environment CELDT Scores Language Classification Special Education Status CST Scores Periodic Assessments Etc. Studied CST/CAHSEE Correlation at Different High Schools Test Groups: Poly HS: Class of 2006 Poly HS: Class of 2007 Santee HS: Class of 2007 LAUSD Class of 2008 (37,000+ scores) LAUSD Class of 2007 (36,000+ scores) Polytechnic High School Class of 2007 9th Grade ELA CST vs. 10th Grade ELA CAHSEE – Matched Scores Note: 800+ scores included Source: Secondary Student Information System South LA High School #1 - Class of 2007 9th Grade ELA CST vs. 10th Grade ELA CAHSEE – Matched Scores LAUSD Class of 2008 - 9th Grade ELA CST vs. 10th Grade ELA CAHSEE – Matched Scores LAUSD 9th Grade CST ELA vs. 10th Grade CAHSEE ELA 450 400 CAHSEE ELA SS 05-06 350 300 Y = 189.48 + 0.5512X R2 = 0.6672 250 250 300 350 400 450 500 550 600 CST ELA SS 04-05 Note: 37,000+ scores included Source: School Information Branch LAUSD Class of 2008 - 9th Grade Algebra I CST vs. 10th Grade Math CAHSEE – Matched Scores LAUSD 9th Grade Algebra I CST vs 10th Grade CAHSEE 450 CAHSEE SS 05-06 400 350 300 y = 0.4796x + 219.73 R2 = 0.4555 250 200 250 300 350 400 450 500 550 600 CST Algebra I SS 04-05 Note: 23,282 scores included Source: School Information Branch LAUSD Class of 2008 - 9th Grade Geometry CST vs. 10th Grade Math CAHSEE – Matched Scores LAUSD 9th Grade Geometry CST vs 10th Grade CAHSEE 450 CAHSEE Math 05-06 400 350 300 y = 0.4537x + 252.82 R2 = 0.602 250 100 200 300 400 500 600 CST Geometry SS 04-05 Note: 8,410 scores included Source: School Information Branch LAUSD Class of 2007 - 9th Grade MATH CST vs. 10th Grade Math CAHSEE – Matched Scores LAUSD 9th Grade Math CST vs. 10th Grade Math CAHSEE 450 CAHSEE Math SS 04-05 400 350 300 y = 0.5189x + 211.05 2 R = 0.4843 250 100 200 300 400 500 600 CST Math SS 03-04 Note: 36,190 scores included Source: School Information Branch Math CAHSEE Predictors Mathematics – Geometry Mathematics – Algebra I Model Model 9th Grade CST score of 214 or 9th Grade CST score of 271 or higher predicts a passing score of higher on Algebra I CST predicts 350 or higher on the CAHSEE a passing score of 350 or higher on the CAHSEE 9th Grade CST score of 280 or higher predicts Proficient score of 9th Grade CST score of 334 or 380 or higher on the CAHSEE higher predicts a Proficient score (AYP) of 380 or higher on the CAHSEE Summary of Predictors English Language Arts Model Based on LAUSD Class of 2008 (37,000+ scores) 9th Grade English CST score of 292 or higher predicts a passing score of 350 or higher on the CAHSEE 9th Grade CST score of 346 or higher predicts Proficient score of 380 or higher on the CAHSEE (To meet AYP) ELA – “Reliability” Matrix for predicting Passing CAHSEE ELA score – 83.4% accurate Data from the LAUSD School Information Branch – Class of 2008 Predicted Results Model: 292 ELA CST = 350 ELA CAHSEE Pass Fail Total Actual Pass 24071 2914 26985 Results Fail 3549 8212 11761 24071 8212 83.4% Accuracy 24071 8212 2914 3549 Therefore the model for predicting a passing score or failing score on the English portion of the CAHSEE based on the 9th grade CST is correct 83.4% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 ELA – “Reliability” Matrix for predicting Passing CAHSEE ELA Proficient or Above score – 84.6% accurate Data from the LAUSD School Information Branch – Class of 2008 Predicted Results Model: 346 ELA CST = 380 ELA CAHSEE Prof/Above Below Prof Actual Prof/Above 11288 2135 Results Below Prof 3846 21477 11288 21477 84.6% 11288 21477 2135 3846 Therefore this model is correct 84.6% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Algebra I - “Reliability” Matrix for predicting Passing CAHSEE Math Passing score – 72.9% accurate Data from the LAUSD School Information Branch – Class of 2008 Predicted Results Model: 271 Alg. I CST = 350 MATH CAHSEE Pass Fail Actual Pass 11231 2930 Results Fail 3383 5737 11231 5737 72.9% 11231 5737 2930 3383 Therefore the model for predicting a passing or failing score on the Mathematics portion of the CAHSEE based on the 9th grade CST is correct 72.9% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Algebra I – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score – 85.1% accurate Data from the LAUSD School Information Branch – Class of 2008 Predicted Results Model: 334 Alg. I CST = 380 MATH CAHSEE Prof/Above Below Prof Actual Prof/Above 2654 2742 Results Below Prof 726 17159 2654 17159 85.1% 2654 17159 2742 726 Therefore the model for predicting a score on the Mathematics portion of the CAHSEE based on the 9th grade CST is correct 85.1% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Geometry - “Reliability” Matrix for predicting Passing CAHSEE Math Passing score – 88.7% accurate Data from the LAUSD School Information Branch – Class of 2008 Predicted Results Model: 214 Geom. CST = 350 MATH CAHSEE Pass Fail Actual Pass 7374 68 Results Fail 886 82 7374 82 88.7% 7374 82 68 886 Therefore the model for predicting a passing score or failing score on the Mathematics portion of the CAHSEE based on the 9th grade Geometry CST is correct 88.7% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Geometry – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score 82.1% accurate Data from the LAUSD School Information Branch – Class of 2008 Predicted Results Model: 334 Geom. I CST = 380 MATH CAHSEE Prof/Above Below Prof Actual Prof/Above 4539 721 Results Fail 781 2369 4539 2369 82.1% 4539 2369 721 781 Therefore the model for predicting a Proficient or Above score on the Mathematics portion of the CAHSEE based on the 9th grade Geometry CST is correct 82.1% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Math (ALL) - “Reliability” Matrix for predicting Passing score on CAHSEE Math – 72.7% accurate Data from the LAUSD School Information Branch – Class of 2007 Predicted Results Model: 268 Math CST = 350 MATH CAHSEE Pass Fail Actual Pass 19070 3634 Results Fail 6237 7249 19070 7249 72.7% 19070 7249 3634 6237 Therefore the model for predicting a passing score or failing score on the Mathematics portion of the CAHSEE based on the 9th grade Math CST is correct 72.7% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Math (ALL) – “Reliability” Matrix for predicting Passing CAHSEE Math Proficient or Above score 81.5% accurate Data from the LAUSD School Information Branch – Class of 2008 Predicted Results Model: 326 Math CST = 380 MATH CAHSEE Prof/Above Below Prof Actual Prof/Above 5652 5127 Results Below Prof 1564 23974 5652 23847 81.5% 5652 23847 5127 1565 Therefore the model for predicting a Proficient or Above score on the Mathematics portion of the CAHSEE based on the 9th grade Math CST (regardless of which CST test was taken such as Algebra I, Geometry, General Math etc.) is correct 81.5% of the time. This is a Confusion Matrix as described in the work of Kohavi and Provost, 1998 Summary of Predictors English Language Arts Mathematics Model Model Based on LAUSD Class of 2008 (37,000+ scores) Based on LAUSD Class of 2007 (36,190 scores) 9th Grade English CST score of 9th Grade CST score in Math of 292 or higher predicts a passing 268 or higher predicts a passing score of 350 or higher on the score of 350 or higher on the CAHSEE CAHSEE 9th Grade CST score of 346 or 9th Grade CST score in Math of higher predicts Proficient score of 326 or higher predicts a 380 or 380 or higher on the CAHSEE higher on the CAHSEE (AYP) (AYP) Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model For Program Improvement Schools: A Review of the 2006 AYP Progress Report needs to be made in order determine which subgroups, if any, fell short of meeting the Annual Measurable Objectives (AMO) Recall that the AMOs are directly associated with the percentages of 10th grade students that score Proficient or Above on the ELA and Math portions of the CAHSEE Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model For Program Improvement Schools (cont.): Once a subgroup has been identified, an SSIS extract can be performed to create a list of students needing pre-intervention The extract should produce a list of 10th grade students that scored less than a 346 on their English 9th Grade CST. This list should be sorted in decreasing order by their English 9th Grade CST scaled score. Deciding Which Students To Target – Possible Action Plans For Applying the Math Model For Program Improvement Schools (cont.): Once a subgroup has been identified, an SSIS extract can be performed to create of list of students needing pre-intervention The extract should produce a list of 10th grade students that scored less than a 326 on their Math 10th Grade CST. This list should be sorted in decreasing order by their Math 9th Grade CST scaled score. Deciding Which Students To Target – Possible Action Plans For Applying the ELA Model For Program Improvement Schools (cont.): The resulting lists are prioritized lists of students that will benefit from pre-intervention and would most likely not score a Proficient or Above on the CAHSEE without additional support. Deciding Which Students To Target – Possible Action Plans For Applying the Predictive Model How to identify 10th grade students that will probably fail one or both parts of the CAHSEE? Perform an SSIS extract for 10th Grade students that scored less than a 268 on their 9th Grade Math CST Perform an SSIS extract for 10th grade students that scored less than a 292 on their 9th Grade English CST Deciding Which Students To Target – Technical Assistance Cynthia Lim in the LAUSD School Information Branch has offered to provide schools with technical support on identifying students at- risk of not passing the CAHSEE or not getting a Proficient or Above score. Cynthia.lim@lausd.net CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 Master schedule permitted for approximately 170 seats for CAHSEE preemptive intervention in Spring of 2006 Selected a window of scores predicted to be 300-370 Based on student performance this resulted in a target range as follows: For ELA: CSTs of 212 - 328 For Math: CSTs of 164 - 308 CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 - Results On average, non prepped 10th Graders matched the predicted values. Math predicted mean score: 347.5 Math actual mean score: 348.5 ELA predicted mean score: 353.0 ELA actual mean score: 352.2 CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 - Results ELA Math # In preemptive 130 166 preparation # Predicted to pass 108 118 # Actual did pass 120 143 Increase +12 +25 % Increase 11.1% +21.2 # Predicted proficient 48 16 # Actual Proficient 61 67 Increase +13 +51 % Increase 27.1% 319% CAHSEE Preemptive Intervention Case Study – Polytechnic High School 2006 - Results CAHSEE Preemptive intervention works! Preparation was a cost effective means of positively impacting the study group.