Ca Performance Test Workshop by hmb10625

VIEWS: 18 PAGES: 31

Ca Performance Test Workshop document sample

More Info
									                                                                     STAR Post-Test Workshop 2009



STAR Post-Test Workshop 2009

Objectives
      District STAR coordinators will be able to:
          • Describe the purposes of STAR reports
          • Interpret STAR results
          • Explain key statistics
          • Compare and contrast types of reports
          • Identify proper uses of reports


Agenda
         Registration and take quiz
         Introduction and What’s New
         Results and Statistical Analysis
         Using Results
         Summary and Internet Reports
         Data CDs
         Individual Student Reports
         Teacher Reports
         Early Assessment Program (EAP)




                                                    2009 Post-Test Guides will be delivered to
                                                    districts after August 17. They may be
                                                    downloaded at www.startest.org/reports.html.




August 2009                                 ETS               2009 STAR Post-Test Workshop     1
                                                                                            What’s New for 2009?




What’s New for 2009?
Excerpted from the 2009 STAR Post-Test Guide:

                                       Table I.1 What’s New in 2009
                        Change                                         Affected Report(s)
Reported Test Results
The default mathematics test for students in grade seven who      -   STAR Student Report for CST
took a California Mathematics Standards Test is the California    -   STAR Student Record Label
Standards Test (CST) for Grade 7 Mathematics. Students who        -   Subgroup Summary reports
did not mark a CST for mathematics on their answer document       -   All grade-level reports on which
will receive CST for Grade 7 Mathematics results.                     results for grade seven are
                                                                      presented, such as the Student
                                                                      Master List
California Modified Assessment (CMA) performance levels—          -   STAR Student Report for CMA
advanced, proficient, basic, below basic, and far below basic—    -   STAR Student Record Label
are reported for students in grades three through five who took   -   Subgroup Summary reports
a CMA. English–language arts (ELA) scores for students in         -   All grade-level reports on which
grade four include the results for the California Writing             results for grades three, four,
Modified Assessment if the student took this component of the         and five are presented, such as
ELA test.                                                             the Student Master List
CMA results are reported for students in grades six through       -   STAR Student Report for CMA
eight who took a CMA. Performance levels and scale scores         -   STAR Student Record Label
are not reported this year; only percent-correct scores by        -   Subgroup Summary reports
content area are reported. Students in grade seven who took the   -   All grade-level reports on which
California Writing Modified Assessment component of the               results for grades six, seven, and
CMA for ELA test will also receive results for the writing            eight are presented, such as the
component.                                                            Student Master List
Performance levels for ELA and mathematics for all levels of      -   STAR Student Report for CAPA
the California Alternate Performance Assessment (CAPA)            -   STAR Student Record Label
have been recalibrated.                                           -   Subgroup Summary reports
                                                                  -   All grade-level reports, such as
                                                                      the Student Master List
A performance level is reported for the CAPA for Science test.    -   STAR Student Report for CAPA
                                                                  -   STAR Student Record Label
                                                                  -   Subgroup Summary reports
                                                                  -   All grade-level reports on which
                                                                      results for grades five, eight, and
                                                                      ten are presented, such as the
                                                                      Student Master List




August 2009                                ETS                           2009 STAR Post-Test Workshop         2
                                                                                        What’s New for 2009?



                     Change                                        Affected Report(s)
Because the California Achievement Test/Sixth Edition Survey - STAR Student Report for CST
(CAT/6 Survey) was not offered in 2009, CAT/6 Survey data       - STAR Student Report for CMA
are not reported.                                               - STAR Student Record Label
                                                                - STAR Group Summary
                                                                - STAR Group Summary: CAT/6
                                                                  Survey
                                                                - Subgroup Summary reports
                                                                - All grade-level reports on which
                                                                  results for grades three and
                                                                  seven are presented, such as the
                                                                  Student Master List
Standards-based Tests in Spanish (STS) performance levels— - STAR Student Report for STS
advanced, proficient, basic, below basic, and far below basic— - STAR Student Record Label
are reported for students in grades two through four who took   - Subgroup Summary reports
an STS.                                                         - All grade-level reports on which
                                                                  results for grades two, three, and
                                                                  four are presented, such as the
                                                                  Student Master List
STS results are reported for students in grades five through    - STAR Student Report for STS
seven who took an STS. Performance levels and scale scores      - STAR Student Record Label
are not reported this year; only percent-correct scores by      - Subgroup Summary reports
content area and reporting cluster are reported for             - All grade-level reports on which
reading/language arts (RLA) and grade-level mathematics           results for grades five through
tests.                                                            seven are presented, such as the
                                                                  Student Master List
STS results are reported for students in grades eight through   - STAR Student Report for STS
eleven who took an STS, including the new end-of-course         - STAR Student Record Label
tests, the STS for Algebra I (grades seven through eleven) and - Subgroup Summary reports
the STS for Geometry (grades eight through eleven).             - All grade-level reports on which
Performance levels and scale scores are not reported this year;   results for grades eight through
only percent-correct scores by content area are reported.         eleven are presented, such as the
                                                                  Student Master List
Group and Subgroup Reporting
Because the CAT/6 Survey was not offered in 2009, CAT/6          - STAR Group Summary: CAT/6
Survey data are not reported.                                      Survey
Accommodations and Modifications
For students in grades four and seven who took a CST for         - STAR Student Report for CST
Writing, when marked, option “P” (Student used an audio CD       - STAR Student Record Label
presentation) and option “Z” (text examiner read aloud text in   - All grade-level reports on which
the Writing Prompt and Response Booklet) were                      results for grades four and seven
accommodations, not modifications.                                 are presented, such as the
                                                                   Student Master List




August 2009                                ETS                        2009 STAR Post-Test Workshop        3
                                                                                                  Report Results



       Report Results
       Q. What are purposes of STAR Reports?




Excerpted from the 2009 STAR Post-Test Guide:
  Chapter I.2 Introduction
  Purpose of the Reports and Using the Results
    The results for tests within the Standardized Testing and Reporting (STAR) Program are used for three
  primary purposes:
   1. Communicating students’ progress in attaining proficiency on the state’s academic standards to
      students, parents/guardians, and teachers. In developing the legislation for the STAR Program, the
      Legislature recognized that school districts will conduct their own ongoing diagnostic assessments
      and provide information on the results of these assessments to parents/guardians and teachers on a
      regular basis. The Legislature also recognized that local diagnostic assessment is the primary way in
      which to identify academic strengths and weaknesses (Education Code Section 60602).
   2. Informing decisions, along with local assessment data, that teachers and administrators make about
      helping students improve their achievement and about improving the educational program.
   3. Providing data for state and federal accountability programs. These data are used to calculate each
      school and school district’s Academic Performance Index (API) and adequate yearly progress (AYP)
      to meet the requirements of the No Child Left Behind Act of 2001.


       Q. What is a scale score?


August 2009                                  ETS                            2009 STAR Post-Test Workshop      4
                                                                                           Report Results: Scale Scores



       Report Results: Scale Scores
Excerpted from the 2009 STAR Post-Test Guide:


  Chapter I.3 Interpreting Reports
  Equating and Scaling
                                                                                       A scale score is derived
    When tests are constructed for each grade, every effort is made to make
                                                                                       from a statistical process.
  the tests parallel and of the same level of difficulty from one year to another.     It is not possible to cal-
  However, even with those efforts, small differences in test difficulty still         culate a scale score by
  exist between test forms. A psychometric procedure called equating makes             multiplying a student’s
  adjustments for test difficulty so that students in one year are held to the         percent correct in a con-
  same standards as students in another year.                                          tent area by 600.
              …
  Scale Scores for the STAR Program
    The advantage of the scale score metric is that it allows a particular score (for example, 350) to mean the
  same thing across test forms, even though the difficulty of the test forms may vary. An equating process
  that adjusts for the difficulty of the test form permits this. Because percent-correct scores are defined in
  terms of the number of items answered correctly (the raw score metric) they are, by definition, associated
  with the specific form of the test taken, unadjusted for difficulty—that is, they are dependent on the
  difficulty of the test items and the ability level of those who are taking the test.
    Scale scores are used in the evaluation of overall student performance. Unlike raw scores (that is,
  number-correct scores or percent-correct scores) within the same grade and subject, scale scores provide a
  common reference across years, making interpretation easier. The scale score performance-level cut points
  are held constant from year to year for each grade level and content area, while the number- or percent-
  correct score associated with each scale score may change.




August 2009                                     ETS                             2009 STAR Post-Test Workshop         5
                                                                                  Reporting Clusters or Content Areas



Reporting Clusters or Content Areas
Examples
       Mathematics (from Student Master List Summary, End-of-Course report)




       English–Language Arts (from a STAR CST Student Report)




       Q. Why should you NOT compare clusters from year to year?




August 2009                                              ETS                  2009 STAR Post-Test Workshop         6
                                                                                                    Results: Clusters

Results: Clusters
Excerpted from the 2009 STAR Post-Test Guide:
  Chapter I.3 Interpreting Reports
  Interpreting Results
    Because cluster scores are constructed from test questions of like content, the test questions may be
  easier or more difficult as a group than the overall test form. Thus, percent-correct values based on the
  cluster scores may even differ from the percent correct obtained for the total test. Because of this and the
  fact that unadjusted raw scores are used to compute the percent-correct values, the cluster percent-correct
  scores do not behave in the same way as do the scale scores and cannot be used to calculate the scale
  scores.
    Reporting clusters and the number of items that comprise each are provided in Appendix A. CST data
  start on page 129; CMA cluster data start on page 137; STS cluster data start on page 139. There are no
  reporting clusters for the CAPA. Reporting clusters can help teachers and instructional leaders pinpoint
  areas of student strengths and weaknesses. However, reporting clusters should be interpreted cautiously,
  and two very important limitations of CST, CMA, and STS reporting clusters should always be kept in
  mind:
     1. Reporting clusters are based on different numbers of questions. In some cases, the total number of
         questions that make up a reporting cluster may be quite small; the small number results in scores
         that are less reliable than the overall test scores.
     2. Reporting-cluster scores may vary from year to year because the difficulty of the questions in the
         reporting clusters may vary. While the overall test scores are equated to adjust for differences in
         difficulty from year to year, that is not done for the reporting clusters.




August 2009                                    ETS                            2009 STAR Post-Test Workshop         7
                                                                 Reporting Clusters: Post-Test Guide Appendix A Sample

         Reporting Clusters: Post-Test Guide Appendix A
         Sample
         Data for the “Avg & Correct Statewide” columns will be available in the printed Post-Test
         Guides and in the Post-Test Guide posted to startest.org after August 17, 2009.

                         CALIFORNIA HISTORY–SOCIAL SCIENCE STANDARDS TESTS
                            Number of Questions for 2009 Reporting Clusters and the
                              Average Percent Correct on Each Reporting Cluster
                                                                                    Avg % Correct Statewide
Grade/                                                          No. of                    Minimally     Minimally
 Test                     Reporting Cluster                    Questions         All      Proficient    Advanced
Grade Eight (Grades Six, Seven, and Eight Standards)
        World History and Geography: Ancient Civilizations        16
        Late Antiquity and the Middle Ages                        14
        Renaissance/Reformation                                   10
        U.S. Constitution and the Early Republic                  22
        Civil War and Its Aftermath                               13
Grade Eleven (U.S. History)
        Foundations of American Political and Social Thought      10
        Industrialization and the U.S. Role as a World Power      13
        United States Between the World Wars                      12
        World War II and Foreign Affairs                          12
        Post-World War II Domestic Issues                         13
                                      END-OF-COURSE HISTORY–SOCIAL SCIENCE
World History
       Development of Modern Political Thought                    13
       Industrial Expansion and Imperialism                       10
       Causes and Effects of the First World War                  14
       Causes and Effects of the Second World War                 13
       International Developments in the Post-World War II
                                                                  10
       Era




August 2009                                    ETS                            2009 STAR Post-Test Workshop          8
                                                                      Using Results: What Can and Cannot Be Compared?

       Using Results: What Can and Cannot Be Compared?
       Do compare:
          •
          •
          •

                           Do NOT compare:
                              •
                              •
                              •
Excerpted from the 2009 STAR Post-Test Guide:
  Chapter I.4 Comparing Results
  Comparing CST Results
     When comparing results for the CSTs, compare results only within the same content area and grade; that
  is, compare grade two ELA in 2008 to grade two ELA in 2009 or grade six mathematics in 2008 to grade
  six mathematics in 2009. No direct comparisons should be made between grades or between content areas.
  In addition, comparisons should be made only within the same testing program. Results for the CST for
  Grade 3 ELA cannot be compared to results for the CMA for Grade 3 ELA, for example.
     For the CSTs, the reviewer may compare results for the same grade and content area across years within
  a school, between schools, or between a school and its district, its county, or the state. When comparing
  results for the CMA for grades three through five and for the STS for grades two through four, compare
  results only within the same content area and grade for this year only, because 2009 is the first year
  performance-level data are available. No direct comparisons should be made between grades or between
  content areas. When making comparisons, the reviewer should consider comparing the percent of students
  scoring proficient and advanced. This is because the state target is for all students to score at or above
  proficient.
     Two types of comparisons are possible:
               1. Comparing the average scale score; or
               2. Comparing the percent of students scoring at each performance level.
     Comparisons may also be made by calculating the overall percent of students within a school who scored
  proficient and advanced and comparing that percent to the overall percent of students in another school, the
  district, the county, or the state who scored proficient or advanced. The CST Summary Report, which can
  be found at http://star.cde.ca.gov/, provides this information for CST results for a selected reporting
  population. Information about this report can be found on page 118.
     When making comparisons across years within a given grade and content area, it is important to
  understand that even when the number of students is the same, the group’s composition from year to year
  may be quite different if student mobility (transiency) is high.
     When comparisons are made across years, they are actually a comparison of different groups of students
  with different traits taking different tests. Generally, there will be more variance in scores from year to year
  when small numbers of students are tested.
     While there may be a valid comparison to be made between students within a grade and content area, it is
  not valid to subtract a student’s or class’s scale score received one year in a given content area from the
  scale score received the previous year in the same content area in order to show growth. While the scale
  scores may look the same, they are independently scaled so that differences for the same students across
  years cannot be calculated using basic subtraction.
     Any comparison of groups between years should not be used for diagnostic, placement, or
  promotion or retention purposes. Decisions about promotion, retention, placement, or eligibility for
  special programs may use or include STAR Program results only in conjunction with multiple other
  measures including, but not limited to, locally administered tests, teacher recommendations, and
  grades.


August 2009                                    ETS                              2009 STAR Post-Test Workshop         9
                                                 Summary Reports: Student Master List Summary, Grade 7



Summary Reports: Student Master List Summary, Grade 7




August 2009           ETS                                     2009 STAR Post-Test Workshop         10
                                                Summary Reports: Student Master List Summary:
                                                          CST Biology End-of-Course Sample


Summary Reports: Student Master List Summary:
CST Biology End-of-Course Sample




August 2009                       ETS                2009 STAR Post Test Workshop         11
                                             Summary Reports: Subgroup Summary:
                                            Ethnicity for Economically Disadvantaged


Summary Reports: Subgroup Summary:
Ethnicity for Economically Disadvantaged




August 2009                         ETS    2009 STAR Post Test Workshop         12
                                                                                            Internet Reports



Internet Reports
       Select a report (“Test Results”)
       1. Select a test from the dropdown menu.
               You can see reports for the entire state by not selecting County, District, School.
       2. If needed, select County, District, and School from dropdown menus. You must select a
            county to access district reports. You must select a district to access school reports.
       3. If needed, select a demographic group from the dropdown menu.
       4. If needed, select a demographic subgroup from the dropdown menu.
       5. Select View Report.




       Internet Reports: CST Sample




August 2009                                       ETS                  2009 STAR Post Test Workshop     13
                                                                                   Internet Reports



       Internet Reports: CST Summary (only available for CST. Select Test dropdown.)




       Internet Reports: CMA Sample




August 2009                                  ETS               2009 STAR Post Test Workshop    14
                                                                Internet Reports



       Internet Reports: STS Sample




August 2009                           ETS   2009 STAR Post Test Workshop    15
                                                                   Internet Reports



        Internet Reports: CAPA Samples
State




August 2009                              ETS   2009 STAR Post Test Workshop    16
                                                                                                Your District



Your District

       To whom/which groups will you make presentations?


       What data will you present? In what format? (e.g. tables, this year vs. last, etc.)


       Where will you obtain the data?



       Practice:
       Work with a partner to explain what you will use to make a presentation to your district, using
       the information you completed above.




August 2009                                         ETS                  2009 STAR Post Test Workshop    17
                                                                       Data CDs



       Data CDs
       What do they contain?




       How should you use them?




       Text File within Text Editor




       Data Layout




Locate Gender on the text file above.




August 2009                             ETS   2009 STAR Post Test Workshop   18
                                                          Individual Reports: Student Record Label
Individual Reports: Student Record Label
       Student Record Label: Grade 10 CST Sample




       Student Record Label: Grade 3 CST/CMA Sample




       Student Record Label: Grade 6 STS Sample




       Student Record Label: Grade 5 CAPA Sample




August 2009                                        ETS   2009 STAR Post Test Workshop         19
                                                 Individual Reports: Student Master List



Individual Reports: Student Master List
       Student Master List: Grade 3 Sample




       Student Master List: Grade 10 Sample




August 2009                           ETS     2009 STAR Post Test Workshop          20
                                                       Individual Reports: CST Student Report Sample



Individual Reports: CST Student Report Sample
       CST Student Report: Grade 10 Sample, page one




August 2009                          ETS                   2009 STAR Post Test Workshop         21
                                                       Individual Reports: CST Student Report Sample



       CST Student Report: Grade 10 Sample, page two




August 2009                          ETS                   2009 STAR Post Test Workshop         22
                                                       Early Assessment Program on Grade 11 Report



Early Assessment Program on Grade 11 Report
       CST Student Report: Grade 11 Sample, page two




August 2009                          ETS                   2009 STAR Post Test Workshop        23
                                                      Individual Reports: CMA Student Report Sample
                                                                                      New Page Two


Individual Reports: CMA Student Report Sample
New Page Two
       CMA Student Report: Grade 7 Sample, page two




August 2009                         ETS                   2009 STAR Post Test Workshop         24
                                  Individual Reports: CAPA Student Report Sample:
                                                                   New Page Two


Individual Reports: CAPA Student Report Sample:
New Page Two




August 2009           ETS               2009 STAR Post Test Workshop         25
                                   Individual Reports: STS Student Report Sample:
                                       New Page Two, Grades Five Through Seven


Individual Reports: STS Student Report Sample:
New Page Two, Grades Five Through Seven




August 2009            ETS             2009 STAR Post Test Workshop          26
                                                                     Teachers Explain Reports to Parents/Guardians



Teachers Explain Reports to Parents/Guardians
       Train site coordinators to train teachers in your district how to explain reports to parents and
       guardians by modeling how.



       Practice:
       Use the student report on pages 21 and 22.
       Select a subject.
       Work with a partner as if she/he is a parent. Explain the meaning of the scale score,
       performance level, and reporting clusters.




August 2009                                 ETS                          2009 STAR Post Test Workshop         27
                                        Unmatched Report: Grade 4 Example



Unmatched Report: Grade 4 Example




August 2009          ETS            2009 STAR Post Test Workshop      28
                                            Teacher Report



Teacher Report




August 2009      ETS   2009 STAR Post Test Workshop    29
                                         Teacher Report




August 2009   ETS   2009 STAR Post Test Workshop    30
                                                                             STAR Post Test Workshop Evaluation



STAR Post Test Workshop Evaluation
       Please answer the following questions and leave the evaluation before you exit the workshop.

       1. What is the most important thing you learned at this workshop?



       2. What could we do to improve the workshop?



       3. What questions do you have about the STAR Program that are still unanswered?



       4. What information from this workshop will you use the most in your district?



       5. Please rate the following on a scale of 1–5 (circle the number.)
                                                                                       Far Below
                                  Advanced Proficient     Basic    Below Basic           Basic
              Workshop overall       5           4          3            2                  1
              Presenter’s style      5           4          3            2                  1
              Video                  5           4          3            2                  1
              Handouts               5           4          3            2                  1
              Slides                 5           4          3            2                  1

       6. Do you prefer to have the Post-Test Workshops scheduled during the first week of August?
       ___ Yes
       ___ No
       If no, when would you suggest having them?




       Additional comments:



       7. Optional:



       Your Name                                           Your School District



August 2009                                ETS                         2009 STAR Post Test Workshop         31

								
To top