# Ca Performance Test Workshop by hmb10625

VIEWS: 18 PAGES: 31

Ca Performance Test Workshop document sample

• pg 1
```									                                                                     STAR Post-Test Workshop 2009

STAR Post-Test Workshop 2009

Objectives
District STAR coordinators will be able to:
• Describe the purposes of STAR reports
• Interpret STAR results
• Explain key statistics
• Compare and contrast types of reports
• Identify proper uses of reports

Agenda
Registration and take quiz
Introduction and What’s New
Results and Statistical Analysis
Using Results
Summary and Internet Reports
Data CDs
Individual Student Reports
Teacher Reports
Early Assessment Program (EAP)

2009 Post-Test Guides will be delivered to
districts after August 17. They may be

August 2009                                 ETS               2009 STAR Post-Test Workshop     1
What’s New for 2009?

What’s New for 2009?
Excerpted from the 2009 STAR Post-Test Guide:

Table I.1 What’s New in 2009
Change                                         Affected Report(s)
Reported Test Results
The default mathematics test for students in grade seven who      -   STAR Student Report for CST
took a California Mathematics Standards Test is the California    -   STAR Student Record Label
Standards Test (CST) for Grade 7 Mathematics. Students who        -   Subgroup Summary reports
did not mark a CST for mathematics on their answer document       -   All grade-level reports on which
presented, such as the Student
Master List
California Modified Assessment (CMA) performance levels—          -   STAR Student Report for CMA
advanced, proficient, basic, below basic, and far below basic—    -   STAR Student Record Label
are reported for students in grades three through five who took   -   Subgroup Summary reports
a CMA. English–language arts (ELA) scores for students in         -   All grade-level reports on which
grade four include the results for the California Writing             results for grades three, four,
Modified Assessment if the student took this component of the         and five are presented, such as
ELA test.                                                             the Student Master List
CMA results are reported for students in grades six through       -   STAR Student Report for CMA
eight who took a CMA. Performance levels and scale scores         -   STAR Student Record Label
are not reported this year; only percent-correct scores by        -   Subgroup Summary reports
content area are reported. Students in grade seven who took the   -   All grade-level reports on which
California Writing Modified Assessment component of the               results for grades six, seven, and
CMA for ELA test will also receive results for the writing            eight are presented, such as the
component.                                                            Student Master List
Performance levels for ELA and mathematics for all levels of      -   STAR Student Report for CAPA
the California Alternate Performance Assessment (CAPA)            -   STAR Student Record Label
have been recalibrated.                                           -   Subgroup Summary reports
-   All grade-level reports, such as
the Student Master List
A performance level is reported for the CAPA for Science test.    -   STAR Student Report for CAPA
-   STAR Student Record Label
-   Subgroup Summary reports
-   All grade-level reports on which
results for grades five, eight, and
ten are presented, such as the
Student Master List

August 2009                                ETS                           2009 STAR Post-Test Workshop         2
What’s New for 2009?

Change                                        Affected Report(s)
Because the California Achievement Test/Sixth Edition Survey - STAR Student Report for CST
(CAT/6 Survey) was not offered in 2009, CAT/6 Survey data       - STAR Student Report for CMA
are not reported.                                               - STAR Student Record Label
- STAR Group Summary
- STAR Group Summary: CAT/6
Survey
- Subgroup Summary reports
- All grade-level reports on which
seven are presented, such as the
Student Master List
Standards-based Tests in Spanish (STS) performance levels— - STAR Student Report for STS
advanced, proficient, basic, below basic, and far below basic— - STAR Student Record Label
are reported for students in grades two through four who took   - Subgroup Summary reports
an STS.                                                         - All grade-level reports on which
results for grades two, three, and
four are presented, such as the
Student Master List
STS results are reported for students in grades five through    - STAR Student Report for STS
seven who took an STS. Performance levels and scale scores      - STAR Student Record Label
are not reported this year; only percent-correct scores by      - Subgroup Summary reports
content area and reporting cluster are reported for             - All grade-level reports on which
tests.                                                            seven are presented, such as the
Student Master List
STS results are reported for students in grades eight through   - STAR Student Report for STS
eleven who took an STS, including the new end-of-course         - STAR Student Record Label
tests, the STS for Algebra I (grades seven through eleven) and - Subgroup Summary reports
the STS for Geometry (grades eight through eleven).             - All grade-level reports on which
Performance levels and scale scores are not reported this year;   results for grades eight through
only percent-correct scores by content area are reported.         eleven are presented, such as the
Student Master List
Group and Subgroup Reporting
Because the CAT/6 Survey was not offered in 2009, CAT/6          - STAR Group Summary: CAT/6
Survey data are not reported.                                      Survey
Accommodations and Modifications
For students in grades four and seven who took a CST for         - STAR Student Report for CST
Writing, when marked, option “P” (Student used an audio CD       - STAR Student Record Label
presentation) and option “Z” (text examiner read aloud text in   - All grade-level reports on which
the Writing Prompt and Response Booklet) were                      results for grades four and seven
accommodations, not modifications.                                 are presented, such as the
Student Master List

August 2009                                ETS                        2009 STAR Post-Test Workshop        3
Report Results

Report Results
Q. What are purposes of STAR Reports?

Excerpted from the 2009 STAR Post-Test Guide:
Chapter I.2 Introduction
Purpose of the Reports and Using the Results
The results for tests within the Standardized Testing and Reporting (STAR) Program are used for three
primary purposes:
1. Communicating students’ progress in attaining proficiency on the state’s academic standards to
students, parents/guardians, and teachers. In developing the legislation for the STAR Program, the
Legislature recognized that school districts will conduct their own ongoing diagnostic assessments
and provide information on the results of these assessments to parents/guardians and teachers on a
regular basis. The Legislature also recognized that local diagnostic assessment is the primary way in
which to identify academic strengths and weaknesses (Education Code Section 60602).
2. Informing decisions, along with local assessment data, that teachers and administrators make about
helping students improve their achievement and about improving the educational program.
3. Providing data for state and federal accountability programs. These data are used to calculate each
school and school district’s Academic Performance Index (API) and adequate yearly progress (AYP)
to meet the requirements of the No Child Left Behind Act of 2001.

Q. What is a scale score?

August 2009                                  ETS                            2009 STAR Post-Test Workshop      4
Report Results: Scale Scores

Report Results: Scale Scores
Excerpted from the 2009 STAR Post-Test Guide:

Chapter I.3 Interpreting Reports
Equating and Scaling
A scale score is derived
When tests are constructed for each grade, every effort is made to make
from a statistical process.
the tests parallel and of the same level of difficulty from one year to another.     It is not possible to cal-
However, even with those efforts, small differences in test difficulty still         culate a scale score by
exist between test forms. A psychometric procedure called equating makes             multiplying a student’s
adjustments for test difficulty so that students in one year are held to the         percent correct in a con-
same standards as students in another year.                                          tent area by 600.
…
Scale Scores for the STAR Program
The advantage of the scale score metric is that it allows a particular score (for example, 350) to mean the
same thing across test forms, even though the difficulty of the test forms may vary. An equating process
that adjusts for the difficulty of the test form permits this. Because percent-correct scores are defined in
terms of the number of items answered correctly (the raw score metric) they are, by definition, associated
with the specific form of the test taken, unadjusted for difficulty—that is, they are dependent on the
difficulty of the test items and the ability level of those who are taking the test.
Scale scores are used in the evaluation of overall student performance. Unlike raw scores (that is,
number-correct scores or percent-correct scores) within the same grade and subject, scale scores provide a
common reference across years, making interpretation easier. The scale score performance-level cut points
are held constant from year to year for each grade level and content area, while the number- or percent-
correct score associated with each scale score may change.

August 2009                                     ETS                             2009 STAR Post-Test Workshop         5
Reporting Clusters or Content Areas

Reporting Clusters or Content Areas
Examples
Mathematics (from Student Master List Summary, End-of-Course report)

English–Language Arts (from a STAR CST Student Report)

Q. Why should you NOT compare clusters from year to year?

August 2009                                              ETS                  2009 STAR Post-Test Workshop         6
Results: Clusters

Results: Clusters
Excerpted from the 2009 STAR Post-Test Guide:
Chapter I.3 Interpreting Reports
Interpreting Results
Because cluster scores are constructed from test questions of like content, the test questions may be
easier or more difficult as a group than the overall test form. Thus, percent-correct values based on the
cluster scores may even differ from the percent correct obtained for the total test. Because of this and the
fact that unadjusted raw scores are used to compute the percent-correct values, the cluster percent-correct
scores do not behave in the same way as do the scale scores and cannot be used to calculate the scale
scores.
Reporting clusters and the number of items that comprise each are provided in Appendix A. CST data
start on page 129; CMA cluster data start on page 137; STS cluster data start on page 139. There are no
reporting clusters for the CAPA. Reporting clusters can help teachers and instructional leaders pinpoint
areas of student strengths and weaknesses. However, reporting clusters should be interpreted cautiously,
and two very important limitations of CST, CMA, and STS reporting clusters should always be kept in
mind:
1. Reporting clusters are based on different numbers of questions. In some cases, the total number of
questions that make up a reporting cluster may be quite small; the small number results in scores
that are less reliable than the overall test scores.
2. Reporting-cluster scores may vary from year to year because the difficulty of the questions in the
reporting clusters may vary. While the overall test scores are equated to adjust for differences in
difficulty from year to year, that is not done for the reporting clusters.

August 2009                                    ETS                            2009 STAR Post-Test Workshop         7
Reporting Clusters: Post-Test Guide Appendix A Sample

Reporting Clusters: Post-Test Guide Appendix A
Sample
Data for the “Avg & Correct Statewide” columns will be available in the printed Post-Test
Guides and in the Post-Test Guide posted to startest.org after August 17, 2009.

CALIFORNIA HISTORY–SOCIAL SCIENCE STANDARDS TESTS
Number of Questions for 2009 Reporting Clusters and the
Average Percent Correct on Each Reporting Cluster
Avg % Correct Statewide
Test                     Reporting Cluster                    Questions         All      Proficient    Advanced
World History and Geography: Ancient Civilizations        16
Late Antiquity and the Middle Ages                        14
Renaissance/Reformation                                   10
U.S. Constitution and the Early Republic                  22
Civil War and Its Aftermath                               13
Foundations of American Political and Social Thought      10
Industrialization and the U.S. Role as a World Power      13
United States Between the World Wars                      12
World War II and Foreign Affairs                          12
Post-World War II Domestic Issues                         13
END-OF-COURSE HISTORY–SOCIAL SCIENCE
World History
Development of Modern Political Thought                    13
Industrial Expansion and Imperialism                       10
Causes and Effects of the First World War                  14
Causes and Effects of the Second World War                 13
International Developments in the Post-World War II
10
Era

August 2009                                    ETS                            2009 STAR Post-Test Workshop          8
Using Results: What Can and Cannot Be Compared?

Using Results: What Can and Cannot Be Compared?
Do compare:
•
•
•

Do NOT compare:
•
•
•
Excerpted from the 2009 STAR Post-Test Guide:
Chapter I.4 Comparing Results
Comparing CST Results
When comparing results for the CSTs, compare results only within the same content area and grade; that
six mathematics in 2009. No direct comparisons should be made between grades or between content areas.
In addition, comparisons should be made only within the same testing program. Results for the CST for
Grade 3 ELA cannot be compared to results for the CMA for Grade 3 ELA, for example.
For the CSTs, the reviewer may compare results for the same grade and content area across years within
a school, between schools, or between a school and its district, its county, or the state. When comparing
results for the CMA for grades three through five and for the STS for grades two through four, compare
results only within the same content area and grade for this year only, because 2009 is the first year
performance-level data are available. No direct comparisons should be made between grades or between
content areas. When making comparisons, the reviewer should consider comparing the percent of students
scoring proficient and advanced. This is because the state target is for all students to score at or above
proficient.
Two types of comparisons are possible:
1. Comparing the average scale score; or
2. Comparing the percent of students scoring at each performance level.
Comparisons may also be made by calculating the overall percent of students within a school who scored
proficient and advanced and comparing that percent to the overall percent of students in another school, the
district, the county, or the state who scored proficient or advanced. The CST Summary Report, which can
be found at http://star.cde.ca.gov/, provides this information for CST results for a selected reporting
When making comparisons across years within a given grade and content area, it is important to
understand that even when the number of students is the same, the group’s composition from year to year
may be quite different if student mobility (transiency) is high.
When comparisons are made across years, they are actually a comparison of different groups of students
with different traits taking different tests. Generally, there will be more variance in scores from year to year
when small numbers of students are tested.
While there may be a valid comparison to be made between students within a grade and content area, it is
not valid to subtract a student’s or class’s scale score received one year in a given content area from the
scale score received the previous year in the same content area in order to show growth. While the scale
scores may look the same, they are independently scaled so that differences for the same students across
years cannot be calculated using basic subtraction.
Any comparison of groups between years should not be used for diagnostic, placement, or
promotion or retention purposes. Decisions about promotion, retention, placement, or eligibility for
special programs may use or include STAR Program results only in conjunction with multiple other
measures including, but not limited to, locally administered tests, teacher recommendations, and

August 2009                                    ETS                              2009 STAR Post-Test Workshop         9
Summary Reports: Student Master List Summary, Grade 7

Summary Reports: Student Master List Summary, Grade 7

August 2009           ETS                                     2009 STAR Post-Test Workshop         10
Summary Reports: Student Master List Summary:
CST Biology End-of-Course Sample

Summary Reports: Student Master List Summary:
CST Biology End-of-Course Sample

August 2009                       ETS                2009 STAR Post Test Workshop         11
Summary Reports: Subgroup Summary:

Summary Reports: Subgroup Summary:

August 2009                         ETS    2009 STAR Post Test Workshop         12
Internet Reports

Internet Reports
Select a report (“Test Results”)
1. Select a test from the dropdown menu.
You can see reports for the entire state by not selecting County, District, School.
2. If needed, select County, District, and School from dropdown menus. You must select a
county to access district reports. You must select a district to access school reports.
3. If needed, select a demographic group from the dropdown menu.
4. If needed, select a demographic subgroup from the dropdown menu.
5. Select View Report.

Internet Reports: CST Sample

August 2009                                       ETS                  2009 STAR Post Test Workshop     13
Internet Reports

Internet Reports: CST Summary (only available for CST. Select Test dropdown.)

Internet Reports: CMA Sample

August 2009                                  ETS               2009 STAR Post Test Workshop    14
Internet Reports

Internet Reports: STS Sample

August 2009                           ETS   2009 STAR Post Test Workshop    15
Internet Reports

Internet Reports: CAPA Samples
State

August 2009                              ETS   2009 STAR Post Test Workshop    16

To whom/which groups will you make presentations?

What data will you present? In what format? (e.g. tables, this year vs. last, etc.)

Where will you obtain the data?

Practice:
Work with a partner to explain what you will use to make a presentation to your district, using
the information you completed above.

August 2009                                         ETS                  2009 STAR Post Test Workshop    17
Data CDs

Data CDs
What do they contain?

How should you use them?

Text File within Text Editor

Data Layout

Locate Gender on the text file above.

August 2009                             ETS   2009 STAR Post Test Workshop   18
Individual Reports: Student Record Label
Individual Reports: Student Record Label
Student Record Label: Grade 10 CST Sample

Student Record Label: Grade 3 CST/CMA Sample

Student Record Label: Grade 6 STS Sample

Student Record Label: Grade 5 CAPA Sample

August 2009                                        ETS   2009 STAR Post Test Workshop         19
Individual Reports: Student Master List

Individual Reports: Student Master List
Student Master List: Grade 3 Sample

Student Master List: Grade 10 Sample

August 2009                           ETS     2009 STAR Post Test Workshop          20
Individual Reports: CST Student Report Sample

Individual Reports: CST Student Report Sample
CST Student Report: Grade 10 Sample, page one

August 2009                          ETS                   2009 STAR Post Test Workshop         21
Individual Reports: CST Student Report Sample

CST Student Report: Grade 10 Sample, page two

August 2009                          ETS                   2009 STAR Post Test Workshop         22
Early Assessment Program on Grade 11 Report

Early Assessment Program on Grade 11 Report
CST Student Report: Grade 11 Sample, page two

August 2009                          ETS                   2009 STAR Post Test Workshop        23
Individual Reports: CMA Student Report Sample
New Page Two

Individual Reports: CMA Student Report Sample
New Page Two
CMA Student Report: Grade 7 Sample, page two

August 2009                         ETS                   2009 STAR Post Test Workshop         24
Individual Reports: CAPA Student Report Sample:
New Page Two

Individual Reports: CAPA Student Report Sample:
New Page Two

August 2009           ETS               2009 STAR Post Test Workshop         25
Individual Reports: STS Student Report Sample:
New Page Two, Grades Five Through Seven

Individual Reports: STS Student Report Sample:
New Page Two, Grades Five Through Seven

August 2009            ETS             2009 STAR Post Test Workshop          26
Teachers Explain Reports to Parents/Guardians

Teachers Explain Reports to Parents/Guardians
Train site coordinators to train teachers in your district how to explain reports to parents and
guardians by modeling how.

Practice:
Use the student report on pages 21 and 22.
Select a subject.
Work with a partner as if she/he is a parent. Explain the meaning of the scale score,
performance level, and reporting clusters.

August 2009                                 ETS                          2009 STAR Post Test Workshop         27

August 2009          ETS            2009 STAR Post Test Workshop      28
Teacher Report

Teacher Report

August 2009      ETS   2009 STAR Post Test Workshop    29
Teacher Report

August 2009   ETS   2009 STAR Post Test Workshop    30
STAR Post Test Workshop Evaluation

STAR Post Test Workshop Evaluation
Please answer the following questions and leave the evaluation before you exit the workshop.

1. What is the most important thing you learned at this workshop?

2. What could we do to improve the workshop?

3. What questions do you have about the STAR Program that are still unanswered?

4. What information from this workshop will you use the most in your district?

5. Please rate the following on a scale of 1–5 (circle the number.)
Far Below
Advanced Proficient     Basic    Below Basic           Basic
Workshop overall       5           4          3            2                  1
Presenter’s style      5           4          3            2                  1
Video                  5           4          3            2                  1
Handouts               5           4          3            2                  1
Slides                 5           4          3            2                  1

6. Do you prefer to have the Post-Test Workshops scheduled during the first week of August?
___ Yes
___ No
If no, when would you suggest having them?

7. Optional: