ISBE Illinois Alternate Assessment by qye36108


									Illinois Alternate Assessment

   Directors Conference
   July 31, 2008

Misc. Assessment Update

    Vacancy

    2% flexibility


IAA Agenda Overview

    Where we’ve been
    What we’ve learned
    Adjustments
    Scoring Validation
    Calendar
    Resources

IAA “Life Cycle” to Date:

  Fall 2006 Writing Pilot
  February 2007 Portfolios R, M, S
  Spring 2007 Writing Operational
  Fall 2007 Pilot R, M, S
  Spring 2008 Operational for R, M, S, W
  2009 adds Gr. 3 Writing to complete
   operational test

ISBE IAA 2008 Survey

    May, 2008, an electronic survey was
     sent out to teachers and coordinators
     using contact information from the
     Pearson Assessment Network and
     Scoring System.

    345 Teachers & Coordinators
     participated in the survey

Excellent Work, Coordinators!!

  92% of teachers knew in advance when
   the testing window would fall.
  94% of teachers reported the IAA
   coordinator had the online scoring set up
   and ready for scoring.
  95% of teachers said they knew who to
   contact if there was a problem.

Time required for administration:
    Subject     Less than 15    15-30
                  minutes      minutes
   11 Reading       28%          58%
     12 Math        31%         55%
    8 Science       37%         52%
    6 Writing       40%         46%

Concern #1:

 51% of teachers attended a training session

 This raises questions regarding the use of the
   trainer of trainers model. The accuracy of test
   administration and scoring may have been

Concern #2:

 Comparability to regular assessments in
  terms of content coverage.

 “In states I have worked with, there tend to
   be more alternate assessment items
   than (ISBE) has developed.” (peer

Response to the Two Concerns:
Potential for Refinement

2009 IAA Training, Part 1:
 More training sessions will be provided in 2009:

 Northern Illinois including Chicago
      January 26-30, 2009
 Central and Southern Illinois
      February 2-6, 2009

 Watch the ISBE website for signup
  opportunities in your area of the state in
  Fall, 2008.

2009 Training, (Possible) Part 2:

 To further ensure precise administration
  and scoring of the tasks, especially for
  those teachers who have not been

  Rubric language could be embedded in
   the teacher administration instructions.

Built-In Task Characteristics:
  Task-specific texts, symbols, and artwork
   embedded in the task.
  The “General Prompt” embedded in the task
   (in the event it is needed).
  All tasks would follow the same rubric
   “template” for teacher administration

    Flexibility in presentation of tasks and how the
     student responds would be maintained.

Sample Science Task:
     Primary Task:

  a dog and a cat    a cat and a plant   three bushes

 Which picture shows only mammals?

 General Prompt:

         Rabbits are mammals.
Sample Draft Teacher Instructions:
 Level 4:
 Show the student the primary task and say: “Which picture shows only mammals?” Present the answer
     options in order and say “a dog and a cat, a cat and a plant, three bushes.”

 The student receives a score of 4 for a correct response.
 If the student does not respond or responds incorrectly, you may repeat the question exactly as it appears
 one time only. If the student again chooses an incorrect answer or does not respond, move to Level 3.

 Level 3:
 Show the student the general prompt and say: “Rabbits are mammals.”
 Re-present the primary task and say: “Which picture shows only mammals?” Re-present the answer
     options in order and say “a dog and a cat, a cat and a plant, three bushes”.

 The student receives a score of 3 for a correct response. If no response or incorrect response, move to
     Level 2.

 Level 2:
  Indicate the specific prompt and say: “Cats and dogs are mammals.”
 Re-present the primary task and say: “Which picture shows only mammals?” Indicate the choices in order
     and say: “a dog and a cat, a cat and a plant, three bushes”.

 The student receives a score of 2 for a correct response. If no response or incorrect response, move to
     Level 1.

 Level 1:
 If no response or an incorrect response after providing level 2 support, the student receives a score of 1.

 Correct Answer: a dog and a cat                                                                               15
Build a Better IAA by:

    Extending
     content coverage
     requires increasing
     the quantity of items.

2008 IAA Operational Test

   Reading                 Math               Science    Writing

  9 tasks*            10 tasks*             6 tasks*    5 tasks*
  2 FT                2 FT                  2 FT        1 FT
  Total: 11           Total: 12             Total: 8    Total: 6

  * Required tasks covering identified priorities.

   FT – field test task (pilot) for future tests
Sample Content Coverage Plan:

 Each Standard assessed with:
 • At least 1 task for each objective
 • 2 tasks for each identified high priority
   objective (2 priority objectives at each
   subject’s grade level).
 • 4 objectives always assessed (tasks
   assessed annually as “anchor” items)

Using the previous content coverage plan,
the 2009 Operational Test would look like:

    Reading                 Math               Science          Writing

   17 tasks*           19 tasks*             19 tasks*         5 tasks
   4 FT                4 FT                  4 FT              1 FT
   Total: 21           Total: 23             Total: 23         Total: 6

   * Required tasks covering all objectives in each subject.

    FT – field test task (pilot) for future tests
Some Things Never Change
  Assessment frameworks remain the same.
  IAA Participation Guidelines remain the same.
  Use of mode of communication specific to the
   individual student remains the same.
  Options in types of student responses remains the
  Score submission online remains the same.
  Number of Writing tasks remains the same.
  Content coverage is expanded, not changed, in
   Reading, Math, and Science.
  Training will be provided on administration of the

Potential Results?

 Refinements should:

 •   Enhance content coverage.
 •   Improve likelihood that the IAA
        will pass peer review.
 •   Avoid rejection of the IAA
        by the federal agency.

August 4-5, 2008
 To enhance the quality and quantity of the IAA
   performance tasks:
  Item Writers will be working on

   Number of Writing tasks is sufficient at this

IAA Scoring Validation Has Not Changed

     Recruiting more Coordinators to serve
          Second scorers
          Super raters

     If interested, contact Pearl
      at 217/782-4823 or

IAA 2009 Proposed Calendar
 •   Jan. 26-30 & Feb. 2-6, 2009
     IAA training sessions throughout the state

 •   Feb. 17, 2009
     Tentative date for delivery of IAA test materials to

 •   February, 2009
     Online scoring system IDs & passwords distributed

    March 9-27, 2009 – Regular test window

IAA Online Resources

  Implementation Manual
  Online Scoring User Guides
  Frameworks
  Scoring Rubric
  RCDTS codes
  Accommodations Guidelines

Pearson Contact Information
    Pearson’s Assessment Network

    Pearson Customer Support
     800/627-7990 State Code 814

    Pearson Online Scoring Questions

ISBE Contact Information

    Pearl Schneider, IAA Coordinator
     Phone: 217/782-4823


To top