US Department of Education Updates by rdp21471


									                                               EDUCATION INFORMATION
                                      MANAGEMENT ADVISORY CONSORTIUM
                                          SUBCOMMITTEE ON ASSESSMENT
                                                                            Washington Hilton
                                                                              Washington, DC
                                                                           October 17-18, 2006

US Department of Education Updates
Meredith Miller
Abigail Potts
Kathleen Leos

Meredith Miller, Abby Potts, and Kathleen Leos gave an update to the assessment
subcommittee on important issues coming from the US Department of Education (ED)
including the growth model pilot program, peer reviews, accountability plan changes, and
the LEP Partnership.

Growth Models
Growth models kicked off the discussion with an update about the pilot program. States
that originally submitted, but did not receive the flexibility had the opportunity to make
revisions and resubmit their plans. These revised plans are currently being reviewed and
final decisions should be made by mid-November. ED has laid out cross-cutting issues
found in each of the proposals, which may be useful to states that may apply for future

Growth will not move out of the pilot phase until Congress reauthorizes ESEA, since the
law would have to change for all states to be eligible. There is a possibility of expanding
beyond ten states after all the slots are taken and upon further evaluation by the Secretary.

Standards and Assessments Peer Review
To date: five states have received full approval, six states have received approval with
recommendations, three states approval expected, 27 states are approval pending, and one
state is not approved. States have been allowed to ask for reconsideration of their status.
Several states‟ categories have changed and some are still under review. States being re-
reviewed will be assigned the same peers.

Accountability Plan Amendments 2005-2006
A summary of amendments that were requested and approved last year:
    Minimum n changes. Eight of nine requests for increasing n size were rejected.
      Two states decreased n size. A state asked for ED‟s rationale in approving or
      rejecting n size requests. ED responded that each case is very different and there
      must be a statistical rationale for the change in size.
    Confidence Intervals. Five states requested changes with one approved.
      2% interim flexibility. 24 of 26 states had approval for their request. Five states
       request interim option 2 (use of an assessment based on modified achievement
       standards) with four approved. Once the final regulation is released all
       assessments that are used for accountability purposes will be peer-reviewed. ED
       will provide guidance at the same time.
      Late AYP. Due to standard-setting activities for new assessments ten states were
       approved to make late AYP determinations.
      Index systems. Eight states are approved to use an index system for ‟05-‟06.
       Alaska, Florida, New Hampshire, Rhode Island, Vermont, Washington,
       Wisconsin, and Iowa.
      AP/IB substitute assessments. Two states (Virginia and Maryland) were approved
       to use AP and IB as subs for state high school assessments for ‟05-‟06 year only,
       while each state conducts studies to determine alignment to grade-level content
       and comparability of achievement.

Title I regulation regarding LEP students
This September, ED announced new Title I regulation that will help „recently arrived‟
LEP students learn English and other subjects while giving states and local school
districts flexibility on assessment and accountability under NCLB.
     Expansion from 10 months to 12 for defining recently arrived LEP student.
         Exempt from one administration of state‟s reading/language arts assessment.
     Exempted recent arrived LEP from reading/language arts assessment – states must
         publicly report number of students exempted.
     States and LEAs must remain responsible for providing appropriate and adequate
         instruction for these students to gain English language skills and master content
         knowledge in reading/language arts and other subjects.
„Former LEP students‟: new regulations address concern about credit for progress of LEP
students in AYP determinations. In response, „former LEP‟ can be included in LEP
category in making AYP determinations (for performance not participation) for up to 2
years after they no longer meet the state‟s definition for LEP. BUT „former LEP‟ cannot
be included in LEP category for other reporting purposes (like achievement results).
Flexibility is for the AYP part of the reporting – accountability, not assessment.

NAEP Update
Suzanne Triplett, ED

There has been a large amount of interest from the Administration in how state
assessments map to each other and to NAEP. Don McLaughlin had done a study mapping
state achievement levels onto the NAEP scale. NCES has asked Henry Brown (ETS) to
look at the study from a technical aspect and make suggestions. There was a strong
feeling in the group that the heart of the study is not where states are in relation to NAEP,
but that it allows you to directly compare states to each other. This study may become
public in the next 3 to 6 months, but they still need to recalculate with 2005 data.
NCES has offered to host a meeting in January for state testing directors to discuss this
issue further. The subcommittee was in favor of this possibility along with inviting chiefs
and PIOs to the meeting.

Discussion of End-of-Course High School Exams
Teri Siskind, facilitator

The subcommittee had a facilitated discussion around the use of high school end-of-
course exams. Specifically the group looked at Achieve‟s American Diploma Project for
Algebra II and the adoption of ACT and SAT as mandatory state exams for students.

CCSSO Initiatives on Testing Contractor Standards
Don Long, CCSSO
G. Harris, Association of Test Publishers

Overview of collaborative: CCSSO and ATP in partnership with ED will work to develop
and communicate “Best Practices” with respect to critical operational issues surrounding
K-12 NCLB statewide and/or local assessments. It would be best if there were standards
the entire industry is held to.

Three preliminary meetings have been held between CCSSO & ATP (6/25/06; 7/17/06;
8/17/06) to discuss the structure and goals of the initiative. There will be an executive
committee (2 chiefs, 2 state assessment directors, one CCSSO staff person plus
representatives from ATP) along with task forces to focus on specific areas for
improvement. The desired outcome for the testing enterprise is to have a voluntary
agreement (i.e. “what to do” rather than “how to do”) with non-regulatory best practices.

The subcommittee is very supportive of this initiative and looks forward to working with
the testing committee. There were some suggestions to focus the work of the initiative by
reducing the number of task forces and creating a shorter timeline for completion. A
number of directors would like to see some deliverables at the Large-Scale Assessment
Conference. Task forces proposed by the group include: contract management,
scanning/scoring/reporting, and item development/test construction/technical defense. A
number of people asked about the involvement of the federal government and suggested
that NAEP could be a good resource given their use of testing contractors.

Preliminary Recommendations from Assessment Subcommittee

   The Chiefs should be made aware that studies mapping state standards to the
     NAEP scale will be made public (probably within 3-6 months). These studies
     will not only allow for comparisons between state performance standards and
     NAEP performance standards, but also will facilitate comparison among states.
     The Assessment Subcommittee recommends that this issue be fully explored in a
     meeting to which all state Assessment directors are invited (January). State
       Chiefs and PIOs should be briefed subsequent to the State Assessment Director
      The Assessment Subcommittee has expressed concern about the validity of data
       collected through the assessment of twelfth grade students due to a number of
       factors including lack of motivation with respect to NAEP assessment. In
       recognition of the varying viewpoints on the appropriateness of assessing students
       in twelfth grade, the Assessment Subcommittee recommends that state NAEP at
       grade 12 remain voluntary at the state level as well as the school and student
      The Chiefs should be made aware that state NAEP will include either writing or
       science in alternate cycles in addition to mathematics and reading.
      The process for vetting changes/actions/decisions about NAEP should be explicit,
       transparent, and open to comment.
      Socio-economic Status (SES) indicator surveys should be reviewed for
       appropriateness, clarity, sensitivity, and legality, and field-tested prior to full-
       scale implementation.
      The practice of identifying and notifying selected schools in the school year prior
       to the year of administrations should be continued and extended to all
      NCES should carefully consider the potential negative consequences of involving
       school staff in administration. Westat should prepare a script for advising schools
       as to their administration options; the script should be reviewed and approved by
       the NAEP State Coordinator prior to presentation to any school. Schools should
       be informed about the potential impact of accommodations on administration
      To ensure valid assessment, NAEP should consider a process for permitting a
       state to substitute a school (using the NAEP substitution mechanism) when the
       school is facing unusual circumstances (such as reconstitution under state or
       federal law).
      Research conducted with the states Kansas, Massachusetts, and New York
       resulting in products/tools relating to the alignment between state and NAEP
       frameworks and assessments should be distributed to all states.

Test Publishers
    In addition to quality control metrics and requirements, testing contractor
       standards should address behavioral and ethical standards for test publishers to
       include sales and lobbying activities within a state.
    Chiefs should be made aware of inappropriate techniques employed by test
       publishers and others with regard to assessment. Such techniques include, but are
       not limited to, inaccurate characterization of content alignment and misleading
       interpretation of correlational studies.
    CCSSO should commission a publication to provide policy-makers a guide to
       assessment and accompanying materials to help evaluate publisher claims and to
       inform them about federal requirements and test development standards.
    The Chiefs should be made aware that the states and the Association of Test
       Publishers (ATP) through CCSSO will collaborate on the development of Testing
       Contractor Standards with a goal of completion by June 2007. Best practice
       documents should be developed to support the standards.

   Technical assistance should be provided in a timely fashion and address clearly
     stated needs and goals with a specific and on-going plan, consonant with best
     practice for professional development and technical assistance.
   Regulations should be promulgated with enough advance time to allow for
     implementation, recognizing that the test development cycle takes 2-3 years at
   The interim flexibility for students with disabilities (2%) should be extended into
     the 2006-07 school year (and possibly beyond per the previous two
   Expectations regarding science assessments and clarifications about the use of
     end-of-course tests for science should be disseminated well in advance of the
     2007-08 school year.
   Chiefs should be made aware of the implications from the September 13
     regulations for the assessment of Limited English Proficient students:
         o Beginning in 2007-08, recent arrivals must be tested in science;
         o The number of recent arrivals exempted from reading language arts tests
             must be publicly reported for each school and district as well as the state;
         o Former LEP students may be included, for up to two years after they no
             longer meet the State definition for LEP, in the LEP category for AYP;
         o Performance of Former LEP students factors in after minimum N size is
             established (e.g, if the LEP category does not meet the minimum N before
             inclusion of former LEP students, LEP performance would not be
             reported); and
         o Former LEP students may not be included in the LEP subgroup for any
             other purpose.
   Chiefs should be made aware that ED is giving mixed messages about peer
     review for English language proficiency tests under Title III.
   The assessment of LEP students should be carefully considered in the
     reauthorization of ESEA in light of research about the acquisition of language for
     non-native speakers. Background factors and age should be considered.
   The approval process for accountability and assessment should be transparent and
   Peer review decisions should be made on the basis of rule, law, and/or research.

To top