EDUCATION INFORMATION MANAGEMENT ADVISORY CONSORTIUM (EIMAC) SUBCOMMITTEE ON ASSESSMENT Washington Hilton Washington, DC October 17-18, 2006 US Department of Education Updates Meredith Miller Abigail Potts Kathleen Leos Meredith Miller, Abby Potts, and Kathleen Leos gave an update to the assessment subcommittee on important issues coming from the US Department of Education (ED) including the growth model pilot program, peer reviews, accountability plan changes, and the LEP Partnership. Growth Models Growth models kicked off the discussion with an update about the pilot program. States that originally submitted, but did not receive the flexibility had the opportunity to make revisions and resubmit their plans. These revised plans are currently being reviewed and final decisions should be made by mid-November. ED has laid out cross-cutting issues found in each of the proposals, which may be useful to states that may apply for future spots. Growth will not move out of the pilot phase until Congress reauthorizes ESEA, since the law would have to change for all states to be eligible. There is a possibility of expanding beyond ten states after all the slots are taken and upon further evaluation by the Secretary. Standards and Assessments Peer Review To date: five states have received full approval, six states have received approval with recommendations, three states approval expected, 27 states are approval pending, and one state is not approved. States have been allowed to ask for reconsideration of their status. Several states‟ categories have changed and some are still under review. States being re- reviewed will be assigned the same peers. Accountability Plan Amendments 2005-2006 A summary of amendments that were requested and approved last year: Minimum n changes. Eight of nine requests for increasing n size were rejected. Two states decreased n size. A state asked for ED‟s rationale in approving or rejecting n size requests. ED responded that each case is very different and there must be a statistical rationale for the change in size. Confidence Intervals. Five states requested changes with one approved. 2% interim flexibility. 24 of 26 states had approval for their request. Five states request interim option 2 (use of an assessment based on modified achievement standards) with four approved. Once the final regulation is released all assessments that are used for accountability purposes will be peer-reviewed. ED will provide guidance at the same time. Late AYP. Due to standard-setting activities for new assessments ten states were approved to make late AYP determinations. Index systems. Eight states are approved to use an index system for ‟05-‟06. Alaska, Florida, New Hampshire, Rhode Island, Vermont, Washington, Wisconsin, and Iowa. AP/IB substitute assessments. Two states (Virginia and Maryland) were approved to use AP and IB as subs for state high school assessments for ‟05-‟06 year only, while each state conducts studies to determine alignment to grade-level content and comparability of achievement. Title I regulation regarding LEP students This September, ED announced new Title I regulation that will help „recently arrived‟ LEP students learn English and other subjects while giving states and local school districts flexibility on assessment and accountability under NCLB. Expansion from 10 months to 12 for defining recently arrived LEP student. Exempt from one administration of state‟s reading/language arts assessment. Exempted recent arrived LEP from reading/language arts assessment – states must publicly report number of students exempted. States and LEAs must remain responsible for providing appropriate and adequate instruction for these students to gain English language skills and master content knowledge in reading/language arts and other subjects. „Former LEP students‟: new regulations address concern about credit for progress of LEP students in AYP determinations. In response, „former LEP‟ can be included in LEP category in making AYP determinations (for performance not participation) for up to 2 years after they no longer meet the state‟s definition for LEP. BUT „former LEP‟ cannot be included in LEP category for other reporting purposes (like achievement results). Flexibility is for the AYP part of the reporting – accountability, not assessment. NAEP Update Suzanne Triplett, ED There has been a large amount of interest from the Administration in how state assessments map to each other and to NAEP. Don McLaughlin had done a study mapping state achievement levels onto the NAEP scale. NCES has asked Henry Brown (ETS) to look at the study from a technical aspect and make suggestions. There was a strong feeling in the group that the heart of the study is not where states are in relation to NAEP, but that it allows you to directly compare states to each other. This study may become public in the next 3 to 6 months, but they still need to recalculate with 2005 data. NCES has offered to host a meeting in January for state testing directors to discuss this issue further. The subcommittee was in favor of this possibility along with inviting chiefs and PIOs to the meeting. Discussion of End-of-Course High School Exams Teri Siskind, facilitator The subcommittee had a facilitated discussion around the use of high school end-of- course exams. Specifically the group looked at Achieve‟s American Diploma Project for Algebra II and the adoption of ACT and SAT as mandatory state exams for students. CCSSO Initiatives on Testing Contractor Standards Don Long, CCSSO G. Harris, Association of Test Publishers Overview of collaborative: CCSSO and ATP in partnership with ED will work to develop and communicate “Best Practices” with respect to critical operational issues surrounding K-12 NCLB statewide and/or local assessments. It would be best if there were standards the entire industry is held to. Three preliminary meetings have been held between CCSSO & ATP (6/25/06; 7/17/06; 8/17/06) to discuss the structure and goals of the initiative. There will be an executive committee (2 chiefs, 2 state assessment directors, one CCSSO staff person plus representatives from ATP) along with task forces to focus on specific areas for improvement. The desired outcome for the testing enterprise is to have a voluntary agreement (i.e. “what to do” rather than “how to do”) with non-regulatory best practices. The subcommittee is very supportive of this initiative and looks forward to working with the testing committee. There were some suggestions to focus the work of the initiative by reducing the number of task forces and creating a shorter timeline for completion. A number of directors would like to see some deliverables at the Large-Scale Assessment Conference. Task forces proposed by the group include: contract management, scanning/scoring/reporting, and item development/test construction/technical defense. A number of people asked about the involvement of the federal government and suggested that NAEP could be a good resource given their use of testing contractors. Preliminary Recommendations from Assessment Subcommittee NAEP The Chiefs should be made aware that studies mapping state standards to the NAEP scale will be made public (probably within 3-6 months). These studies will not only allow for comparisons between state performance standards and NAEP performance standards, but also will facilitate comparison among states. The Assessment Subcommittee recommends that this issue be fully explored in a meeting to which all state Assessment directors are invited (January). State Chiefs and PIOs should be briefed subsequent to the State Assessment Director meeting. The Assessment Subcommittee has expressed concern about the validity of data collected through the assessment of twelfth grade students due to a number of factors including lack of motivation with respect to NAEP assessment. In recognition of the varying viewpoints on the appropriateness of assessing students in twelfth grade, the Assessment Subcommittee recommends that state NAEP at grade 12 remain voluntary at the state level as well as the school and student levels. The Chiefs should be made aware that state NAEP will include either writing or science in alternate cycles in addition to mathematics and reading. The process for vetting changes/actions/decisions about NAEP should be explicit, transparent, and open to comment. Socio-economic Status (SES) indicator surveys should be reviewed for appropriateness, clarity, sensitivity, and legality, and field-tested prior to full- scale implementation. The practice of identifying and notifying selected schools in the school year prior to the year of administrations should be continued and extended to all assessments. NCES should carefully consider the potential negative consequences of involving school staff in administration. Westat should prepare a script for advising schools as to their administration options; the script should be reviewed and approved by the NAEP State Coordinator prior to presentation to any school. Schools should be informed about the potential impact of accommodations on administration time. To ensure valid assessment, NAEP should consider a process for permitting a state to substitute a school (using the NAEP substitution mechanism) when the school is facing unusual circumstances (such as reconstitution under state or federal law). Research conducted with the states Kansas, Massachusetts, and New York resulting in products/tools relating to the alignment between state and NAEP frameworks and assessments should be distributed to all states. Test Publishers In addition to quality control metrics and requirements, testing contractor standards should address behavioral and ethical standards for test publishers to include sales and lobbying activities within a state. Chiefs should be made aware of inappropriate techniques employed by test publishers and others with regard to assessment. Such techniques include, but are not limited to, inaccurate characterization of content alignment and misleading interpretation of correlational studies. CCSSO should commission a publication to provide policy-makers a guide to assessment and accompanying materials to help evaluate publisher claims and to inform them about federal requirements and test development standards. The Chiefs should be made aware that the states and the Association of Test Publishers (ATP) through CCSSO will collaborate on the development of Testing Contractor Standards with a goal of completion by June 2007. Best practice documents should be developed to support the standards. NCLB Technical assistance should be provided in a timely fashion and address clearly stated needs and goals with a specific and on-going plan, consonant with best practice for professional development and technical assistance. Regulations should be promulgated with enough advance time to allow for implementation, recognizing that the test development cycle takes 2-3 years at minimum. The interim flexibility for students with disabilities (2%) should be extended into the 2006-07 school year (and possibly beyond per the previous two recommendations). Expectations regarding science assessments and clarifications about the use of end-of-course tests for science should be disseminated well in advance of the 2007-08 school year. Chiefs should be made aware of the implications from the September 13 regulations for the assessment of Limited English Proficient students: o Beginning in 2007-08, recent arrivals must be tested in science; o The number of recent arrivals exempted from reading language arts tests must be publicly reported for each school and district as well as the state; o Former LEP students may be included, for up to two years after they no longer meet the State definition for LEP, in the LEP category for AYP; o Performance of Former LEP students factors in after minimum N size is established (e.g, if the LEP category does not meet the minimum N before inclusion of former LEP students, LEP performance would not be reported); and o Former LEP students may not be included in the LEP subgroup for any other purpose. Chiefs should be made aware that ED is giving mixed messages about peer review for English language proficiency tests under Title III. The assessment of LEP students should be carefully considered in the reauthorization of ESEA in light of research about the acquisition of language for non-native speakers. Background factors and age should be considered. The approval process for accountability and assessment should be transparent and documented. Peer review decisions should be made on the basis of rule, law, and/or research.
Pages to are hidden for
"US Department of Education Updates"Please download to view full document