Assessment Recommendation by sdy12419

VIEWS: 8 PAGES: 24

More Info
									Assessment of English Language
          Learners
                    Jamal Abedi
           University of California, Davis

                Presented at:
  The Race to the Top Assessment Program
       Public & Expert Input Meeting
             December 2, 2009
              Denver, Colorado
Assessment results have major impact on ELL students’
   academic life more so than on non-ELL students


Classification (for ELLs)
Instruction (assessment before instruction for ELLs)
Accountability (multiple accountability requirements for ELLs)
Promotion (reclassification)
Graduation
      Assessment challenges for ELL
               students
•   ELL students go through assessment and
    accountability requirements and challenges twice:

•   Title III, the English language proficiency
    assessment (AMAO1 and AMAO 2)

•   Title I assessment in reading/language arts, math,
    and science

•   They are faced with the assessment issues in both
    areas
Focus of this presentation

 A. General assessment issues that apply to ELL
    students as well

 B. Assessment issue specific to ELLs
 C. Answers to the RTTT questions regarding ELL
    assessment and my recommendations
A.   General assessment issues that apply to ELL students
      (Already being discussed at the earlier RTTT presentations by Baker, Bennett, Braun, Darling

                                Hammond, Gong, Marion and others.)


     1.   Theory of Action

     2.   Link between assessment and instruction

     3.   Using multiple measures

     4.   Interim & formative assessments

     5.   Preparation of RFP

     6.   Use of technology

     7.   Common content standards

     8.   Growth measures over time

     9.   Providing teacher professional development opportunities

     (We will elaborate on some that are more related for ELLs)
2. Link between assessment and instruction (currently, a disconnect)
 •   As Darling-Hammond indicated, “..high-achieving nations used open-
     ended performance tasks and school-based, curriculum-embedded
     assessments to give students opportunities to develop and demonstrate
     higher-order thinking skills.”

 •   Instruction should inform development of assessment and assessments
     should inform instruction

 •   It is therefore essential to involve state assessment folks and teachers in
     the process of test item writing and test development

 •   Teachers should be trained and be involved in all different phases of
     test development process and use for ELL students including item
     writing, scoring, and interpreting the results

 •   The assessment process should help teacher in preparing students for
     college and career ready standards
              3. Using multiple measures
•   Once a year assessments in Title I and Title III, with all their
    limitations particularly for ELL students, may not produce
    valid and fair outcomes.

•   There are concerns over reliability and validity of these
    instruments for ELLs (e.g., for some content areas reliabilities
    are as low as .50)

•   These assessments include a substantial amount of
    measurement error which make high-stakes decisions based
    solely on their outcomes quite risky

•   A series of measures from different tests with different format
    and different tasks, given at different times, would be needed
    to make fair decisions about classification and academic
    performance of ELL students.
      4. Interim & formative assessments

•   Summative assessments while quite important for accountability
    purposes may be too little too late to inform curriculum and
    instructions

•   They are used mostly for accountability purposes without much
    instructional values

•   Interim and formative assessments provide teachers with the
    information needed to help ELL students

•   Outcomes of formative assessments may also help parents of ELL
    students to identify areas that they children need attention
                  6. Use of technology

•   As an example of the use of technology in assessment,
    computerized assessment system can be discussed
•   Computerized assessment system has the flexibility and
    capability of incorporating many accessibility features for
    assessments of ELL students
•   Examples are: English and bilingual glossary, read aloud of
    content-based assessment items and providing extra time
•   Assigning test items with different levels of linguistic
    complexities to students at different levels of language
    proficiency
•   Providing opportunities for students to be tested in a
    language that produces the most valid assessment outcomes
   7. Growth measures over time

 Growth measures are important for ELLs
 English language proficiency (Title III) lends itself
  well to growth over time
 An example of application of the concept of growth
  model is for the AMAO 2
 While it is extremely helpful to think about growth
  concept and its measurement model, it is also
  important to think about its limitations such as:
       issues concerning baseline
       changes in the measures overtime
       the nature of the content being assessed
       establishing a meaningful vertical scale
B. Assessment issues specific to ELLs

The misconception that ELL students are only quantitatively
    different from native speakers of English should be discussed
1.   Understanding of the two different assessment systems for
     ELLs, their similarities, differences and interactions
2.   Lack of interaction between the two systems, English language
     proficiency and content assessments
3.   Construct-irrelevant sources in measurement of ELL students
4.   Applicability of measurement theories in ELL assessments
5.   Impact of L1 proficiency on the assessments and instructions
     for ELL students
6.   Classification, reclassification, and misclassification of ELL
     students
                          Recommendations
1. Understanding of the two different assessment systems
  for ELLs, their similarities, differences and interactions

•   There are two different assessment systems for ELLs, Title III ELP
    and Title I content assessments

•   While they involve different tests based on different content
    standards and different objectives, they should work together.

•   We have made significant progress in the assessment of ELP for ELL
    students due to the implementation of NCLB, e.g., ACCESS for ELLs,
    ELDA, LAS LINK, SELP, etc

•   There is a need for substantial work on improving Title I
    assessments for ELLs

                            Recommendation:

Provide support for more improvement in the Title III assessments and
   support creating more valid assessment system in measuring ELL
   content knowledge (RTTT).
    2. Lack of interaction between ELP
           and content assessments
•   ELL students must be at a certain level of English proficiency to be
    able to meaningfully participate in the Title I assessment

•   Only students at the proficiency level 4 or above may be able to
    participate in Title 1 assessments.

•   However, there is a disconnect between student’s level of ELP and
    their participation in content-based assessments.

                            Recommendation:

Include ELL students in content assessments in English if they are at
   the proficiency level to meaningfully participate (level 4 or above);
   otherwise, provide valid alternatives such as native language
   testing, relevant accommodations, etc.
    3. Construct-irrelevant sources in measurement
                     of ELL students

•   There are different sources of construct irrelevant variance
    affecting ELL students assessment outcomes

•   Unnecessary linguistic complexity of assessment as a source of
    construct irrelevant variance adds an additional factor (dimension)
    to the assessments for ELLs

•   Other sources of construct irrelevant variance such as cultural
    biases also add additional dimensions to the assessment outcomes
    for ELL students

                           Recommendation:

Provide ELL professional training to the test item writers and include
   teachers and linguistic/cultural experts at the item development
   process to control for these sources.
    4. Applicability of measurement theories in ELL
                       assessments

•    A major assumption underlying classical theory of
     measurement is uni-dimensionality

•    Construct-irrelevant sources for ELL students introduce
     complexity into the measurement model for these students

•    While multidimensional assessment can be handled both in
     classical and IRT models, those dimension should be clearly
     relevant to the assessment

                         Recommendation

Revisit psychometric principles in light of current research on
  the assessment of ELLs and make appropriate revisions
    5. Impact of L1 proficiency on the assessments
           and instructions for ELL students

•   A major oversight in the assessment of ELL students is the lack of
    attention to their level of native language proficiency

•   Proficiency in L1 would help in both instruction and assessment of
    ELLs

•   Native language instruction and assessment could be a great
    success if students are academically proficient in their native
    language.

                            Recommendation

Include valid and comprehensive measures of ELL students’ level of L1
   academic proficiency in all proficiency domains (reading, writing,
   speaking and listening) and seriously consider the results in
   planning their curriculum and assessment
          6. Classification, reclassification, and
               misclassification of ELL students


•   If students are not properly classified as ELLs/non-ELLs,
    instructions, assessments and accommodations for ELL students will
    not be productive

•   Similarly, if ELL students are not properly re-classified as fluent in
    English when they reach fulency, they may not benefit from proper
    instruction and their assessment outcomes may not be valid

•   There are major concerns on misclassification of ELL students as
    those having learning disabilities.

                             Recommendation

•   Multiple reliable and valid criteria should inform decisions about
    classification/reclassification for ELL students.
    C. My answers to the RTTT questions regarding
       ELL assessment and my recommendations


•    Provide recommendations for the development and administration of
     assessments for each content area that are valid and reliable for English
     language learners
•    How do you recommend that assessments take into account the variations in English
     language proficiency of students?

•    How can technology be utilized to make assessments more inclusive to ELL
     students?

•    what are the relative merits of developing and administering content
     assessments in native language?

•    What are the technical, logistical, and financial requirements?
Question1a: Provide recommendations for the development
and administration of assessments for each content area
that are valid and reliable for English language learners

General recommendations:

•   ELL students should not be treated differently in the content being
    assessed; otherwise comparability of assessment outcomes may
    become a major concern

•   However, assessments in the content-based areas (such as math,
    science and social sciences) should be free of unnecessary linguistic
    complexities and cultural biases

•   Multiple measures should be implemented in both Title I and Title III
    that utilize different measurement approaches

•   Provide accommodations that help reduce the effects of construct-
    irrelevant factors
Question 1b: How would you recommend that the assessments take into
account the variations in English language proficiency of students in a
manner that enables them to demonstrate their knowledge and sills in core
academic areas?



   •   Provide assessments with varying degree of
       linguistic complexity

   •   Provide appropriate accommodations for ELL
       students to help them with their common needs
       of language assistance

   •   Provide native language testing opportunities for
       students who are academically proficient and are
       instructed in their native language
Question 1c: How can technology be utilized to make
assessments more inclusive to ELL students?


 •   Provide computerized assessments that are capable of
     offering effective and valid accessibility features including
     appropriate accommodations

 •   Provide web-based tutorials for ELL students for having
     additional opportunities to learn

 •   Provide web-based training for parents to help their students

 •   Provide diagnostic information to teachers, parents, and
     students on the areas need attention.
Question 2a: In the context of reflecting student achievement, what
are the relative merits of developing and administering content
assessments in native language?

•   If students are proficient in their native academic language and
    have been instructed in native language then native language
    assessments would be productive

•   Assessments in native language could be less affected by linguistic
    and cultural biases

      Language is no longer a source of construct-irrelevant variance
•   If students have not been instructed in their native language, then
    these assessments would not help in measuring student
    achievement
Question 2b: What are the technical, logistical, and financial
requirements?

  Logistical:

  •   Alignment between the language of assessments and language of
      instructions

  •   Issues addressing all of the languages spoken in our schools

  •   Issues concerning translations in different languages

  Financial:

  •   Issues with developing, field testing and preparing operational
      forms for all languages
For more information please contact

           Jamal Abedi
               at
         CRESST/UC Davis

          (530) 754-9150
                 or
        jabedi@ucdavis.edu

								
To top