Document Sample
PRACTICAL SOLUTIONS K-16 REFORM EFFORTS Stanley Rabinowitz_ Ph.D Powered By Docstoc
					Making Consistent Decisions About
      Accommodations for
  English Language Learners
        – Research Summit –

     Texas Comprehensive Center @ SEDL
               Austin, Texas
             March 16–17, 2009
Linguistic Accommodations on Assessments:
           Linguistic Modification

                 Edynn Sato, Ph.D.
         Director, Special Populations for AACC
    Content and Language
What is the relationship?

         Content                  Language

             Content        Language


       What is Linguistic Modification?

•   A theory- and research-based approach to
    clarifying and simplifying the language of
    the assessment without simplifying the
    content of the assessment.
•   Linguistic modification is not simply ―good
    editing‖ practice and does not result in
    ―easier‖ test items.

          What is Linguistic Modification?
•   Linguistic modification involves a process intended to
    increase English learners’ access to assessment content and
    subsequently increase the validity of the assessment results.
     –   Access is the minimization or removal of conditions (i.e., sources of
         construct-irrelevant variance, such as aspects of presentation/format of
         test information, aspects of response requirements, sociocultural contexts
         or references that may disadvantage certain students, etc.) that may
         interfere with students’ ability to meaningfully engage with content or
         demonstrate their content knowledge and skills.
     –   Appropriate access does not significantly change the targeted construct
     –   Providing students the opportunity to learn, exposing them to grade-level
         content, or applying broadly the principles of Universal Design do not
         necessarily address the particular access needs English learners
     –   Lack of appropriate access has consistently posed a threat to validity.

        Why Linguistic Modification?

•   Relative to other strategies such as portfolios
    and native language translations, linguistic
    modification lends itself more readily to
    standardization and broader application
    across language groups.

     Linguistic Modification: Key Points

•   Addresses the linguistic needs of ELLs
•   Increases the linguistic accessibility of the
    test item/task
•   Does not change the content/construct tested
•   Does not alter essential grade-level content-
    related language

       Assessment of English Learners:
           General Considerations
Critical and interdependent factors include:
•    Population
      –   Characteristics (diverse)
      –   Access needs (linguistic)
•    Content
      –   Language skill vs. content skill (how interrelated are the
          language and content skills)
      –   Construct
      –   Language demands
      –   Proficiency expectations
•    Context
      –   Assessment vs. instruction
      –   Design/Presentation

    Linguistic Modification: Considerations

•   Item context
•   Item graphics
•   Item vocabulary/wording
•   Item sentence structure
•   Item format
…and, the interactions among these dimensions

       Linguistic Modification: Strategies
Current Literature/research-based        Additional
Item context                             •Sometimes context is added to: remove
•Familiar to students; no cultural or    passive voice construction, past tense and
linguistic bias                          conditionals; break stem into shorter, less
                                         complex sentences (create a familiar story
                                         •Helps make abstract/highly-generalized
                                         language more concrete
Item graphics                            •Clarify: key/central aspects of
•Familiar to students; no cultural or    content/construct; what the student is
linguistic bias                          expected to do (purposeful, meaningful)
•Symbols, legends, key vocabulary are    •Represents the situation in its totality (not
relevant to the construct and familiar   misleading)
•Graphics support understanding          •Consistent labeling/naming

Excerpt from Sato (2008)

Linguistic Modification: Strategies (continued)
Current Literature/research-based      Additional

Item Vocabulary/Wording                •Consistent with standards/expectations
•High frequency words                  •―Common‖ and ―familiar‖ is not necessarily
•Common/familiar words                 appropriate—consider Academic English
•Relevant technical terms; technical   necessary
terms defined as appropriate           •Repeat key words—providing synonyms
•No ambiguous words or unnecessary     within the item may not be helpful
words with multiple meanings           •Avoid words that are both nouns and verbs
•No irregularly spelled words          (e.g., value, cost, carpet)
•Use of formal proper names are        •Avoid hyphenated and compound words
relevant and appropriate               •Formal proper names should be ones that
•Naming conventions are consistent     students are familiar reading
                                       •Relative pronouns (e.g., which) should have
                                       a referent
Excerpt from Sato (2008)

    Study of Linguistic Modification:
• 4,617 seventh and eighth grade students from
  13 schools in California
• ELs (Spanish), low reading ability non-ELs,
  high reading ability non-ELs
• Matched sets of original and linguistically
  modified versions of mathematics items
  (number sense, measurement)
• Random assignment of students

     Study of Linguistic Modification
Preliminary findings:
• Created a set of linguistically modified items that
  did not significantly alter the mathematics
  constructs assessed
   – expert judgment, cognitive interviews of EL and non-
     EL students, analysis of non-EL student performance
     on linguistically modified item and matched original
     items, analyses of non-EL student performance on
     statewide academic achievement tests and on the two
     versions of items developed for the study

     Study of Linguistic Modification
Preliminary findings:
• Positive effect, though not significant, of linguistic
  modification of test items on the mean scores of
  EL students and on the mean scores of non-EL
  students with low-reading ability

   Framework for understanding
    language demands on ELLs
Language Demands
• A language demand is categorized as either a
  Linguistic Skill or an Academic Language
  Function based on whether the demand is
  fundamental to the development and use of
  language or is a contextual application of

 From: Sato, Lagunoff, Worth, Bailey, and Butler (2005); Bailey, Butler, and
 Sato (2007)

Language Demands

                   From Sato (2008, June)

                   Language Complexity—Density

                       Low                                                          High
Length ranges from a word to paragraphs                       Length ranges from a word to paragraphs
 No/little variation in words and/or phrases in               Some variation in words and/or phrases in
sentences/paragraphs; consistent use of language              sentences/paragraphs
 Repetition of key words/phrases/sentences                    Repetition of key words/phrases/sentences
reinforces information                                        introduces new or extends information
 Language is used to present critical/central details         Language is used to present critical/central details,
 No/little abstraction; language reflects more               but non-essential detail also is presented
literal/concrete information; illustrative language is         Some abstraction; language may or may not be used
used; language is used to define/explain abstract             to define/explain abstract information; illustrative
information                                                   language may or may not be used; technical
 Graphics and/or relevant text features reinforce            words/phrases are used
critical information/details                                   Graphics and/or relevant text features may or may
                                                              not reinforce critical information/details

                                                                                                From Sato (2008, June)

              Language Complexity—Construction

                      Simple                                                     Complex
Mostly common/familiar words/phrases; no/few                    Some common/familiar words/phrases; some
uncommon words/phrases, compound words, gerunds,                uncommon words/phrases, compound words, gerunds,
figurative language, and/or idioms                              figurative language, and/or idioms
 Language is organized/structured                               Language may or may not be organized/structured
 Mostly simple sentence construction                            Varied sentence construction, including complex
 No/little passive voice                                       sentence construction
 Little variation in tense                                      Some passive voice
 Mostly one idea/detail per sentence                            Variation in tense
 Mostly familiar construction                                   Multiple ideas/details per sentence
              (e.g., ’s for possessive; s and es for plural)     Some less familiar/irregular construction
 Mostly familiar text features                                  Some less familiar text features
              (e.g., bulleted lists, bold face)                               (e.g., pronunciation keys, text boxes)

                                                                                                 From Sato (2008, June)

         Types of “Expertise” Needed

•   Knowledge of language development and
    language acquisition
•   Knowledge of English language learners
•   Knowledge of content (English language arts,
    mathematics, science, social studies)
•   Knowledge of (large-scale) assessment
•   Knowledge of instruction

•   Bailey, A. L., Butler, F. A., & Sato E. (2007). Standards-to-standards
    linkage under Title III: Exploring common language demands in ELD and
    science standards. Applied Measurement in Education, 20(1), 53–78.
•   Sato, E., Lagunoff, R., Worth, P., Bailey, A. L., & Butler, F. A. (2005). ELD
    standards linkage and test alignment under Title III: A pilot study of the CELDT
    and the California ELD and content standards (Final Report to the California
    Department of Education). San Francisco: WestEd.
•   Sato, E. (2008). Linguistic modification: Part II — A guide to linguistic
    modification: Increasing English language learner access to academic content.
    Washington, DC: LEP Partnership.

•   Sato, E. (2008, June). Language as Content and Language of Content. What's the
    Difference? — Defining the Language Construct: Strategies and Implications.
    Presentation at the annual conference of the Council of Chief State School
    Officers National Conference on Student Assessment. Orlando, FL.

For more information:
 Edynn Sato, Ph.D.
 Director, Special Populations
 Assessment and Accountability
  Comprehensive Center (AACC)

To learn more about the AACC: