PowerPoint Presentation by qiF57a


									       Part 5 Test Adaptations
   1. Conceptualizations
   2. Example
      What is the central issue in
   Producing instruments that measure
    target constructs adequately in
    target cultures
        A Note on Terminology
   Translation
    • Conventional term, still often used
   Adaptation
    • Has become generic term for modern
      translation practices
    • Based on increased sensitivity for non-
      linguistic factors in translations, such as
      cultural norms of address, relevance of
      thorough knowledge of target culture
        Main Applications of
   Comparative Studies
    • Comparison of construct or mean scores
      across cultures
    • High demands on comparability of
         Maximizing comparability
   Monocultural studies in target culture
    • Main issue is ensuring validity in new
    • Few demands on comparability scores
    Translations in Historical Perspective
   Stage 1:
    • Close translations were standard practice
    • Techniques were developed (e.g.,
      translation back translation)
   Important societal developments:
    • Globalization and migration (multi-ethnic
   Stage 2:
    • Increasing appreciation that close
      translations have problems, e.g., Grade
      12 = Form 6 = ……?
    • Need for adaptations, localizations
    • Need for standardization of adaptation
     What is a Good Translation/
   Dependent on perspective
    • Linguistic perspective
    • Psychological perspective
   Mapping problem:
    • Translating/adapting can be seen as finding an
      optimal mapping of text in two languages
   What is a good mapping?
    • A good mapping shows equivalence of the
      original and translation
   What is the American equivalent of
    the Dutch item “Hoe heet de koningin
    van Nederland?” (Suppose that item
    is part of a test of crystallized
   Literal/close translation: What is the
    name of the queen of the
    • Problem: Item more difficult for American
      children than for Dutch children
   Adaptation: “What is the name of the
    president of the USA?”
    • Problem: Queen and president are not
      equally known in their respective
What Does “Equivalent” Mean?
   Eusebius Hieronymus (St. Jerome, famous
    bible translator from Greek and Hebrew to
    Latin; ±347—419/420):
    • 2 types of translations: “words” and
      “meanings” (he favored the latter)
   Here two types of equivalence relevant:
         Linguistic Equivalence
   (Broader than similarity of
   Linguistic equivalence refers to
    similarity of linguistic features of
    a text.
   Examples of relevant linguistic
    features are:
    • Lexical similarity
    • Grammatical accuracy
    • In general: emphasis on formal-textual
      characteristics (cf. automatic
     Psychological Equivalence
   Psychological equivalence refers
    to similarity of (psychological)
    meaning and scores
   Similarity in a broad sense:
    • Textual, e.g.,
         Connotation of words, implied context of
         Comprehensibility
    • Metrical:
         Score comparability
   Relationship between Two
  Three possible relations between linguistic and
    psychological features, depending on the
    a. complete                       c. none
                        b. partial

psych.   linguistic

    Translatable      Poorly translatable   Essentially
A psychological test/item is
       Well translatable if linguistic and
        psychological features yield the same
       Poorly translatable if linguistic
        and psychological features do not
        entirely converge (e.g., translation of
        slang: meaning is translatable, but
        conciseness is lost)
       Non-translatable if there is a
        complete or nearly complete
        nonoverlap (e.g., Jabberwocky)
Jabberwocky (Lewis Carroll, 1871)

 'Twas brillig, and the slithy toves
 Did gyre and gimble in the wabe;
 All mimsy were the borogoves,
 And the mome raths outgrabe.

 "Beware the Jabberwock, my son!
 The jaws that bite, the claws that
 Beware the Jubjub bird, and shun
 The frumious Bandersnatch!"
                                       Illustration by John Tenniel
          Framework for
   Need for a theoretical—
    methodological framework that links
    all stages of a project
   Bias and equivalence as key
       Steps in Designing Cross-Cultural Tests
            (Hambleton & Patsula, 1999)
1. Ensure that construct equivalence exists in the language and cultural groups
   of interest.
2. Decide whether to adapt an existing test or develop a new test.
3. Select well-qualified translators.
4. Translate and adapt the test.
5. Review the adapted version of the test and make necessary revisions.
6. Conduct a small tryout of the adapted version of the test.
7. Carry out a more ambitious field-test.
8. Choose a statistical design for connecting scores on the source and target
   language versions of the test.
9. If cross-cultural comparisons are of interest, ensure equivalence of the
   language versions of the test.
10. Perform validation research, as appropriate.
11. Document the process and prepare a manual for the users of the adapted
12. Train users.
13. Monitor experiences with the adapted test, and make appropriate revisions.
        Overview of Common
       Procedures to Examine
      Accuracy of Translations/
   Procedures as opportunities to
    strengthen the quality of a
    translation/adaptation project
   Two taxonomies presented here:
    • common: (back) translations vs.
      committee approach
    • use of existing/new material
    Theoretical and Methodological
   Crucial concept in translations is
    • Linguistic
         Mapping of linguistic meaning (word
          meaning, sentence meaning)
    • Psychological
         Mapping of psychological meaning (serves
          the same psychological function in all
   A good translation combines these
   Adoption (Close “literal” translation)
    • Advantage: maintains metric equivalence
    • Disadvantage: adequacy (too) readily
      assumed, should be demonstrated
   Adaptation (changing contents of one
    or more items so as to increase
    cultural appropriateness)
    • Advantage: more flexible, more tailored to the
    • Disadvantage: fewer statistical techniques
      available to compare scores across cultures
   Assembly (composing a new
    • Advantage: very flexible
    • Disadvantage: almost no comparability
 A Sample of Possible Procedures
                 (after Harkness, 2003)

                       • Translation back translation
                       • Committee approach (forward
 Translation stage       translations)
                       • Mixed approaches (e.g., independent

                       •Think alouds, focus groups
 Pretesting stage
                       •Feedback from mono- and bilinguals
   (qualitative)       •Comprehension and readability checks

Pretesting or actual
                       • Equivalence and bias analyses
                         (DIF, structural equivalence)
   Strength and Weakness of
  Translations Back Translation
      Main strengths               Main weaknesses

• Well accepted quality       • Capitalizes on linguistic,
  check; standard               cultural, and item-
  procedures well known         writing skills of (usually)
  in scientific community       a single person
  (incl. researchers, grant   • Can produce stilted
  institutions and journal      language
  boards)                     • Readability and
• No knowledge of target        comprehensibility in
  language required             target language may be
       What is the Best Option?
   One type is not intrinsically better or
    worse than another
   Main question is
         What is globally the best choice?
         What is the best choice in a specific
 Four Important Perspectives
(Harkness & Van de Vijver, in preparation):
       Adaptation Perspectives

              Integration   Indicator


   A good translation/adaptation
    combines equivalence
   What is a good translation/
    •A translation or adaptation is
      good when it combines high
      levels of construct, cultural,
      linguistic, and measurement
    Is There a Best Way to Translate
             an Instrument?

   Simple items often straightforward to
    • Close translations will do well, various
      kinds of equivalence jointly maximized
   More complex items often require
    choices about which equivalence will
    be maximized:
    • Maximizing comparability or cultural
      appropriateness ?
   Different perspectives on
    equivalence often, but not
    always compatible

   Example: cross-cultural differences
    in modes of address
    • Maximizing linguistic equivalence may
      challenge cultural appropriateness (e.g.,
      requests may be too direct)
    • Maximizing cultural appropriateness
      may challenge statistical equivalence
      (e.g., rephrasing may threaten
      comparability of scores)
Taxonomy of Adaptations
Most examples come from

                               Traveling with Cognitive Tests:

                    Testing the Validity of a KABC-II Adaptation in India

                                           Maike Malda

                                   Fons J. R. van de Vijver

                                 Krishnamachari Srinivasan

                                     Catherine Transler

                                     Prathima Sukumar

 Accepted for publication in Assessment.
Kaufman Assessment Battery
 for Children (second edition)
   Atlantis
   Number Recall
   Rover
   Triangles
   Word Order
   Pattern Reasoning
   Story Completion  replaced by
    adaptation of
    WISC(/-R/-III) Picture Arrangement
      1. Example Construct-Driven
   Problems with the behaviors or
    attitudes associated with the
    construct or with communication
    norms pertaining to these behaviors
    or attitudes

   Usage of somatic and psychological
    symptoms in depression inventories
         Differential norms in allowance to express
          psychological symptoms across cultures
   Patel, Abas, Broadhead, Todd, & Reeler (2001)
     • In Zimbabwe, multiple somatic complaints such as
       headaches and fatigue are the most common
       presentations of depression. On inquiry, however,
       most patients freely admit to cognitive and
       emotional symptoms. Many somatic symptoms,
       especially those related to the heart and the head,
       are cultural metaphors for fear or grief. Most
       depressed individuals attribute their symptoms to
       “thinking too much” (kufungisisa), to a
       supernatural cause, and to social stressors. Our
       data confirm the view that although depression in
       developing countries often presents with somatic
       symptoms, most patients do not attribute their
       symptoms to a somatic illness and cannot be said
       to have “pure” somatisation. This means that it is
       vital to understand the culture specific terminology
       used by patients and to assess mood in those with
       multiple somatic complaints.
   Consequence
     • Common western measures of depression will
       under-diagnose depression in Shona speakers.
        2. Example Culture-Driven
Example: ‘Burglar’ (Picture Arrangement; adapted for use
in low-SES children in Bangalore, India )


        Problems:
        1. Unclear whether the burglar was getting in
           or getting out;
        2. Man not recognized as burglar;
        3. Window was not recognized (vertically
           moving windows are uncommon in India)

Malda, Van de Vijver, Srinivasan, Transler (2008): Adapting a Western Cognitive Test for a

Non-Western Context: The KABC-II in Bangalore, India
    3. Example of Language-Driven
   Example: Do you often feel distressed?
   Translation to Dutch:
    • “Distressed” does not have an equivalent word in Dutch
    • Possible solutions
          Composite of different emotions in Dutch; ask for
           frequency of composite (“how often do you feel X and Y?”).
           Problem: composite may not be recognizable
          Choose a single emotion that is as close as possible;
           problem: change of item content if no close match can be
          Describe the emotion in the item (e.g., vignette); problem:
           may require a similar description in English original
    • Need to check adequacy of chosen solution in statistical
    • Combination of judgmental and statistical evidence
      crucial in instruments that are more difficult to
   Language and test content:
    • Adaptation of words in subtest Atlantis:
       Kannada nonsense words
        (e.g., English ‘Dablee’  Kannada
       Important: number of syllables

    • Adaptation of digits in subtest Number
       based on number of syllables (1 in

        English version; first 2 and then 3 in
        Kannada version)
    4. Example of Measurement-Driven
         Adaptation (Unfamiliarity)
   Kaufman ABC used in Bangalore
    (Kannada-speaking children)
   Adaptation of words in subtest Word Order
    based on:
              Unfamiliarity and ambiguity of objects and words
              Number of syllables

 Original version 

Kannada version 

Malda, Van de Vijver, Srinivasan, Transler (in review): Adapting a Western Cognitive Test for
a Non-Western Context: The KABC-II in Bangalore, India
Original version 

Kannada version 

      Problem: word for star in Kannada is too long,
      English word “star” is well known but too short
 Original version 

Kannada version 

    (1) Key was often called ice cream;
    (2) English word “key” was often used, which is
       too short (monosyllabic)
Original version 

Kannada version 

     Problem: original drawing was not easily
     recognized as house, distinguishing features
Example: ‘Painting’

Problem: mirror was not recognized
       Test content:
            Additional instructions in subtest Rover
            One additional instruction in subtest Pattern
            Slight change of subtest composition and item order
            in       subtest Triangles

Sample item Original version 

     Sample item Indian version 

    Problem: original sample item was too difficult; this
    item has been added as actual test item
      Background Reading
(1)             (2)

To top