Learning Center
Plans & pricing Sign in
Sign Out

Understanding and measuring disability


									Day 1: Background

Understanding and measuring
Ken Black and Margie Schneider

Washington Group on Disability
Statistics (WG)

Set up in 2002 – meet annually
Foster international cooperation in the area of
health and disability statistics
Develop comparable measures for
disability for use internationally
Aim to measure equalisation of
opportunities – degree to which people
with difficulties in 6 domains experience
disadvantage in education, employment,
and social inclusion
    The WG was set up to:

•   Untangle the web of confusing and
    conflicting disability estimates
•   Develop a small set of general
    disability measures
•   Develop extended set/s of items to
    measure disability on population
•   Address methodological issues
    associated with disability measurement

    Moving from Concept to Definition to

     The Conceptual Model

     ICF selected as the conceptual model:
       • Common point of reference
       • Common vocabulary

       •   Does not provide an operational
           definition or a way to measure the
                                The ICF Model
                                       Health Condition

         Body Function &                   Activities        Participation
      Structure (Impairment)              (Limitation)       (Restriction)

                            Personal               Environmental
                            Factors                   Factors

Source: World Health Organization, 2001

               Moving from Concept to Definition to
             The Definitional Paradox
             • There is no single operational definition of
             • Different operational definitions lead to different
             • The question you are trying to answer (the
               purpose) will determine which definition to use
             • Need to understand the choices that are being
               made when
                • a purpose & a definition are chosen
                • time, expenses and respondent burden limit
                  number of questions
        Purpose of Data Collection
3 major classes of purposes at aggregate
•       Service Provision
•       Monitoring functioning in the population
•       Assess equalization of opportunities

2 criteria for selection of a purpose
•       Relevance—particularly for policy makers
        and program officials
•       Feasibility

        Purpose: Service provision
    •   Seeks to identify those with specific
        needs, usually the most serious
    •   Requires detailed information about the
        person and the environment
    •   Influenced by the organization and
        structure of service organizations within
        a particular culture
      Purpose: Monitoring functioning in
      the population
•   Seeks to identify Population reporting work limitation
    all those with
    activity or           25

                           Proportion (%)
    limitation            20
•   Response
    comparability         15
    problematic since
    participation is      10
    culturally and
    environmentally         5
                                                          2005   2010   2015   2020   2025

      Purpose: Equalization of

                                                                 % Employed
•   Seeks to identify
    all those at greater                             90
    risk than the
                                    Proportion (%)

    general population
    for limitations in                               60
    activity or
•   Disability as a                                  30

                                                          Nondisabled     Disabled
            Locating Risk in the ICF Model
                                    Health Condition

       Body Function &                                  ?   Participation
    Structure (Impairment)                                  (Restriction)

                          Personal              Environmental
                          Factors                  Factors

Source: World Health Organization, 2001

            Moving from Concept to Definition
            to Measurement:

        Measurement of equalization of opportunities
        • Locate the definition of disability at the most basic
          level of activity/participation in core domains
           • Defined as the ability or inability to carry out
             basic actions at the level of the whole person
             (i.e. walking, climbing stairs, lifting
             packages, seeing a friend across the room)
        • Connection between ‘disability’ and participation
          can be made during data analysis
Possible question options
(basic activities)
                         Cognitive functions
 • Walking
                         • Learning
 • Climbing stairs
                         • Remembering
 • Bending or stooping
                         • Making decisions
 • Reaching or lifting
                         • Concentrating
 • Using hands
                         Emotional functioning
                         • Interpersonal
 • Seeing                  interactions
 • Hearing               • Psychological well-being
 • Understanding

 • Speaking

   Criteria for inclusion of domains

   •   Cross cultural comparability
   •   Suitability for self-report
   •   Parsimony
   •   Validity across various methodological
      Washington Group Short Set
     Introductory phrase:
     The next questions ask about difficulties you may have doing
     certain activities because of a HEALTH PROBLEM.

     Response options: No - no difficulty; Yes – some difficulty;
     Yes – a lot of difficulty; Cannot do at all

1.   Do you have difficulty seeing, even if wearing glasses?
2.   Do you have difficulty hearing, even if using a hearing aid?
3.   Do you have difficulty walking or climbing steps?
4.   Do you have difficulty remembering or concentrating?
5.   Do you have difficulty (with self-care such as) washing all over
     or dressing?
6.   Using your usual (customary) language, do you have difficulty
     communicating, for example understanding or being

      WG Extended Set

         Need more detailed information on:
            Who has difficulties (prevalence)
            What is their experience (outcomes)
            Cause, onset and impact on daily

         Establish level of cross country
                              WG extended set
                                   More domains (basic and
                                   More detailed information on each
                                   Basic environmental factors
                                Age of onset of difficulty
                                Cause of difficulty
                                Impact on daily and less regular
                              (See questionnaire for questions)

                                          Basic Activity Domains                                Complex Activity Domains
  R                                                         Communi-     Cognition/     New     ADL/                  Life       Participation
  O           Questionnaire   Vision   Hearing   Mobility
                                                             cation    remembering    Domains   IADL
                                                                                                       Along with
                                                                                                                    Activities    in Society
  W            Topic/Type                                                                                People

                Short Set
                Single ?s

                                                                           Upper body, Learning, Affect, Pain, Fatigue
               Use of AD/

                Micro-E       Technical & personal assistance: wheel chair, eye glasses, personal attendant
                 with                                                                            ?

    5         Age at Onset

    6            Cause

    7           Duration

                               To be captured through analysis of outcome measures and qualitative case
    8            Impact

                 Meso-        Environment beyond the person: home accommodations, transportation,
              Environment     service provision, attitudes of others
              Environment     Affects the entire country: policies & legislation, societal attitudes & practices
     Basic Activity Domains
        Vision                       •   Upper body
        Hearing                      •   Learning
        Mobility                     •   Affect
        Communication                •   Pain
        Cognition                    •   Fatigue

        Indicates part of WG
        short set

     Complex activity domains
•   ADL and IADL
       Bathing, dressing, toileting
     • Household activities / shopping
•   Getting along with people/social relations
     • Making friends / maintaining friendships
     • Interacting with strangers / persons in
•   Life activities
     • Going to school / getting and keeping a job
•   Participation in society
     • Social / Religious / Civic activities
    Indicates part of WG short set
Question Sets: Rows
              Short set:
              Capacity Row 1:
                  six single questions
                  functioning without assistance*

         •    Expanded short set:
         •    Capacity Row 1:
              •   eight single questions (upper body and
                  learning added)
              •   functioning without assistance*

*with the exception of vision and hearing

Question Sets: Rows
     •       Extended Set A
             Capacity Row 2:
             Multiple questions/functioning without
             Performance Rows 3-4:
             Use of technical and personal assistance
             Multiple questions/functioning with assistance
     •       Extended Set B / Rows 5-8:
             Age at onset, Cause, Duration, Impact
     •       Extended Set C / Row 9:
             Meso-environment: beyond the person (may
               or may not be domain specific)
    Measures of Capacity
    •   Ask for difficulty functioning in basic
        or complex activities (ADL) without
        the use of assistive devices or the
        help of others.
    •   With the exception of:
        •   Vision: Do you have difficulty seeing,
            even if wearing glasses?
        •   Hearing: Do you have difficulty
            hearing, even if using a hearing aid?
        •   since limitations in these domains can
            often be overcome with the use of
            glasses or hearing aids.

    Measures of Performance
•   Ask for use of assistive devices:
    technical or personal assistance, e.g.
    wheelchair or personal assistant
•   Ask for difficulty functioning in basic
    activities with the use of assistive
    devices or the help of others, i.e. in their
    current environment
•   Measure functioning with all available
    accommodation – direct measure of the
    interaction of individual characteristics
    and the micro-environment
                                                 Basic activity domains
Question topic                                             Mobility
Short set              Do you have difficulty walking or climbing steps?
                       1.Do you have difficulty walking 500 (100) meters on level ground – without
                       using your aids?
                       2.Do you have difficulty waling up and down a flight of stairs/12 steps/a small
                       hill – without using your aids?
                       1.Do you use any aids or equipment or receive help for walking or moving
Use of AD/Micro-
                       2.Which aid(s) do you use? E.g. a cane, walker or wheelchair, crutches?
                       1.Do you have difficulty walking 500 (100) meters on level ground – even
Functioning with       when using your aids?
Assistance             2.Do you have difficulty waling up and down a flight of stairs/12 steps/a small
                       hill – even when using your aids?
                       How old were you when you first started having difficulty walking or climbing
Age at onset
                       What condition or health problem is the reason for you difficulty walking
                       and/or climbing steps?
                       How long have you had your current level of difficulty walking and/or climbing
Impact                 How does your difficulty affect your ability to work, attend school, etc?
                       Is the availability/accessibility of transportation a problem for you?

               Still to do:

               •   Develop an extended question set
                   for Block 2: age of onset, cause,
                   duration, and impact
                   •    Agree on an approach to measurement
                        of impact
               •   Develop/agree upon a question set
                   for the meso-environment
               •   Agree upon an approach to
                   measuring/ capturing complex
                   activity limitations

      Inclusive as ask about ‘difficulty’
      and not ‘disability’
      Use multiple response options – get
      grading beyond just ‘yes’ and ‘no’
      Ability link to other demographic
      and socioeconomic variables
      included in collection vehicle

    Intended use of data
•   Identify people with difficulties/disabilities
•   Describe experience: Compare levels of
    participation in employment, education, or
    family life for those with disability versus
    those without disability to see if persons
    with disability have achieved social inclusion
•   Monitor effectiveness of programs / policies
    to promote full participation can be
•   Monitor prevalence trends for persons with
    limitations in specific basic activity domains
•   One set of measures will not satisfy
    multiple needs for disability data
•   Excluded populations:
    •   Very young children
    •   Institutionalized population
    •   Homeless
    •   Floating populations
•   It is not our purpose to:
    •   identify every person with a disability within
        every community
    •   replicate a population evaluated across a wider
        range of domains that would be possible with
        other forms of data collection

    Plan for testing
     Cognitive testing –
         how people understand the questions
         What are problematic aspects of questions
         Revise questions based on findings of
         cognitive testing
     Field testing
         Revised Qs are used on large representative
         in 7 countries
         Test out different versions
         Compare to Questions used in Round 1 of
         piloting (2005/2006).
         Compare results across countries –
         comparable or not?
Day 3 and 4

         Field testing

Purpose of field testing
Look at ‘real world’ issues/problems
with implementation
Compare question options being
Compare response options being
Obtain interviewer feedback
Examine retest reliability
Test processes and translation
Process of field testing

Revisions from Cognitive testing
Translation of questionnaire and adding
country specific Qs (e.g. education codes)
and problems
Selection of fieldworkers
Training of fieldworkers
Field methodology
Data capturing


  Selection of 1,000 respondents
  Both urban and rural areas
  (minimum 20% of either – but
  preferable to have close to 50% in
  Can be clustered i.e. can select one
  village if sufficient sample obtained.
Sampling (cont.)

 One adult from each selected
 One child where present from each
 selected household

    Selection determined by ‘next birthday’

Sampling (cont.)

 Allocation of versions 1 and 2 of
    First selected household where an
    interview(s) conducted receives version
    Second selected household where an
    interview(s) conducted receives version
    Pattern repeats until all interviews
Fieldworker selection

 Use experienced interviewers where
 practical to do so
 Need to be able to:
   Gain respondent’s trust and
   Ask questions as worded
   Accurately record responses
   Record any issues/problems as they

Fieldworker training (1)

 Length of training
   2 to 3 days, including field exercise

 Sensitisation to disability
    awareness training
    presentation from person with a
   disability and/or disability advocate
General points
 Select some disabled interviewers
 Importance of all interviewers
 having a good understanding of
 what disability is and is not and how
 it relates to the questions asked
 Avoid using term ‘disabled’ or ‘with
 All interviews are face to face
 Translation training

General points contd.
Confidentiality and understanding what this
Getting informed consent (ethics
Interviewers must be able to explain
purpose of survey
Read questions in set order and with set
Editing in field of completed questionnaires
Submitting of completed interviews to head
Types of interviews

 Direct: respondent answers for him
 or herself
 Interpreted: an interpreter
 ‘translates’ and respondent answers
 Facilitated: a third party assist in
 explaining (e.g. intellectually
 disabled person)
 Proxy: a person responds for
 another (e.g. child)

Interviewing disabled people
 Show respect and treat like anyone
 Don’t use first names unless
 permitted to do so
 Address the person directly (not
 their attendant)
 Ask how you can adapt your
 presentation to make it easier (no
 need to ask what is wrong with
Hearing difficulties
Lip reading
Face person
Get attention before speaking
Reduce background noise
Set context – especially when
changing topics
Use written communication (literate)

Physical difficulties
Accessibility of building where
conducting interviews
Presence of attendant and
confidentiality issues
Get to same level (e.g. sitting for
person using wheelchair)
Person to be seated comfortably
Address person directly
Pointing may be difficult
Visual difficulties

 Large print and small print for cue
 Braille versions of cue cards
 Good contrast printing for pictures
 and print (black on white or yellow)
 Identify yourself and others in the
 room verbally

Communication difficulties

 Clarify preferred mode of
 Repeat what you think was said to
 clarify unclear speech
 Limit to yes / no questions
Specific learning difficulties

 Manage problems in spatial
 orientation, hand-eye coordination
 Limit auditory, visual and tactile
 Avoid written text
 Explain carefully (if verbal language
 skills are affected)

Intellectual difficulties

 Be careful with informed consent
 Explain terms simply
 Listen carefully
 Have familiar person (friend or
 relative) close by
 Use pictures or role play with little
 human or animal figures
Emotional of mental health difficulties

        Side effects of medication
        Break up interview if too fatigued
        Give encouragement and support
        Manage expressions of frustration
        Manage stress

    Hidden difficulties

         Might not come forward with
         information because of fear of
         Effect of medication
         Need to break up interview
Interviewer debriefing
Interviewers well placed to evaluate
process and content at early stage
Interviewers to note comments in
margins of questionnaires
Different approaches:
  Group discussion (focus group technique)
  Interviewer rating forms
  Standardised interviewer questionnaires
  Combination of all three

Field methodology (1)
 Selecting households
   Household selection based on skip interval
   applied within country specific sample frame
   Selection from within both urban and rural
   Can be tightly clustered e.g. one village
   representative of others

 Selecting individuals within households
   Adults: select adult (if more than one adult,
   select the one with the next birthday)
   Children: select child (if more than one child,
   select the one with the next birthday)
Field methodology (2)

 If a household or individual refuses
 to be interviewed, do not substitute
 a different household
 Do 10% retest (for reliability
   Retest every 10th household OR
   Randomly select 100 for retesting at
   start of fieldwork
   Retest both the child and adult

Ethical clearance and consent

 Do you have an ethics committee in
 country? What are regulations for
 Statistical office?
 Need consent from adults for self or
 Ensure confidentiality and
Data capturing

 Field editing of questionnaires
   Who’s responsibility?
 Central checking and final
   Sending back to field if incomplete
 Enter into Excel
 ?Double data entry
 Data cleaning

   Translation by small group comprising stats
   and health people + one or two disabled
   Review by 2 external reviewiers
   Committee to go over comments of reviewers
   and revise accordingly
   Conceptual translation
 Problems experienced:
   Inability to translate certain concepts
   Setting up committee
Preparation for field testing
 Do sampling – decide on location of field
 Complete translation
 Review cognitive testing findings for own
 Decide on fieldworker selection and
 recruit enough (? Include disabled people)
 Set dates and plan location and length of
 Plan 2 days of piloting after training –
 with strong support and supervision

To top