Understanding and measuring disability

Document Sample
Understanding and measuring disability Powered By Docstoc
					Day 1: Background


Understanding and measuring
disability
Ken Black and Margie Schneider




Washington Group on Disability
Statistics (WG)

Set up in 2002 – meet annually
Foster international cooperation in the area of
health and disability statistics
Develop comparable measures for
disability for use internationally
Aim to measure equalisation of
opportunities – degree to which people
with difficulties in 6 domains experience
disadvantage in education, employment,
and social inclusion
    The WG was set up to:

•   Untangle the web of confusing and
    conflicting disability estimates
•   Develop a small set of general
    disability measures
•   Develop extended set/s of items to
    measure disability on population
    surveys
•   Address methodological issues
    associated with disability measurement




    Moving from Concept to Definition to
    Measurement:

     The Conceptual Model

     ICF selected as the conceptual model:
       • Common point of reference
       • Common vocabulary


       •   Does not provide an operational
           definition or a way to measure the
           concepts
                                The ICF Model
                                       Health Condition
                                      (disorder/disease)



         Body Function &                   Activities        Participation
      Structure (Impairment)              (Limitation)       (Restriction)




                            Personal               Environmental
                            Factors                   Factors




Source: World Health Organization, 2001




               Moving from Concept to Definition to
               Measurement:
             The Definitional Paradox
             • There is no single operational definition of
               disability
             • Different operational definitions lead to different
               estimates
             • The question you are trying to answer (the
               purpose) will determine which definition to use
             • Need to understand the choices that are being
               made when
                • a purpose & a definition are chosen
                • time, expenses and respondent burden limit
                  number of questions
        Purpose of Data Collection
3 major classes of purposes at aggregate
  level
•       Service Provision
•       Monitoring functioning in the population
•       Assess equalization of opportunities

2 criteria for selection of a purpose
•       Relevance—particularly for policy makers
        and program officials
•       Feasibility




        Purpose: Service provision
    •   Seeks to identify those with specific
        needs, usually the most serious
        problems
    •   Requires detailed information about the
        person and the environment
    •   Influenced by the organization and
        structure of service organizations within
        a particular culture
      Purpose: Monitoring functioning in
      the population
•   Seeks to identify Population reporting work limitation
    all those with
    activity or           25
    participation




                           Proportion (%)
    limitation            20
•   Response
    comparability         15
    problematic since
    participation is      10
    culturally and
    environmentally         5
    determined
                                                      0
                                                          2005   2010   2015   2020   2025




      Purpose: Equalization of
      opportunities

                                                                 % Employed
•   Seeks to identify
    all those at greater                             90
    risk than the
                                    Proportion (%)




    general population
    for limitations in                               60
    activity or
    participation
•   Disability as a                                  30
    demographic

                                                     0
                                                          Nondisabled     Disabled
            Locating Risk in the ICF Model
                                    Health Condition
                                   (disorder/disease)



       Body Function &                                  ?   Participation
    Structure (Impairment)                                  (Restriction)




                          Personal              Environmental
                          Factors                  Factors



Source: World Health Organization, 2001




            Moving from Concept to Definition
            to Measurement:

        Measurement of equalization of opportunities
        • Locate the definition of disability at the most basic
          level of activity/participation in core domains
           • Defined as the ability or inability to carry out
             basic actions at the level of the whole person
             (i.e. walking, climbing stairs, lifting
             packages, seeing a friend across the room)
        • Connection between ‘disability’ and participation
          can be made during data analysis
Possible question options
(basic activities)
 Mobility
                         Cognitive functions
 • Walking
                         • Learning
 • Climbing stairs
                         • Remembering
 • Bending or stooping
                         • Making decisions
 • Reaching or lifting
                         • Concentrating
 • Using hands
                         Emotional functioning
 Sensory
                         • Interpersonal
 • Seeing                  interactions
 • Hearing               • Psychological well-being
 Communicating
 • Understanding

 • Speaking




   Criteria for inclusion of domains

   •   Cross cultural comparability
   •   Suitability for self-report
   •   Parsimony
   •   Validity across various methodological
       modes
      Washington Group Short Set
     Introductory phrase:
     The next questions ask about difficulties you may have doing
     certain activities because of a HEALTH PROBLEM.

     Response options: No - no difficulty; Yes – some difficulty;
     Yes – a lot of difficulty; Cannot do at all

     Questions:
1.   Do you have difficulty seeing, even if wearing glasses?
2.   Do you have difficulty hearing, even if using a hearing aid?
3.   Do you have difficulty walking or climbing steps?
4.   Do you have difficulty remembering or concentrating?
5.   Do you have difficulty (with self-care such as) washing all over
     or dressing?
6.   Using your usual (customary) language, do you have difficulty
     communicating, for example understanding or being
     understood?




      WG Extended Set

         Need more detailed information on:
            Who has difficulties (prevalence)
            What is their experience (outcomes)
            Cause, onset and impact on daily
            activities

         Establish level of cross country
         comparability
                              WG extended set
                                   More domains (basic and
                                   complex)
                                   More detailed information on each
                                   domain
                                   Basic environmental factors
                              +
                                Age of onset of difficulty
                                Cause of difficulty
                                Impact on daily and less regular
                                activities
                              (See questionnaire for questions)




                                          Basic Activity Domains                                Complex Activity Domains
                                                                                                        Getting
  R                                                         Communi-     Cognition/     New     ADL/                  Life       Participation
  O           Questionnaire   Vision   Hearing   Mobility
                                                             cation    remembering    Domains   IADL
                                                                                                       Along with
                                                                                                                    Activities    in Society
  W            Topic/Type                                                                                People

                Short Set
                Single ?s
Capacity




              Functioning
                without
              Assistance
                                                                           Upper body, Learning, Affect, Pain, Fatigue
               Use of AD/
Performance




                Micro-E       Technical & personal assistance: wheel chair, eye glasses, personal attendant
              Functioning
                 with                                                                            ?
              Assistance


    5         Age at Onset

    6            Cause

    7           Duration

                               To be captured through analysis of outcome measures and qualitative case
    8            Impact
                                                                studies

                 Meso-        Environment beyond the person: home accommodations, transportation,
    9
              Environment     service provision, attitudes of others
                Macro-
  10
              Environment     Affects the entire country: policies & legislation, societal attitudes & practices
     Basic Activity Domains
        Vision                       •   Upper body
        Hearing                      •   Learning
        Mobility                     •   Affect
        Communication                •   Pain
        Cognition                    •   Fatigue




        Indicates part of WG
        short set




     Complex activity domains
•   ADL and IADL
       Bathing, dressing, toileting
     • Household activities / shopping
•   Getting along with people/social relations
     • Making friends / maintaining friendships
     • Interacting with strangers / persons in
       authority
•   Life activities
     • Going to school / getting and keeping a job
•   Participation in society
     • Social / Religious / Civic activities
    Indicates part of WG short set
Question Sets: Rows
              Short set:
              Capacity Row 1:
                  six single questions
                  functioning without assistance*

         •    Expanded short set:
         •    Capacity Row 1:
              •   eight single questions (upper body and
                  learning added)
              •   functioning without assistance*

*with the exception of vision and hearing




Question Sets: Rows
     •       Extended Set A
             Capacity Row 2:
             Multiple questions/functioning without
             assistance
             Performance Rows 3-4:
             Use of technical and personal assistance
             Multiple questions/functioning with assistance
     •       Extended Set B / Rows 5-8:
             Age at onset, Cause, Duration, Impact
     •       Extended Set C / Row 9:
             Meso-environment: beyond the person (may
               or may not be domain specific)
    Measures of Capacity
    •   Ask for difficulty functioning in basic
        or complex activities (ADL) without
        the use of assistive devices or the
        help of others.
    •   With the exception of:
        •   Vision: Do you have difficulty seeing,
            even if wearing glasses?
        •   Hearing: Do you have difficulty
            hearing, even if using a hearing aid?
        •   since limitations in these domains can
            often be overcome with the use of
            glasses or hearing aids.




    Measures of Performance
•   Ask for use of assistive devices:
    technical or personal assistance, e.g.
    wheelchair or personal assistant
•   Ask for difficulty functioning in basic
    activities with the use of assistive
    devices or the help of others, i.e. in their
    current environment
•   Measure functioning with all available
    accommodation – direct measure of the
    interaction of individual characteristics
    and the micro-environment
                                                 Basic activity domains
Question topic                                             Mobility
Short set              Do you have difficulty walking or climbing steps?
                       1.Do you have difficulty walking 500 (100) meters on level ground – without
Functioning
                       using your aids?
without
                       2.Do you have difficulty waling up and down a flight of stairs/12 steps/a small
assistance
                       hill – without using your aids?
                       1.Do you use any aids or equipment or receive help for walking or moving
Use of AD/Micro-
                       around?
Environment
                       2.Which aid(s) do you use? E.g. a cane, walker or wheelchair, crutches?
                       1.Do you have difficulty walking 500 (100) meters on level ground – even
Functioning with       when using your aids?
Assistance             2.Do you have difficulty waling up and down a flight of stairs/12 steps/a small
                       hill – even when using your aids?
                       How old were you when you first started having difficulty walking or climbing
Age at onset
                       steps?
                       What condition or health problem is the reason for you difficulty walking
Cause
                       and/or climbing steps?
                       How long have you had your current level of difficulty walking and/or climbing
Duration
                       steps?
Impact                 How does your difficulty affect your ability to work, attend school, etc?
Meso-
                       Is the availability/accessibility of transportation a problem for you?
Environment




               Still to do:

               •   Develop an extended question set
                   for Block 2: age of onset, cause,
                   duration, and impact
                   •    Agree on an approach to measurement
                        of impact
               •   Develop/agree upon a question set
                   for the meso-environment
               •   Agree upon an approach to
                   measuring/ capturing complex
                   activity limitations
    Features

      Inclusive as ask about ‘difficulty’
      and not ‘disability’
      Use multiple response options – get
      grading beyond just ‘yes’ and ‘no’
      Ability link to other demographic
      and socioeconomic variables
      included in collection vehicle




    Intended use of data
•   Identify people with difficulties/disabilities
•   Describe experience: Compare levels of
    participation in employment, education, or
    family life for those with disability versus
    those without disability to see if persons
    with disability have achieved social inclusion
•   Monitor effectiveness of programs / policies
    to promote full participation can be
    monitored
•   Monitor prevalence trends for persons with
    limitations in specific basic activity domains
    Limitations
•   One set of measures will not satisfy
    multiple needs for disability data
•   Excluded populations:
    •   Very young children
    •   Institutionalized population
    •   Homeless
    •   Floating populations
•   It is not our purpose to:
    •   identify every person with a disability within
        every community
    •   replicate a population evaluated across a wider
        range of domains that would be possible with
        other forms of data collection




    Plan for testing
     Cognitive testing –
         how people understand the questions
         What are problematic aspects of questions
         Revise questions based on findings of
         cognitive testing
     Field testing
         Revised Qs are used on large representative
         sample
         in 7 countries
         Test out different versions
         Compare to Questions used in Round 1 of
         piloting (2005/2006).
         Compare results across countries –
         comparable or not?
Day 3 and 4



         Field testing




Purpose of field testing
Look at ‘real world’ issues/problems
with implementation
Compare question options being
tested
Compare response options being
tested
Obtain interviewer feedback
Examine retest reliability
Test processes and translation
Process of field testing

Revisions from Cognitive testing
Translation of questionnaire and adding
country specific Qs (e.g. education codes)
and problems
Sampling
Selection of fieldworkers
Training of fieldworkers
Field methodology
Data capturing
Analysis




Sampling

  Selection of 1,000 respondents
  Both urban and rural areas
  (minimum 20% of either – but
  preferable to have close to 50% in
  each)
  Can be clustered i.e. can select one
  village if sufficient sample obtained.
Sampling (cont.)

 One adult from each selected
 household
 One child where present from each
 selected household

    Selection determined by ‘next birthday’




Sampling (cont.)

 Allocation of versions 1 and 2 of
 questionnaire:
    First selected household where an
    interview(s) conducted receives version
    A
    Second selected household where an
    interview(s) conducted receives version
    B
    Pattern repeats until all interviews
    completed
Fieldworker selection

 Use experienced interviewers where
 practical to do so
 Need to be able to:
   Gain respondent’s trust and
   cooperation
   Ask questions as worded
   Accurately record responses
   Record any issues/problems as they
   occur




Fieldworker training (1)

 Length of training
   2 to 3 days, including field exercise

 Sensitisation to disability
    awareness training
    presentation from person with a
   disability and/or disability advocate
   recommended
General points
 Select some disabled interviewers
 Importance of all interviewers
 having a good understanding of
 what disability is and is not and how
 it relates to the questions asked
 Avoid using term ‘disabled’ or ‘with
 disabilities’
 All interviews are face to face
 Translation training




General points contd.
Confidentiality and understanding what this
means
Getting informed consent (ethics
clearance?)
Interviewers must be able to explain
purpose of survey
Read questions in set order and with set
wording
Editing in field of completed questionnaires
Submitting of completed interviews to head
office
Types of interviews

 Direct: respondent answers for him
 or herself
 Interpreted: an interpreter
 ‘translates’ and respondent answers
 directly
 Facilitated: a third party assist in
 explaining (e.g. intellectually
 disabled person)
 Proxy: a person responds for
 another (e.g. child)




Interviewing disabled people
 Show respect and treat like anyone
 else
 Don’t use first names unless
 permitted to do so
 Address the person directly (not
 their attendant)
 Ask how you can adapt your
 presentation to make it easier (no
 need to ask what is wrong with
 person)
Hearing difficulties
Lip reading
Lighting
Face person
Get attention before speaking
Reduce background noise
Set context – especially when
changing topics
Use written communication (literate)




Physical difficulties
Accessibility of building where
conducting interviews
Presence of attendant and
confidentiality issues
Get to same level (e.g. sitting for
person using wheelchair)
Person to be seated comfortably
Address person directly
Pointing may be difficult
Visual difficulties

 Large print and small print for cue
 cards
 Braille versions of cue cards
 Good contrast printing for pictures
 and print (black on white or yellow)
 Identify yourself and others in the
 room verbally




Communication difficulties

 Clarify preferred mode of
 communication
 Repeat what you think was said to
 clarify unclear speech
 Limit to yes / no questions
Specific learning difficulties

 Manage problems in spatial
 orientation, hand-eye coordination
 Limit auditory, visual and tactile
 distractions
 Avoid written text
 Explain carefully (if verbal language
 skills are affected)




Intellectual difficulties

 Be careful with informed consent
 Explain terms simply
 Listen carefully
 Have familiar person (friend or
 relative) close by
 Use pictures or role play with little
 human or animal figures
Emotional of mental health difficulties

        Side effects of medication
        Break up interview if too fatigued
        Give encouragement and support
        Manage expressions of frustration
        Manage stress




    Hidden difficulties

         Might not come forward with
         information because of fear of
         stigma
         Effect of medication
         Need to break up interview
Interviewer debriefing
Interviewers well placed to evaluate
process and content at early stage
Interviewers to note comments in
margins of questionnaires
Different approaches:
  Group discussion (focus group technique)
  Interviewer rating forms
  Standardised interviewer questionnaires
  Combination of all three




Field methodology (1)
 Selecting households
   Household selection based on skip interval
   applied within country specific sample frame
   Selection from within both urban and rural
   areas
   Can be tightly clustered e.g. one village
   representative of others

 Selecting individuals within households
   Adults: select adult (if more than one adult,
   select the one with the next birthday)
   Children: select child (if more than one child,
   select the one with the next birthday)
Field methodology (2)

 If a household or individual refuses
 to be interviewed, do not substitute
 a different household
 Do 10% retest (for reliability
 test/retest)
   Retest every 10th household OR
   Randomly select 100 for retesting at
   start of fieldwork
   Retest both the child and adult
   respondents




Ethical clearance and consent

 Do you have an ethics committee in
 country? What are regulations for
 Statistical office?
 Need consent from adults for self or
 child
 Ensure confidentiality and
 anonymity
Data capturing

 Field editing of questionnaires
   Who’s responsibility?
 Central checking and final
 corrections
   Sending back to field if incomplete
 Enter into Excel
 ?Double data entry
 Data cleaning




Translation
 Process:
   Translation by small group comprising stats
   and health people + one or two disabled
   people
   Review by 2 external reviewiers
   Committee to go over comments of reviewers
   and revise accordingly
   Conceptual translation
 Problems experienced:
   Inability to translate certain concepts
   Setting up committee
Preparation for field testing
 Do sampling – decide on location of field
 test
 Complete translation
 Review cognitive testing findings for own
 country
 Decide on fieldworker selection and
 recruit enough (? Include disabled people)
 Set dates and plan location and length of
 training
 Plan 2 days of piloting after training –
 with strong support and supervision

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:8
posted:3/20/2012
language:
pages:29