Outcome Measurements Study Design Statistics by pptfiles

VIEWS: 11 PAGES: 57

									Outcome Measurements
    Study Design

 Liz Lingard BPhty, MPhil, MPH
      Research Coordinator
   Department of Orthopaedics
        Freeman Hospital
Orthopaedic Clinical Research
   Traditionally focused on reporting clinical and
    radiographic results

   Rates of complications following procedures

   Survivorship analysis of the life of a prosthesis

   Many studies are observational
Need For Outcome Measures
   Technological advances, epidemiologic and
    demographic trends and rising public demand had
    rapidly increased health care expenditure over the
    past three decades

   As health care resources are limited they need to
    be allocated according to clinical effectiveness and
    cost-effectiveness
Orthopaedic Interventions
   Surgical treatment for one or more of the
    following musculoskeletal problems
      Pain

      Loss of range of motion

      Instability

      Functional limitation

      Deformity

      Trauma

      Tumour
International Classification of
Functioning, Disability and Health (ICF)
 Functioning and Disability
    Body function and structures

    Activities and participation

 Contextual Factors
    Environmental factors

    Personal factors
Interaction Between ICF Components

                     Health condition
                   (disorder or disease)


  Body Functions        Activities          Participation
  and Structures



           Environmental             Personal
              Factors                Factors
Uses of Outcome Measures

 Clinical   research
 Audit    and quality assurance
 Assess    health care needs of populations
 Assist   in individual patient care
Outcome Measures
 Assessment of disease or injury severity
 Measure change over time due to:
    Natural history

    Intervention

 Supplement mortality, clinical, radiological
  and laboratory measures
 Outcome instruments
    Observer administered

    Patient administered
Observer Administered
   Functional Tests
      Timed Up & Go

      6 minute walk

      Activity Monitoring



   Global rating scores
     Knee Society Score

     Harris Hip Score

     Foot and Ankle Score
Activity Monitoring
Ambulatory activity monitor
    worn for 7 days

ActivPal Monitor
                         matched walking periods




Subject A: 72 year old THR patient           Subject B: 65 year old spouse
                     Activity Summary

 A


  B
                    6           3         0           3       6
           Time sittng (h)                             Time upright (h)
sitting   standing   stepping (slow)   Stepping (moderate)   Stepping (vigerous)
            Stepping Summary

A       2554               6552




B                7202                             8956



    0                   5000              10000          15000              20000
                                     Steps per day

         stepping (slow)          Stepping (moderate)     Stepping (vigerous)
Global Rating Scores
 May combine measures of pain, joint range
  of motion, stability, muscle power,
  deformity, function and even complications
  and investigations
 Weight of scoring of each item varies
 Inter / intra-rater reliability & observer bias
Pitfalls of global scores
   Different scoring systems assign different
    weights to parts of the score and the
    summary results are often inconsistent
    giving contrary measures of success in the
    same patient.
    (Bryant et al. JBJS-Br 1993)
Patient Administered
   Report summary scores for different health
    dimensions
       Physical Function
       Symptoms – pain
       Global judgements of health
       Psychological well-being
       Social well-being
       Cognitive functioning
       Role activities
       Personal constructs
       Satisfaction
Patient Administered
 Disease-specific (WOMAC, AIMS)
 Site or region-specific (KOOS, DASH, ODI)
 Dimension-specific (McGill Pain Questionnaire)
 General health status (SF-36, NHP)
 Health state utility (EQ-5D)
 Satisfaction (Management / Outcome)
WOMAC
   Western Ontario & McMaster University OA index
   Tri-dimensional scale (Likert and VAS versions):
      Pain (5 items)

      Stiffness (2 items)

      Function (17 items)

   Validated on patients with OA of hip and knee
      Sensitive and responsive for detecting change
       over time due to NSAIDs and joint arthroplasty
   Translated into over 50 different languages
WOMAC
   Responses for each item are none, mild, moderate,
    severe or extreme (pain, stiffness or difficulty)
   Scores transformed to 0-100 (100 best)
      100 = none

      75 = mild

      50 = moderate

      25 = severe

      0 = extreme
         Preoperative WOMAC
None     100
          90
          80
Mild
          70
          60                                            Primary TKA
Moderate 50                                             Primary THA
                36 34 36     36        38   37 36 35
          40                      32                    Primary BHR
          30
Severe
          20
          10
Extreme    0
                  Pain       Function       Stiffness


       Note: Data on Primary Joints – 1,061 TKA, 760 THA & 174 BHR
        1-Year WOMAC
None     100
                    88 90             85
          90   81                                    84
                                 79             80
          80                74             72
Mild
          70
          60                                              Primary TKA
Moderate 50                                               Primary THA
          40                                              Primary BHR
          30
Severe
          20
          10
Extreme    0
                    Pain    Function       Stiffness


        Note: Data on Primary Joints - 490 TKA, 387 THA & 75 BHR
Knee injury and Osteoarthritis
Outcome Score (KOOS)
   Developed and validated by Ewa Roos

   Meniscus injury, ACL injury and/or OA

   Adds dimensions to the WOMAC scale:
     Sport and recreation function
     Knee-related quality of life


   Improves sensitivity of evaluation of
    younger or more physically active persons
KOOS – Sports/Recreation
   Squatting

   Running

   Jumping

   Twisting/pivoting

   Kneeling
KOOS – Knee Related QOL
 How often are you aware of your knee
  problems?
 Have you modified your lifestyle to avoid
  potentially damaging activities to your knee?
 How much are you troubled with lack of
  confidence in your knee?
 In general, how much difficulty do you have
  with your knee?
KOOS Profile
100

 80

 60
                                              Controls
                                              OA
 40
       WOMAC
 20
                                     KOOS
  0
 in




                                          L
             s



                       n


                                ec
           es


                    tio




                                      O
Pa




                              t/R
         n




                                      Q
                  nc
      iff




                            or
                 Fu
      St




                           Sp




             Roos EM et al, Osteoarthritis & Cartilage 1999; 7:216-221
KOOS Profile
100

 80                                              Preop
                                                 3 mo postop
 60                                              6 mo postop
                                                 12 mo postop
 40

 20

  0
   Pain   Symptoms   ADL   Sport/Rec   QOL



                           Roos EM et al, JOSPT 1998; 78:88-96
WOMAC Pain and Function
Patients at 10 years after primary TKR
None       100
                       88
            90
                                                 79
Mild        80
            70
            60
Moderate    50
            40
Severe      30
            20
            10
Extreme      0
                      Pain                     Function

                             Kinemax 10-Year
How often are you aware of your
knee problem?
Patients at 10 years after primary TKR
              100
               90
               80
               70
 % patients




               60
               50    40
               40
                                                         28
               30
               20             15                                   11
               10                           6
                0
                    Never   Monthly      Weekly         Daily   Constantly
                                      Kinemax 10-Year
In general, how much difficulty do
you have with your knee?
Patients 10 years after primary TKR
              100
               90
               80
               70
 % patients




               60
               50
                    40
               40
                           29
               30
                                       20
               20
                                                      9
               10                                               1
                0
                    None   Mild     Moderate        Severe   Extreme
                                  Kinemax 10-Year
Short-Form 36 (SF-36)
   Reports on 8 health dimensions
       Physical Functioning
                                      Physical
       Bodily Pain
                                     component
       Role Physical                  score
       General Health Perception
       Mental Health
                                        Mental
       Role Emotional                component
       Vitality                        score
       Social Functioning
   Each subscale given a 0-100 score (100 best)
 SF-36 – Primary TKA
100
 90
80                                                                                                          75 74
                                                                                71 70         70 69    69
70
                                                   58 60 60                              59
60                                       54 55
                          51 50                                    51 50   50
           48 47
50
                                                              40
40
                     27             29
30    23
20
10
 0
       Physical     Role Physical   Bodily Pain     General    Vitality      Social        Role       Mental Health
      Functioning                                   Health                 Functioning   Emotional


                     Preoperative                 1-Year      2-Year            US Norm (65-74)

 Note: Data on 1,061 at preop, 490 1-year & 243 at 2-years
  SF-36 – Primary THA
100
 90
                                                                                   77 77         78 78
80                                                                                                             75 76
70                                                                                                        66
                                                         63 62
                                          60 58     58
                           57 57                                                            55
60         52                                                         54 53
                49
50                                                                            42
40                                                               36

30                    22             23
      18
20
10
 0
       Physical      Role Physical   Bodily Pain     General      Vitality      Social        Role       Mental Health
      Functioning                                    Health                   Functioning   Emotional


                      Preoperative                 1-Year        2-Year            US Norm (65-74)

  Note: Data on 760 at preop, 387 at 1-year & 186 at 2-years
 SF-36 – Primary BHR
100                                                                                                         95
                                                                                             91
 90                             86
                83                                                                      82             82
                                               78                                                                           80
                           75                                                                                          77
80         73                                                  73
                                          70              68
70                                                                            63                  63              63
                                                     57                  58
60
50                                                                                 44
40                                                                  36
                      28
30    25                             24
20
10
 0
       Physical      Role Physical   Bodily Pain      General        Vitality        Social         Role         Mental Health
      Functioning                                     Health                       Functioning    Emotional


                      Preoperative                  1-Year          2-Year              US Norm (45-54)

 Note: Data on 174 at preop, 75 at 1-year & 35 at 2-years
Satisfaction with outcome
   Asked on follow-up questionnaires
   Patient rates how satisfied they are with the results
    of surgery:
      Overall

      Pain relief

      Ability to do ADL

      Ability to do recreational activities

   Responses are very satisfied, somewhat satisfied,
    somewhat dissatisfied and very dissatisfied
Satisfaction – Primary TKA
              100
               90
               80        75
                    71
               70
 % patients




                                                                                    Overall
               60             53
                                   46                                               Pain Relief
               50
                                                                                    ADL
               40                               33 34
               30
                                                                                    Recreation
                                        20 17
               20                                                   14
                                                         7 6   10
                                                                               6
               10                                                        2 2 4
                0
                     Very               Somewhat        Somehwat        Very
                    Satisfied            Satisfied      Dissatisfied Dissatisfied


Note: Data on 490 primary TKA patients at 1-Year
Satisfaction – Primary THA
              100
               90   83 85
               80
               70           63
 % patients




                                 58                                               Overall
               60
                                                                                  Pain Relief
               50
               40
                                                                                  ADL
                                              28 29                               Recreation
               30
               20                     13 12
                                                             7 8           5
               10                                      3 2           1 1 3
                0
                     Very             Somewhat        Somewhat        Very
                    Satisfied          Satisfied      Dissatisfied Dissatisfied


Note: Data on 388 primary THA patients at 1-Year
ICF Quiz
                    Health condition
                  (disorder or disease)


 Body Functions        Activities          Participation
 and Structures



          Environmental             Personal
             Factors                Factors
Choosing an Outcome Measure
 Formulate a precise study question
 Know your patient population
 Know what independent variables may
  affect the outcome
 Commonly used outcome measure does not
  necessarily mean it is the best measure to
  use for your study
Independent Variable Examples
 Socio-demographics
 Body mass index
 Comorbid medical conditions
 Comorbid musculoskeletal conditions
 Mental health status
 Common geriatric problems
Appropriateness
 Disease / injury to be studied
 Patient population
 Intervention
 Dimension of health likely to be affected
 Acceptability
 Feasibility
Acceptability
 Minimise burden to patients:
   Avoid distress to patients

   Reduce refusal rate

   Reduce incomplete data

 Need to consider:
   Literacy of patients

   Legibility and layout of questionnaire

   Time taken to complete
Feasibility
   Impact on clinical staff / researchers
      Staff attitudes are key determinant of data
       collection
   Data collection
      Brief questionnaires

      Lengthy questionnaires with interviews

      Postal / telephone

   Coding and entering data
   Analysis of data
Validity
 Measure what it says it does
 Content – appropriate range of items
 Face – measures what they aim to measure
 Construct – correlates to other measures
 Criterion – correlates to a ‘gold standard’
Reliability
 Measures accurately
 Internal consistency
 Test – re-test
 Inter-rater
 Intra-rater
Responsiveness
   Sensitive to detecting changes over time that are
    important to the patient

   Correlates with changes in disease severity

   Effect size index =
    (mean1 – mean2)/ pooled SD

   Standardised response mean =
    (mean1 – mean2)/ SD of change in score
Precision
 Ceiling and floor effects
 Same amount of change across different
  parts of the scale may not be equal
 Instruments that make more distinctions are
  not more precise
Interpretation
 Outcome measures produce numeric scores
 Score ranges need to be clear
 Need to know what is the smallest clinical
  meaningful changes that are perceptible to a
  patient for that particular instrument
Study Design
Study Design
 Retrospective
    Deliberate intervention

    Observational studies

 Prospective
    Deliberate intervention

    Observational studies
Study Design
   Cross-sectional studies
     Disease description

     Diagnosis and staging

        Abnormal ranges

        Disease severity

     Disease process
Study Design
   Longitudinal studies
     Prospective

        Randomised

        Non-randomised
Randomised vs. non-randomised
   RCT is seen as the ‘gold standard’
     Should ensures patients differ only by their
      exposure to the intervention
     Often difficult to generalise results due to
      inclusion/exclusion criteria
     Blinded trials in orthopaedics almost
      impossible introducing observer/patient bias
   Non-randomised trials
     Cohort and case-controlled studies

     More inclusive but may have selection bias
Study design
   For studies that compare health care
    interventions to be valid they must produce
    results that are:
      Generalisable to to other potential
       patients (high external validity)
      Estimate outcome effects that can be
       reliably attributed to the intervention
       (high internal validity)
Solutions to threats of validity
   Internal validity
      Allocation bias (confounding) – risk adjustment
       and subgroup analysis
      Patient preference – preference arms or
       adjustment for preference
   External validity
      Exclusions – expand inclusion criteria

      Non-participation – multi-centre, pragmatic
       design
      Non-participation – less rigorous consent
Potential Problems
   Recruitment
     Need to know what proportion of the reference
      population have been recruited

       Reasons for non-participation – does this group
        of patients differ from participants. Knowing
        baseline details about this group strengthens
        your study

       Volunteerism likely to be associated with age,
        gender, socioeconomic status, education etc.
        which may influence outcomes
Potential Problems
   Compliance
     Depends on how well informed patients are at
      start of trial of what is expected
       If intervention is ongoing (e.g. physical vs.
        surgical intervention) then poor compliance
        with intervention can affect statistical power
       If patients are non-compliant with parts of
        study this is preferable to total non-compliance
       Need to track all patients and document reasons
        for lost to follow-up
Statistical Support
   Speak to a statistician
     After you have reviewed the literature, selected
      your primary outcome measure and drafted a
      protocol
        Sample size calculation

        Method of data collection

        Advice about coding of data

        Analysis of data

    BUT
     Before you submit to either R&D or LREC

								
To top