Docstoc

Statistics in Clinical Trials for Non-statisticians - Bulletin of the NYU

Document Sample
Statistics in Clinical Trials for Non-statisticians - Bulletin of the NYU Powered By Docstoc
					The analysis of longitudinal studies
        in rheumatology:
               Is it done correctly?
     6th Annual Clinical Research Methodology
                      Course
                       Friday, December 16, 2011
                            Emmanuel Lesaffre
    Department of Biostatistics, Erasmus MC, Rotterdam, the Netherlands
                    L-Biostat, K.U.Leuven, Leuven, Belgium

                            In collaboration with
        Karolina Sikorska, Maurits de Rotte, Jolanda Luime, Mieke Hazes




                                                                          1
                           Conclusions
•   Many longitudinal studies in rheumatology are not using an appropriate
    statistical methodology.

•   Repeated significance tests and summary statistics methods are
    inappropriate in practice and can not show the evolution of an outcome
    over time.

•   Classical repeated measurement methods are inappropriate in practice.

•   The mixed effects model and the GEE approach can make use of all
    available cases independently whether they have data on all time points
    studied.

•   The mixed effects model and the GEE approach are sparsely used in
    longitudinal studies rheumatology.



                                                                             2
Description JIA study




                        3
         Juvenile Idiopathic Arthritis
                 (JIA) study
• Description
   – Longitudinal study: 5 time points (0, 3, 6, 9, 12 months)
   – Recruitment at University Medical Centre Utrecht (UMCU) and
     Wilhelmina Children’s Hospital, the Netherlands
   – Children diagnosed with JIA according to the ILAR criteria and started
     methotrexate (MTX) therapy between 1990 and 2006
   – All patients and their parents gave their informed consent and approved
     by the medical ethics committee of the UMCU
   – Treatment was tight-controlled using a standardized report form on
     disease activity every 3 months
   – Information on MTX usage, disease activity, route of administration,
     dosing of MTX, reasons for ending MTX treatment, concomitant therapy
     and laboratory parameters were collected
                                                                          4
                           JIA study
• Description
   – 183 JIA patients of our test cohort were genotyped for ABCB1
     3435C>T.
   – 2 groups: genotype CC homozygous (53 patients, 28%) ó T-carrier
     (129 patients, 72%)
   – Continuous response = erythrocyte sedimentation rate (ESR) was
     obtained on all 5 visits for 106/183 patients (58%): CC homozygous (31
     patients, 29%), T-carrier (75 patients, 71%) = complete cases
   – ESR was log-transformed because of a skewed distribution.
   – Binary response = ACR30 response, obtained for all 183 patients on all
     visits. Of the 183 patients, 46 (25%) responded at 3 months, 85 (46%)
     at 6 months, 110 (60%) at 9 months and 113 (62%) at 12 months.



                                                                          5
Classical analyses for ESR




                             6
                       JIA study
• Classical analysis (ESR)




                                   7
            Classical approaches
      for comparing 2 groups over time

•   Repeated significance tests (t-tests, Wilcoxon tests, etc)

•   Summary Statistic Method: summarizing whole curve by e.g. AUC
    and compare AUC between 2 groups with t-test

•   Repeated measurements ANOVA

•   MANOVA




                                                                    8
1. Repeated significance tests




                                 9
        1. Repeated significance tests
                          JIA study (ESR)


• Here unpaired t-tests giving P= 0.066, 0.12, 0.62, 0.33, 0.44

• Nowhere significant Þ no real issue here

• Suppose    P= 0.066, 0.03, 0.62, 0.33, 0.44 … what to conclude?
• Or         P= 0.066, 0.12, 0.62, 0.33, 0.03 … what to conclude?

• If no adjustment of P-value, then problem of multiple testing




                                                                  10
        1. Repeated significance tests

• Popular approach in medical research

• But

   – Inefficient: only patients are included that are present at visit

   – Likely to be biased: missing data process can bias analysis

   – No good insight in individual evolution: link between subsequent
     observations is lost

   – Turns longitudinal study into several cross-sectional studies

• Not suitable for contemporary rheumatology studies


                                                                         11
                         JIA study
• All individual profiles (ESR)

         CC homozygous               T-carrier




                                                 12
                        JIA study
• 3 individual profiles (ESR)




                                    13
          2. Summary Statistic Method
                            JIA study (ESR)



• Compute Area-under-the-Curve (AUC) for each profile

• On complete cases only:
   CC-Homozygous – T-carrier= -1.33, P = 0.54, 95% CI=(-5.61; 2.96)

• Complete the incomplete cases by LOCF:
   CC-Homozygous – T-carrier= -3.78, P = 0.04, 95% CI=(-7.33; -0.21)

• In addition one could compare
   – Averages of the profiles
   – ….

                                                                      14
                        JIA study
• 3 individual profiles (ESR): incomplete profiles completed by
  Last-Observation-Carried-Forward (LOCF)




                                                              15
         2. Summary Statistic Method
• Not frequently used in medical research

• Recommended for balanced data
   – Balanced: all patients are examined at all (regular) visits

   – Difficult for unbalanced data, i.e. when there are missing
     observations and patients can come at all times

   – LOCF is a popular approach to make data more balanced

• LOCF
   – Imputes unrealistic values

   – Underestimates variability of the data

• Not suitable for contemporary rheumatology studies
                                                                   16
   3. Repeated Measurements ANOVA

• One of only 2 approaches available for analysis of repeated
  measurements 50 years ago

• Requires balanced data
   – Patients with missing values are excluded

   – Time points must be regular, patients with irregular time points are
     excluded

   – Restrictive assumptions on correlation between subsequent responses:
     correlations are equal (corrections available but not sufficient)

   – Splits treatment effect up into time, group and group*time effect

• Not suitable for contemporary rheumatology studies

                                                                         17
    3. Repeated Measurements ANOVA
                            JIA study (ESR)

• On complete cases only:
   – CC-Homozygous – T-carrier=
       • Time effect:            P < 0.001
       • Genotype effect:        P=0.47
       • Time*genotype effect:   P=0.02

• Complete the incomplete cases by LOCF:
   – CC-Homozygous – T-carrier=
       • Time effect:            P < 0.001
       • Genotype effect:        P=0.04
       • Time*genotype effect:   P=0.34


                                              18
    3. Repeated Measurements ANOVA
                                JIA study (ESR)

• Assumption equal correlation??


         TIME (months)    0         3      6       9     12


              0           1        0.71   0.51    0.46   0.42

              3          0.71       1     0.61    0.47   0.48

              6          0.51      0.61    1      0.70   0.64

              9          0.46      0.47   0.70     1     0.69

              12         0.42      0.48   0.64    0.69    1




                                                                19
                          4. MANOVA

• Other of only 2 approaches available for analysis of repeated
  measurements 50 years ago

• Requires balanced data
   – Patients with missing values are excluded

   – Time points must be regular, patients with irregular time points are
     excluded

   – No structure allowed on correlations Þ large studies are required

   – Splits treatment effect up into time, group and group*time effect

   – Output more difficult to understand

• Not suitable for contemporary rheumatology studies
                                                                         20
                            4. MANOVA
                            JIA study (ESR)

• On complete cases only:
   – CC-Homozygous – T-carrier=
       • Time effect:            P < 0.001
       • Genotype effect:        P=0.47
       • Time*genotype effect:   P=0.03

• Complete the incomplete cases by LOCF:
   – CC-Homozygous – T-carrier=
       • Time effect:            P < 0.001
       • Genotype effect:        P=0.04
       • Time*genotype effect:   P=0.36


                                              21
Modern analyses for ESR




                          22
            “Modern” approaches
      for comparing 2 groups over time

•   Types of missing data processes

•   Approaches to deal with missing data

•   Mixed effects models

•   GEE models




                                           23
    1. Types of missing data processes

• What are missing data?
   • Data that are not observed

• How are missing data generated?
   • Patients/clinicians forget to fill item(s) in questionnaire
   • Patients refuse to fill item(s) in questionnaire
   • Lost or damaged biological sample
   • Patients miss a visit because …
   • Patients decide not to return to clinician anymore
   • Patient died
   • …


Þ Different reasons why data are missing
                                                                   24
               Impact of missing data

• What is believed
   • Loss of efficiency

• In reality
   • Loss of efficiency
   • Biased results (often)

Þ Classical statistical methods need to be adapted

• Approach
   • CLEVER explicit or implicit imputation of missing data
   • But: can NEVER replace true data
     But


                                                              25
                       Terminology
                                                 1   2   3   4
•   Monotone missing
    • Also called dropouts                   1

                                             2

                                             M

                                             M

                                             n



•   Non-monotone missing                         1   2   3   4
    • Also called intermittent missingness
                                             1

                                             2

                                             M

                                             M

                                             n
                                                                 26
                More terminology

• Missing completely at random

• Missing at random

• Missing not at random – informative missing



• Terminology might be confusing – Rubin (1975)




                                                  27
Missing completely at random (MCAR)
• Probability of missingness is independent of all responses

• Examples
   • A random selection of teeth in mouth are taken in the study
   • A blood tube is dropped
   • Patient died in a car accident, but careful: patient could have
     experienced a sleep attack when taking a dopamine agonist

• Then
   • Simple mean of response is unbiased estimate of true mean
   • Classical statistical techniques (repeated t-tests, Summary statistics
     method, repeated measurements ANOVA, MANOVA) can be used

• Impact: loss of efficiency
                                                                         28
             Missing at random (MAR)
• Probability of missingness depends on observed responses

• Examples
   • Study design specifies that if blood pressure is not lowered patient will
     be removed from anti-hypertensive trial
   • Multi-stage screening: data are missing at subsequent stages due to
     result at initial stage (negative test)

• Then
   • Simple mean of response is biased estimate of true mean
   • Classical statistical techniques canNOT be used
   • Statistical tests to distinguish versus MCAR exist

• Impact: loss of efficiency + bias, but likelihood analysis can
  correct for bias due to missingness
                                                                                 29
       Missing not at random (MNAR)
• Probability of missingness depends on observed responses &
  unobserved responses

• Examples
   • Patient shows a flare up in the disease unobserved in the study +
     patient decides to leave the study

• Then
   • Simple mean of response is biased estimate of true mean
   • Classical statistical techniques canNOT be used
   • No test to distinguish versus MAR
   • There is no satisfactory analysis, ONLY sensitivity analysis

• Impact: loss of efficiency + bias, only a sensitivity analysis
  can shed light on the problem
                                                                         30
                  Bias of simple mean




COMPLETE   2.07    4.41   6.87   9.03    11.57   13.30
MCAR       2.14    4.07   6.50   8.53    10.95   12.49
MAR        2.07    4.41   7.41   10.30   17.14   20.22
                                                         31
   2. Approaches to deal with missing
                  data
• Prevention and planning

• Analytical remedies
   • Complete case analysis
   • Available case analysis
   • Imputation techniques
   • Likelihood-based analyses
   • Weighted analyses
   • Sensitivity analyses

Þ Most appropriate statistical solutions are COMPLICATED


                                                           32
   2. Approaches to deal with missing
                  data
• Complete case analysis (MCAR)
   – Default analysis in many packages, only ok for MCAR
   – If not MCAR: substantial bias can be the result

• Available case analysis (MCAR)
   – Use for each variable/time point all observations available

• Single value imputation (MCAR, MAR??)
   – Mean value imputation
   – Hot decking
   – LOCF
   methods are based on unrealistic models & underestimate variance
                                                                   33
Example: LOCF

           LOCF




    time

                  34
   2. Approaches to deal with missing
                  data
• Multiple imputation (MAR)

   – Explicit imputation of missing data

   – Incorporate random mechanism

   – Generate M different completed imputed data sets

   – Combine M means and M variances Þ 1 overall mean & variance

   method is based on statistical model for imputation




                                                               35
   2. Approaches to deal with missing
                  data
• Likelihood-based models (MAR)
   – Implicit imputation of missing values

   – Model-based:
       • Linear & generalized linear regression

       • Linear & generalized linear mixed models

       • …

   method is based on statistical model for response

• GEE models (MCAR) and weighted GEE models (MAR)

• Bayesian models (MAR)

                                                       36
  2. Approaches to deal with missing
                 data
• MNAR models

  – Model also missing data mechanism

  – Complex modelling

  – Never completely satisfactory Þ sensitivity analysis

  – BUT: if time points are close to each then MNAR close to MAR




                                                                   37
          3. Mixed effects models

• Two examples




                                    38
           3. Mixed effects models

• Assumption in mixed effects models




                                       39
               3. Mixed effects models
• Not enough used YET in medical research

• Recommended for contemporary rheumatology studies
   – Allows unbalanced data

   – None of the patients are deleted from the study

   – Irregular time points are allowed

   – No explicit imputation of responses

   – Time evolution, effect of covariates and correlation structure: all can be
     flexible and modelled

• BUT
   – One needs statisticians for the job

   – Luckily: they are OFTEN cheap
                                                                              40
             3. Mixed effects models

• Linear mixed effects models
   – Response = continuous

   – Normality assumptions

• Generalized linear mixed effects models
   – Response = continuous or discrete

   – Normality assumptions




                                            41
             3. Mixed effects models
                           JIA study (ESR)

• On ALL cases:
   – CC-Homozygous – T-carrier=
      • Time effect:            P < 0.001
      • Genotype effect:        P=0.03
      • Time*genotype effect:   P=0.09
      • + ESTIMATES and 95% CIs




                                             42
3. Mixed effects models
      JIA study (ESR)




                          43
                      4. GEE models

• Can cope only with MCAR

• Makes only few assumptions about distribution of data

• Most popular for discrete responses

• Can also be used for continuous response




                                                          44
                       4. GEE models
                          JIA study (ACR30)

• Binary response

• No missing data in the study

• Possible analyses
   – Chi-square tests (Fisher’s exact tests) at each time point

   – Generalized mixed effects model

   – GEE model

• Same comments as before

• Here all analyses gave non-significant difference between genotype
  groups
                                                                  45
Literature review




                    46
                      Literature review
•   Literature search (criteria) :
    – PubMed
    – January 1- December 31, 2008
    – Annals of the Rheumatic Diseases & Arthritis and Rheumatism
    – Patients with arthritis were followed up in time over > 3 measurements
    – Response that could vary over time

•   Information collected:
    –   Research aim
    –   Primary outcome, secondary outcomes
    –   Used technique for handling with missing data
    –   Statistical analysis techniques used

•   Eligible studies were ranked according to degree they correctly
    analyzed the longitudinal research question
                                                                           47
                      Literature review
Study characteristics
•   203 longitudinal studies in ARD or in AR
•   156 out of these studies dealt with arthritis
    – 11 studies with JIA patients
    – 58 studies described a RCT
    – 98 studies described a longitudinal cohort study
•   110 studies were included (excluded: 30 studies not longitudinal, 1
    study was not first published in 2008, 16 studies did not include
    patients with arthritis)

Statistical techniques used
•   110 studies were ranked
•   17 made optimal use of modern statistical methods
                                                                     48
              Literature review


                              Description                         Number of studies
Ranking


            Suitable for repeated measurement analysis                         110


   1      Repeated character of the data not taken into account                  78
   2         Use of Bonferroni correction for multiple testing                    7
   2       Use of Repeated Measurements ANOVA or MANOVA                           8
   3      Appropriate repeated measurement techniques used on                   17
                       all suitable research questions




                                                                                  49
Conclusions




              50
                           Conclusions
•   Many of the longitudinal studies in rheumatology are not using an
    appropriate statistical methodology.

•   Repeated significance tests and summary statistics methods are
    inappropriate in practice and can not show the evolution of an outcome
    over time.

•   Classical repeated measurement methods are inappropriate in practice.

•   The mixed effects model and the GEE approach can make use of all cases
    independently whether they have data on all time points studied.

•   The mixed effects model and the GEE approach are sparsely used in
    longitudinal studies rheumatology.




                                                                            51
Thank you for your attention




                               52

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:5/9/2014
language:Unknown
pages:52