Why I love RCT‟s
Peter Szatmari MD, Offord Centre for Child Studies, Dept of Psychiatry and Behavioural Neurosciences
Clinical Scenario
• Jane, 16 years old, admitted to ER with
suicidal ideation • 6 month history of irritability, loss of interest in friends and sports, episodes of crying, lots of family conflict • Broke up with boy-friend, took mother‟s pills, told parents who brought her to ER • Mother treated with TCA and recovered
How do I decide what to do?
• Theories of what causes MDD or • Search, find, appraise and apply the
evidence
Relying on Theory?
• What causes the problem? Deal with family
•
conflict Theories of adolescent development; conflict over independence Support of resiliency and protective factors; get her back into sports Adult and adolescent MDD on a spectrum therefore extend downwards adult treatments
•
•
What do I do? (1980‟s)
• Mother has family history and was
successfully treated with TCA; • Case series show good outcomes with TCA • Cohort studies show good outcomes
• Adolescents with mood disorder like adults • Textbook says treat with TCA
Drug Treatment of Adolescent MD
• RCT in the 80‟s show no benefit, • Risk of overdose, cardio-toxicity • 40% of adolescents with mood disorders
treated with TCA • What about SSRI‟s?
What do I do (1990‟s)?
• RCT of adolescent depression using placebo vs
• • •
fluoxetine, fluvoxemine, paroxetine, Published trials show efficacy Unpublished trials show lack of efficacy Only fluoxetine consistently effective (see Hetrick et al 2007 Selective serotonin reuptake inhibitors for depressive disorders in children and adolescents. Cochrane Database Systematic Reviews)
How can I tell if a Rx works?
• • • •
•
Case series Case control study; sample by outcome Cohort study; sample by exposure (Rx) Controlled observational study; the experimenter assigns treatment Randomized control trial; the experimenter assigns treatment in a random manner Meta-analysis or systematic review
•
Adolescent Mood Disorder
• Case-control; What treatment did adolescents
•
with MD who improved receive compared to those who did not improve? Cohort; Follow adolescents with MDD in a clinic and see what Rx those who improved received? Controlled Observation; Assign CBT and placebo to adolescents with MD based on birthday RCT; Assign CBT/SSRI and placebo to adolescents with MD based on table of random numbers
•
•
The Rationale of RCT‟s
• Change is the only constant thing • “What causes what” is a complex and
dynamic interplay of various factors • The only way to isolate the impact of a Rx is to randomize, so that, “all other things being equal”, the Rx did (or did not) work! • Confounding factors equally distributed between exposure and control groups
Confounding Variables
• Associated with the exposure • A risk factor for the outcome • Really explains the differences between
the group • Many unknown confounding variables • The weaker the evidence the greater the reported benefit
Other Design Issues
• Sample size large enough to detect
clinically relevant differences as statistically significant • Large enough to ensure groups are similar • Blinding of observers of outcome to exposure • Blinding of experimenter to randomization
Other Design Issues
• Clinically relevant outcome measures • Rx is given as stated (treatment fidelity) • Inclusion and exclusion criteria meaningful • Follow up of 80% of the sample • Missing subjects accounted for • Intention to treat analysis
Effect sizes; Categorical Data
• NNT=the number of patients who must
receive the experimental treatment to create one additional improved outcome compared with the control treatment • Calculated by 1/ABI; where ABI=EER-CER • Eg; 60% vs 20%; ABI=40% • NNT=1/.4=2.5 , roughly 3
Categorical Data
• Relative Risk; 60% vs 20% RR=3.0 • 6% vs 2% RR=3.0 • Same RR different NNT • ABI=4% • NNT; 1/.04=25
Comparing NNT‟s
• • • • • •
EBM Vol 10, 16 sig. outcomes Average NNT=57.1 (range 4-250), median=50 EBMH vol 8 #1&2; 22 with sig results Average NNT=6.9 (range 2-32), median= 5 EBM Rx‟s=mostly meds, mostly rare events EBMH Rx‟s=meds, CBT and psychosocial programs, outcomes mostly common events
Effect Sizes of Continuous Data
• Mean scores as continuous measures • ES=Mean score (Exp)-Mean Score (Cont)
divided by Pooled SD • Difference between groups in terms of SD‟s • ES>.5 is clinically significant
The Future of RCT‟s
• From efficacy to effectiveness trials • Large simple trials • Mediation studies • Equivalence studies • NNH • Cluster randomization and community
interventions
Common Criticisms
• You can‟t generalize from that study to my
patient! Each patient is unique! • Patients expect you to individualize treatments • APA (Feb 2005); “we warn against using only these research reviews…in developing public policies that drive access to treatments”
BUT…..
• It‟s the „best available evidence” • The alternative? Informed consent and
shared decision making • Patients/families EXPECT us to use scientific evidence in decision making
Randomized Trials not Always Possible or Needed
• Cannot assign patients to an intervention;
too much risk not enough potential benefit • Intervention carries no risk • Events are too rare; suicidal ideation • Community interventions
Conclusion
• Truth is a quantitative construct, evidence
less dependent on context • RCT‟s allow us to learn the “error of our ways” • Evidence of benefit is exaggerated; be skeptical