An Evidence-Based Approach to Clinical Practice in Communication Disorders
Chris Dollaghan
University of Pittsburgh dollagha@csd.pitt.edu
2004 Donalda J. McGeachy Memorial Lecture University of Toronto May 28, 2004
Evidence-Based Practice
…the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients . . . [by] integrating individual clinical expertise with the best available external clinical evidence from systematic research.
(Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996: 71)
N. B
Evidence is never enough (Guyatt et
al., 2000)
Not all evidence is relevant to clinical questions: Basic research into underlying mechanisms of disease is a necessary but insufficient guide to clinical practice (EBM Working Group, 1992)
Why EBP?
The first diagnostic manual (Kraemer & Sprenger, 1487; cited in Meehl, 1997)
How could well-meaning experts, using best technical evidence of their age, be so wrong? Clinical decisions were based on belief, opinion, and past experience that had not been subjected to systematic, unbiased, and reproducible (i.e., scientific) test. Francis Bacon’s observation
(1620; cited in Meehl, 1997):
The human understanding, once it has adopted opinions, either because they were already accepted and believed, or because it likes them, draws everything else to support and agree with them.
Starting point for EBP
Acknowledgement that humans inherently prefer what they already believe, and thus that strong evidence cannot come without strong tests of such beliefs and preferences, regardless of the eminence of those holding them
Clinical experience and expert opinion can be important starting points, but must be tested systematically, in an unbiased fashion, rather than being accepted at face value Even (especially!) when they “stand to reason”
Seven steps to EBP
(adapted from Sackett et al., 2000)
1.
2.
3. 4. 5.
6.
7.
Formulate a foreground question Find best current evidence Critical appraisal Evidence summary via CAT Decide whether the evidence is strong enough to influence your clinical practice Integrate the evidence with the “intangibles” Update!
Step 1: Formulate a foreground question
The foreground-background distinction Background questions
– ask for general knowledge about a condition, via a question word (who, what, where, when, how, why) – are more frequent when our experience with a disorder is limited – relative proportion of background questions decreases as clinical experience increases
Foreground questions
The focus for EBP Ask a quite specific, answerable question about the best way to diagnose, “prognose,” or treat Have four essential components (PICO)
Formulating a foreground question: PICO (SIGN group, 2000)
P (the patient and/or problem of interest) I (the intervention, defined broadly to encompass clinical decisions about diagnosing, treating, prognosticating, etc.) C (the comparison intervention) O (the clinical outcome of interest)
Clinical question about treatment
– In toddlers with delayed language, does parent-administered treatment result in significantly greater language gains than no treatment?
Clinical question about diagnosis
– Does Test A identify four-year-olds
with language disorders significantly more accurately than a language sample analysis?
Step 2: Find ostensibly best current evidence (i.e., candidate evidence)
Cost-benefit ratio for various potential sources of evidence (e.g., colleagues, consensus statements by respected authorities, notes from classes or workshops, textbooks, professional journals, previous experience with similar cases, current studies in highyield journals)
Some ways of finding high quality evidence, or letting it find you
PubMed demonstration: www.pubmed.com (Please do the tutorial!)
– Clinical query function » Dx » Tx » Meta-analysis or systematic review – Cubby
www.guideline.gov and other regular updates Guiltless reliance on other sources if strong evidence does not exist
Finding ostensibly best evidence means knowing how to judge evidence. . .
Step 3: Critical appraisal
(Sackett et al. 2000) Applying explicit criteria to judge evidence quality Criteria vary slightly according to whether evidence comes from a study of diagnosis, therapy, prognosis, meta-analysis, etc. Three consistent themes
– Validity (avoid bias and confounding) – Importance (effect sizes and impact) – Precision (confidence intervals)
Evaluating validity: An overview
Avoiding confounding
– Design (prospective, controlled, random assignment to groups) – Full disclosure of patient enrollment and loss
Avoiding bias
– Blinding (masking, concealment) – Unblinded studies of treatment have effects an average of 40% larger than blinded studies (Schulz & Chalmers, 1995)
Design types and definitions (note that validity depends on more than design)
Experimental: patients are enrolled prospectively, randomized to group/condition, and some variable is controlled (i.e., some manipulation is imposed by the investigator) Quasi-experimental: prospective but not randomized or controlled Non-experimental: patients are identified retrospectively, no manipulation of variables
Randomized, controlled trial (RCT)
Study is designed before patients are enrolled Patients are assigned at random to one of the groups to be compared. Pros: If randomization is accomplished correctly, best odds that the groups being compared do not differ systematically on some variable other than the one of interest (treatment vs. no-treatment) Cons: Costly, time-consuming, in some cases may raise ethical issues; potential for volunteer bias
Quasi-experimental: prospective but without random assignment
E.g., Cohort study: Patients with and without a variable of interest are identified and followed forward in time to compare their outcomes Pros: Ability to study low-incidence and/or late-emerging problems efficiently Cons: Exposure may be linked to a hidden confounder; blinding is difficult
Non-experimental studies
Cross-sectional/correlational: Group is observed at a single point in time; exposure and outcome are determined simultaneously
– Pros: Cheap, ethically safe – Cons: Causality can’t be established, susceptible to recall bias, confounders may not be equally distributed
Case-control: Patients with and without an outcome are identified post hoc, and previous exposure to a variable of interest is compared
– Pros: May be the only feasible method for very rare disorders, or if there’s a long lag between exposure and outcome – Cons: Recall bias, reliance on retrospective records to determine exposure, potential confounding
Weakest non-experimental designs
Case series: No control group
– Pros: Readily accessible to practitioners – Cons: Nothing improves results like no control group (Barrett-Connor, 2002) due to difficulty of controlling for bias and confounders; generalizability unknown.
Case report: N = 1
– Pros: Readily accessible – Cons: See above, and limited external validity (generalizability)
Weaker study designs can be critically important in the early stages of investigation of a clinical question (e.g., Robey, 2003) We just need to recognize their limitations for making strong inferences about clinical decisions
Evaluating validity II: Avoiding bias
Bias is virtually inescapable even when inadvertent Cues resulting in unblinding can be very subtle Can affect randomization Can affect evaluation
– Treating clinician can’t be the evaluator of the patient – Blinded evaluators should rate variety of cases and controls, without knowing which is which – Blinding might require ratings of , e.g., deidentified recordings randomized to ensure that stage of treatment cannot be inferred from them
Example 1: Adequate blinding?
Tx target: increased social participation as indexed by number of utterances Participants were assessed initially by the clinician, and then randomly assigned to receive either no treatment or 3 treatment sessions/week After 12 weeks, all were re-assessed and the tx group was found to talk significantly more than control group
No: Subtle (or not-so-subtle) expectations of child’s performance based on experiences in treatment with the clinician could affect his or her evaluation of outcome.
Example 2: Adequate blinding?
Study of two therapies to improve vocal quality Patients were identified prospectively and randomly assigned to receive Tx 1 or Tx 2 Observers blinded to group assignment rated vocal quality prior to treatment phase, after final treatment session, and for a sample obtained 3 months after treatment phase
No: Time of measurement (pre-tx, immediate post-tx, and long post-tx) was not concealed from raters, so their expectations of tx impact could have inadvertently influenced their ratings
Example 3: Adequate blinding?
Clinicians in early intervention settings were invited to refer patients with a diagnosis of childhood apraxia of speech to the study All referred children were assessed with the new test, by an examiner who was unaware of the child’s referring site.
Not if evaluator knew that all had been diagnosed previously – expectations associated with evaluating a child with known apraxia could bias evaluation (e.g., cause increased focus on expected symptoms and decrease attention to other characteristics).
End of overview of first theme: evaluating validity: On to second theme (evaluating importance)