Document Sample
SSD Powered By Docstoc
					Single Subject Research and Evidence-based
  Interventions: Are SSDs Really the Ugly
                Ronnie Detrich
               Randy Keyworth
                 Jack States
                Wing Institute
                         The Problem

• Standards of evidence are necessary to identify evidence-
  based interventions.
    Allow us to evaluate the strength of evidence across studies for a
     particular intervention.
• Single subject designs have not always been accepted as a
  legitimate means for demonstrating the impact of an
    What Works Clearinghouse has no standards for single subject
                        The Problem

• In many sub-disciplines within education and psychology
  single subject designs have been primary method for
  identifying effective interventions.
    Developmental disabilities
    Autism
    Severe behavior problems
• If single subject designs are not accorded scientific status
  then many effective interventions will not be validated as
  evidence-based interventions.
     Characteristics of Single Subject Design

• The purpose of SSDs is to demonstrate a functional relation between
  an independent and dependent variable.
     Intense analysis of a few subjects demonstrates the functional relation.
• Reliance on visual inspection good for identifying variables that have
  “whopping” effect.
     Identifies socially significant effects.
• Well designed studies control for threats to internal validity.
     Internal validity: the degree to which alternative explanations for the
      obtained effects have been controlled for through the experimental design.
    Characteristics of Single Subject Design

• Demonstrates the robustness or generality of an
  independent variable through direct and systematic
    Direct replication: exposing the same or different subjects to
     exactly the same experimental arrangement that resulted in
     identifying a functional variable.
        Reversal designs both within and across subjects
        Multiple baselines across subjects
    Systematic replication: varying some features of the original
     experimental arrangement.
        Different subject characteristics, different settings, different
         responses, different “doses” of the functional variable.
         Generality and External Validity
• Generality and External Validity are related but distinct
    Generality describes the boundary conditions of a functional
        Under what conditions does the functional relation “break down”?
    External validity refers to degree to which the results of a research
     study can be extended to other populations, settings, and
        Degree of external validity is always contextual. Depends on the
         similarity between research and intervention conditions.
        Answers actuarial questions that concern program administrators and
         policy makers- “how big a bang will I get and what is the probability
         of impact”?
         Generality and External Validity
• Single subject designs are most often criticized because of
  issues related to external validity.
    In large part behavior analysts have not given much consideration
     to subject characteristics.
    Behavior analysts have been more concerned with establishing the
     robustness of a few variables (reinforcement, stimulus control).
        Body of knowledge is established through direct and systematic
        As we move from the study of single variables and basic behavioral
         processes to multi-component packages the distinction between
         generality and external validity becomes more confused.
        Benefits of Single Subject Design

• A rigorous methodology for identifying functional
• Allows scientist to see pattern of action of the variable of
    Can make informed statements about:
        Acquisition
        Maintenance
        Generalization
        Benefits of Single Subject Design
• It is possible to study low incidence populations and
• Cost-Effective relative to group designs.
    Can evaluate intervention before subjecting to large scale studies.
• Close continuous contact with the data allow for great
• Research can be completed by scientist-practitioner in
  practice settings.
    Can easily test clinical hypothesis.
    Best method for progress monitoring in applied settings.
      Limitations of Single Subject Design

• Does not answer “actuarial” questions related to external
  validity very well.
    Was not intended to answer those questions.
• Reliance on visual inspection may result in unreliable
    There are no established standards for visually evaluating data.
        Several researchers have criticized relying on visual inspection as
         means of interpretation. (DeProspero & Cohen,1979) because of
         relatively low agreement between observers.
      Limitations of Single Subject Design

• Methods for aggregating results across studies have not
  been established.
    Meta-analysis approaches may be useful.
        This is very important for validating interventions as evidence-based.
        Practitioners and decision-makers do not have time nor access to all
         of the primary source data.
• Standards for validating interventions as evidence-based
  with SSDs are just emerging.
    No consensus among these standards.
    Examples of Standards for Single Subject
Source            Number of           Number of            Number of          Number of
                  Levels              Studies              Subjects           Investigators
National Autism   6 levels            Strongest            N≥3 per study;     Results replicated
Center            (Strongest          evidence=6 with      minimum of 18 Ss   across independent
                  Evidence-Decidely   no conflicting       over all           researchers.
(proposed not                         results; 9 w/no
adopted)                              more than 1
                                      conflicting study
                                      rated as having
                                      Strong evidence or
Exceptional       3 levels            5 studies            20 subjects        3 different
Children          (Research-based                                             researchers across
                  Practice-Emerging                                           3 different
(proposed not                                                                 locations.
           Are SSDs the Ugly Stepchild?

• Should not be
    As long as they are used to identify functional variables.

• But not everyone agrees:
    Some excellent texts on group designs poorly describe SSDs.

• We have work to do.

• Develop appropriate meta-analysis methods for single
  subject research.
• Develop standards for visual inspection.
    Complex, very politically sensitive task.
• Work with national organizations such as What Works
  Clearinghouse to assure that single subject research given
  equal status to group designs.

Shared By: