Single Subject Research and Evidence-based
Interventions: Are SSDs Really the Ugly
Stepchild?
Ronnie Detrich
Randy Keyworth
Jack States
Wing Institute
The Problem
• Standards of evidence are necessary to identify evidence-
based interventions.
Allow us to evaluate the strength of evidence across studies for a
particular intervention.
• Single subject designs have not always been accepted as a
legitimate means for demonstrating the impact of an
intervention.
What Works Clearinghouse has no standards for single subject
designs.
The Problem
• In many sub-disciplines within education and psychology
single subject designs have been primary method for
identifying effective interventions.
Developmental disabilities
Autism
Severe behavior problems
• If single subject designs are not accorded scientific status
then many effective interventions will not be validated as
evidence-based interventions.
Characteristics of Single Subject Design
• The purpose of SSDs is to demonstrate a functional relation between
an independent and dependent variable.
Intense analysis of a few subjects demonstrates the functional relation.
• Reliance on visual inspection good for identifying variables that have
“whopping” effect.
Identifies socially significant effects.
• Well designed studies control for threats to internal validity.
Internal validity: the degree to which alternative explanations for the
obtained effects have been controlled for through the experimental design.
Characteristics of Single Subject Design
• Demonstrates the robustness or generality of an
independent variable through direct and systematic
replication.
Direct replication: exposing the same or different subjects to
exactly the same experimental arrangement that resulted in
identifying a functional variable.
Reversal designs both within and across subjects
Multiple baselines across subjects
Systematic replication: varying some features of the original
experimental arrangement.
Different subject characteristics, different settings, different
responses, different “doses” of the functional variable.
Generality and External Validity
• Generality and External Validity are related but distinct
concerns.
Generality describes the boundary conditions of a functional
relation.
Under what conditions does the functional relation “break down”?
External validity refers to degree to which the results of a research
study can be extended to other populations, settings, and
conditions.
Degree of external validity is always contextual. Depends on the
similarity between research and intervention conditions.
Answers actuarial questions that concern program administrators and
policy makers- “how big a bang will I get and what is the probability
of impact”?
Generality and External Validity
• Single subject designs are most often criticized because of
issues related to external validity.
In large part behavior analysts have not given much consideration
to subject characteristics.
Behavior analysts have been more concerned with establishing the
robustness of a few variables (reinforcement, stimulus control).
Body of knowledge is established through direct and systematic
replication.
As we move from the study of single variables and basic behavioral
processes to multi-component packages the distinction between
generality and external validity becomes more confused.
Benefits of Single Subject Design
• A rigorous methodology for identifying functional
variables.
• Allows scientist to see pattern of action of the variable of
interest:
Can make informed statements about:
Acquisition
Maintenance
Generalization
Benefits of Single Subject Design
• It is possible to study low incidence populations and
behaviors.
• Cost-Effective relative to group designs.
Can evaluate intervention before subjecting to large scale studies.
• Close continuous contact with the data allow for great
flexibility.
• Research can be completed by scientist-practitioner in
practice settings.
Can easily test clinical hypothesis.
Best method for progress monitoring in applied settings.
Limitations of Single Subject Design
• Does not answer “actuarial” questions related to external
validity very well.
Was not intended to answer those questions.
• Reliance on visual inspection may result in unreliable
interpretation.
There are no established standards for visually evaluating data.
Several researchers have criticized relying on visual inspection as
means of interpretation. (DeProspero & Cohen,1979) because of
relatively low agreement between observers.
Limitations of Single Subject Design
• Methods for aggregating results across studies have not
been established.
Meta-analysis approaches may be useful.
This is very important for validating interventions as evidence-based.
Practitioners and decision-makers do not have time nor access to all
of the primary source data.
• Standards for validating interventions as evidence-based
with SSDs are just emerging.
No consensus among these standards.
Examples of Standards for Single Subject
Designs
Source Number of Number of Number of Number of
Levels Studies Subjects Investigators
National Autism 6 levels Strongest N≥3 per study; Results replicated
Center (Strongest evidence=6 with minimum of 18 Ss across independent
Evidence-Decidely no conflicting over all researchers.
(proposed not results; 9 w/no
Discredited)
adopted) more than 1
conflicting study
rated as having
Strong evidence or
better.
Exceptional 3 levels 5 studies 20 subjects 3 different
Children (Research-based researchers across
Practice-Emerging 3 different
(proposed not locations.
Practice)
adopted)
Are SSDs the Ugly Stepchild?
• Should not be
As long as they are used to identify functional variables.
• But not everyone agrees:
Some excellent texts on group designs poorly describe SSDs.
• We have work to do.
Recommendations
• Develop appropriate meta-analysis methods for single
subject research.
• Develop standards for visual inspection.
Complex, very politically sensitive task.
• Work with national organizations such as What Works
Clearinghouse to assure that single subject research given
equal status to group designs.