Group 2 – Threats to
Internal Validity
Dossier White
Saroya Rosales
Stephanie Jones
Carmen Davis
Paul Ayers
Mortality
Defined as participants dropping out of the study
between the pretest and the posttest
Also known as Subject/ Participant Attrition
Effect:
May cause the make-up of the group to be different at stages
of measurement.
This can create a false treatment effect further disabling the
researcher’s ability to obtain valid conclusions.
Reduces statistical power of your study (More participants =
stronger claim)
Mortality
Example:
If you are evaluating a marriage counseling program, and a
number of couples drop out, the effect you observe at the
end of the program may be due to the fact that the couples
who dropped out were those who were not willing to make
an effort, while those who stayed really wanted to work their
problems out.
A positive effect may simply indicate that people who are
willing to make an effort are more likely to resolve their
marital problems.
Mortality
Causes:
Underestimation of commitment involved to study
Time
Effort
Overestimation of benefits to be received from
involvement
Unexpected events in a person’s life
Medical/health
Wage/Work
Familial
Mortality
Ways to reduce participant attrition include:
Effective informed consent procedures
Regular contact with participants
Incentives
Ways to minimize effect of participant attrition
Large sample sizes (losing 5 of 300 has much less of
an effect than losing 5 of 30)
Lessen statistical rigor of the study (must be done on
the front end)
Diffusion of Treatment
Defined as…
a condition when a control group within a study
cannot be prevented from receiving the experimental
treatment
a social interaction threat which can lead to
misinterpretation of the cause-effect relationship.
Effect: Skews the results in such a way as to
make the control group appear more similar to
the experimental group at the end of the study
than they were at the beginning
Diffusion of Treatment
Example:
A study of an experimental treatment where a group of African
Americans were given instructions on modifying their diet and
exercise behavior to lower their blood pressure.
Another group was randomly assigned from the population to act
as controls -- meaning they would not receive instruction.
The evaluation team measured blood pressure in the treatment
group and in the control group before the program was
implemented.
Control group members began changing their exercise and dietary
behavior.
They had learned of the new diet and exercises from the members
of the treatment group. The two groups did not differ significantly
on their posttest results.
Diffusion of Treatment
Reduce possibility of Diffusion of Treatment
Confidentiality agreements
Keep participants anonymous from each other
Geographical distance between groups
Interactions with Selection
Defined as interaction of the selection with any
of the other threats to internal validity
Commonly found in studies as Selection –
Maturation Interaction
Interactions with Selection
Example:
Suppose that subjects in two comparison groups differ with
respect to the independent variable and a subject-related
variable such as age.
Suppose also that the dependent variable is measured twice
for each group, once at Time A and later at Time B, and that
the independent variable is introduced in the interim.
If the change in scores on the dependent measure from Time
A to Time B differs between the two groups, this discrepancy
may be due to the independent variable or to distinctive
naturally occurring developmental processes for the two age
categories that comprise the two comparison groups.
Interactions with Selection
Example:
Two groups of diabetic patients may have similar
disease indicators at the start of a study, yet a
treatment effect could result if a larger percentage of
patients in whom an effect of maturation (e.g.,
progressive worsening of disease) is more prevalent
are assigned to one group.
In this case the participants’ medical history may
have an interaction with their selection for the study
Interactions with Selection
Reducing Threat of Interaction with Selection
Randomized assignment
References
Medscape Today -
http://www.medscape.com/viewarticle/414875_3
Dealing with threats to internal validity that randomization does
not rule out -
http://www.jstor.org/view/0013189x/ap040143/04a00040/0?fr
ame=noframe&userID=8de1bde3@memphis.edu/01c0a848670
050f2afb&dpi=3&config=jstor
Mitchell, M. and Jolley, J. (2001). Research Design Explained
(4th Ed) New York:Harcourt.
H. R. Bernard. (2005). Research Methods in Anthropology, 4 th
Ed. Altamira Press.