# PowerPoint Presentation by 90wJ25m

VIEWS: 0 PAGES: 35

• pg 1
```									Outline

1. Threats to validity
2. Our goal – control
3. 3 ways of achieving control:
i.     Statistical control
ii.    Experimental control
iii.   Replication
Threats to validity

Earlier, we discussed           • Now, we’ll look at some
threats to the validity of a      sophisticated solutions to
conclusion, including threats     validity problems
to:

• internal validity
• construct validity, and
• external validity
Our goal is control

We want to eliminate every     • Every – every plausible
plausible alternative            alternative must be ruled
explanation for our results.
out – if you allow any to
– every                      survive, you don’t know
right.
Our goal is control

We want to eliminate every     • Plausible – a judgment,
plausible alternative            one the science
explanation for our results.
community makes
– every
– plausible
Our goal is control

We want to eliminate every     • Alternative explanation –
plausible alternative            not the one you are
explanation for our results.
– every
– plausible
– alternative
How do we do this?

Three basic ways of    • Statistical control
countering plausible   • Experimental control
alternatives:          • Replication
Types of Control

– Statistical control

• Experimental control

• Replication
Statistical control - definition

Statistical control       • Equating people on
means equating              paper requires use of
subjects on paper when      sophisticated
they cannot be equated      statistical analysis
in fact.                    techniques, beyond
the scope of this
course
Statistical Control – basic idea

• Any behavior has          • These characteristics
many characteristics          can be produced by a
variety of influences
– Is it fast? accurate?
kind? absent-minded?         – IQ, motivation,
education, caffeine in
the blood, etc.
Statistical Control – basic idea
Age
• Statistical control
Instruction
requires measuring and         IQ

removing the effects of
such influences when
they are not of interest.

Caffeine

Mood
Time of day
Statistical control - example

• Suppose you want to     • What do you do if
compare two methods         GPA distributions for
of instruction              the classes are
• You have two 1st year     different?
university classes to
work with, one for each
of the methods you’re
comparing

• A distribution shows
how frequently scores   Class 1
occur along the
dimension being
measured
Class 2
• Here, the dimension
shown on the X-axis
is GPA
Statistical control - example

• What do you do if GPA • Remove from the
distributions for the     performance scores
classes are different?    the “contribution” due
to GPA
• Measure GPA           • Remaining variability
• Measure performance     in performance
scores now has
nothing to do with
GPA
Types of Control

• Statistical control

– Experimental control

• Replication
Experimental control

Experiments give us control,   • They establish the
in two senses:                   equivalence of two units
(groups, subjects,
conditions) before testing.

• They constrain and guide
sources of variability.
Types of Control

• Statistical control

– Experimental control
(1) Establishing equivalence
(2) Constraining variability

• Replication
Types of Control

•    Experimental control

(1) Establishing equivalence
A. Between-groups approaches
B. Within-subjects approaches

(2) Constraining variability
Between groups approaches

• Subjects serve in         C   E

either a control group
or a treatment group,
not both
Between groups approaches

• How do we make groups   – Random assignment
equivalent before       – Matching
treatment?
Match subjects in treatment groups if…

(i) There is a variable which   • influences performance
• varies among subjects
• can be controlled by
matching
• is not of interest to you
Match subjects in treatment groups if…

(ii) You can measure your      – First put matched
subjects on this variable       subjects in sets
before you start collecting   – Then assign sets of
data.                           matched subjects one
to each treatment
group.
– This assignment
should be random
Group 1        Group 2        Group 3

89     92      90

111 112 110
Types of Control

•    Experimental control

(1) Establishing equivalence
A. Between-groups approaches
B. Within-subjects approaches

(2) Constraining variability
Within subjects approaches

• Each subject serves in all   – Advantage – best
conditions, acting as their      possible control
practice effects
– See Chapter 11 for
more.
Types of Control

• Statistical control

– Experimental control

(1) Establishing equivalence
(2) Constraining variability

• Replication
Constraining variability

We can control aspects of      A. Research preparation
the research situation other
than how subjects are             – What task do subjects
assigned to conditions:            do?
The research preparation

McBurney (p. 193)

“[O]ne of the researcher’s goals is to choose
the most suitable preparation for studying a
given problem. Some of the most important
contributions to psychology have been made by
people who devised a new preparation for
studying a given phenomenon .”
Research preparation examples

Semantic priming             • Preparation: subject
• Used to study the            in quick succession,
structures and processes     responding only to the
of our stored knowledge
of the world.                second in each pair.
• RT and accuracy vary
with semantic relationship
between the two words.
Research preparation examples

Neglect                   • Standard neurological
test for neglect.
– neuropsychological
disorder in which     • Victor Mark’s modification
patients ignore one
half of space and
everything in it.
Constraining variability

We can control aspects of      A. Research preparation
the research situation other
than how subjects are             – What task do subjects
assigned to conditions:            do?

B. Instrumentation
– how is performance
measured?
Instrumentation

• Psychophysics             • For 100 years, we
believed that stimulus
– the study of how          intensity (S) was related
perceptual experience     to perceived intensity (I)
varies with the           logarithmically:
stimulus
– S = k log I
Instrumentation

• Stevens developed new     • This new method
Magnitude Estimation        demonstrated that, in
procedure:
fact:
– subjects assign
numbers to stimuli on      – S = a In
basis of how intense
they appear to be.
Types of Control

• Statistical control

• Experimental control
– Establishing equivalence
– Constraining variability

– Replication
Replication

Finally, one way to deal with     • Will results generalize to
threats to external validity is     other classes of people?
to replicate, with new
subjects and materials. This
lets you answer these             •   Will results generalize to
important questions:                  other types of stimuli?
Summary

• When possible, we exert      • When such control is not
control during data            possible, we control
collection, by
– randomly assigning          nuisance variability
subjects                    statistically, by equating
– using subjects as their     subject groups on paper.
own controls
– controlling stimulus      • When statistical control is
presentation.
not possible, we control
through replication.

```
To top