PowerPoint Presentation by 90wJ25m

VIEWS: 0 PAGES: 35

									Outline

1. Threats to validity
2. Our goal – control
3. 3 ways of achieving control:
  i.     Statistical control
  ii.    Experimental control
  iii.   Replication
Threats to validity

Earlier, we discussed           • Now, we’ll look at some
threats to the validity of a      sophisticated solutions to
conclusion, including threats     validity problems
to:

• internal validity
• construct validity, and
• external validity
Our goal is control

We want to eliminate every     • Every – every plausible
plausible alternative            alternative must be ruled
explanation for our results.
                                 out – if you allow any to
    – every                      survive, you don’t know
                                 whether your theory is
                                 right.
Our goal is control

We want to eliminate every     • Plausible – a judgment,
plausible alternative            one the science
explanation for our results.
                                 community makes
    – every
    – plausible
Our goal is control

We want to eliminate every     • Alternative explanation –
plausible alternative            not the one you are
explanation for our results.
                                 advocating
    – every
    – plausible
    – alternative
How do we do this?

Three basic ways of    • Statistical control
countering plausible   • Experimental control
alternatives:          • Replication
Types of Control

  – Statistical control

• Experimental control

• Replication
Statistical control - definition

Statistical control       • Equating people on
means equating              paper requires use of
subjects on paper when      sophisticated
they cannot be equated      statistical analysis
in fact.                    techniques, beyond
                            the scope of this
                            course
Statistical Control – basic idea

• Any behavior has          • These characteristics
many characteristics          can be produced by a
                              variety of influences
  – Is it fast? accurate?
  kind? absent-minded?         – IQ, motivation,
                                education, caffeine in
                                the blood, etc.
Statistical Control – basic idea
                                                      Age
• Statistical control
                                      Instruction
requires measuring and         IQ

removing the effects of
such influences when
they are not of interest.


                        Caffeine


                                                    Mood
                                    Time of day
Statistical control - example

• Suppose you want to     • What do you do if
compare two methods         GPA distributions for
of instruction              the classes are
• You have two 1st year     different?
university classes to
work with, one for each
of the methods you’re
comparing
A note about distributions

• A distribution shows
  how frequently scores   Class 1
  occur along the
  dimension being
  measured
                                Class 2
• Here, the dimension
  shown on the X-axis
  is GPA
Statistical control - example

• What do you do if GPA • Remove from the
distributions for the     performance scores
classes are different?    the “contribution” due
                          to GPA
• Measure GPA           • Remaining variability
• Measure performance     in performance
                          scores now has
                          nothing to do with
                          GPA
Types of Control

• Statistical control

  – Experimental control

• Replication
Experimental control

Experiments give us control,   • They establish the
in two senses:                   equivalence of two units
                                 (groups, subjects,
                                 conditions) before testing.

                               • They constrain and guide
                                 sources of variability.
Types of Control

• Statistical control

  – Experimental control
     (1) Establishing equivalence
     (2) Constraining variability


• Replication
Types of Control

•    Experimental control

    (1) Establishing equivalence
       A. Between-groups approaches
       B. Within-subjects approaches


    (2) Constraining variability
Between groups approaches

• Subjects serve in         C   E

  either a control group
  or a treatment group,
  not both
Between groups approaches

• How do we make groups   – Random assignment
  equivalent before       – Matching
  treatment?
Match subjects in treatment groups if…

(i) There is a variable which   • influences performance
    you believe :                 on your task
                                • varies among subjects
                                • can be controlled by
                                  matching
                                • is not of interest to you
Match subjects in treatment groups if…

(ii) You can measure your      – First put matched
 subjects on this variable       subjects in sets
 before you start collecting   – Then assign sets of
 data.                           matched subjects one
                                 to each treatment
                                 group.
                               – This assignment
                                 should be random
Group 1        Group 2        Group 3




          89     92      90


          111 112 110
Types of Control

•    Experimental control

    (1) Establishing equivalence
       A. Between-groups approaches
       B. Within-subjects approaches


    (2) Constraining variability
Within subjects approaches

• Each subject serves in all   – Advantage – best
conditions, acting as their      possible control
own control                    – Disadvantage –
                                 practice effects
                               – See Chapter 11 for
                                 more.
Types of Control

• Statistical control

   – Experimental control

      (1) Establishing equivalence
      (2) Constraining variability


• Replication
Constraining variability

We can control aspects of      A. Research preparation
the research situation other
than how subjects are             – What task do subjects
assigned to conditions:            do?
The research preparation

McBurney (p. 193)

“[O]ne of the researcher’s goals is to choose
the most suitable preparation for studying a
given problem. Some of the most important
contributions to psychology have been made by
people who devised a new preparation for
studying a given phenomenon .”
Research preparation examples

Semantic priming             • Preparation: subject
                               reads pairs of two words
• Used to study the            in quick succession,
  structures and processes     responding only to the
  of our stored knowledge
  of the world.                second in each pair.
                             • RT and accuracy vary
                               with semantic relationship
                               between the two words.
Research preparation examples

Neglect                   • Standard neurological
                            test for neglect.
  – neuropsychological
    disorder in which     • Victor Mark’s modification
    patients ignore one
    half of space and
    everything in it.
Constraining variability

We can control aspects of      A. Research preparation
the research situation other
than how subjects are             – What task do subjects
assigned to conditions:            do?

                               B. Instrumentation
                                  – how is performance
                                   measured?
Instrumentation

• Psychophysics             • For 100 years, we
                              believed that stimulus
  – the study of how          intensity (S) was related
    perceptual experience     to perceived intensity (I)
    varies with the           logarithmically:
    stimulus
                               – S = k log I
Instrumentation

• Stevens developed new     • This new method
  Magnitude Estimation        demonstrated that, in
  procedure:
                              fact:
  – subjects assign
    numbers to stimuli on      – S = a In
    basis of how intense
    they appear to be.
Types of Control

• Statistical control

• Experimental control
   – Establishing equivalence
   – Constraining variability


   – Replication
Replication

Finally, one way to deal with     • Will results generalize to
threats to external validity is     other classes of people?
to replicate, with new
subjects and materials. This
lets you answer these             •   Will results generalize to
important questions:                  other types of stimuli?
Summary

• When possible, we exert      • When such control is not
  control during data            possible, we control
  collection, by
   – randomly assigning          nuisance variability
     subjects                    statistically, by equating
   – using subjects as their     subject groups on paper.
     own controls
   – controlling stimulus      • When statistical control is
     presentation.
                                 not possible, we control
                                 through replication.

								
To top