Microsoft PowerPoint - experimental design b_a ch 9 by tyndale


                                                           Creation of meaningful comparisons
                                                              Deliberate manipulation of independent variables
                                                              Take advantage of “natural” manipulations to create
           Experimental Designs                               comparisons
                                                               • Ex post facto studies
                                                               • Quasi-experimental manipulations
                Claudia J. Stanny                             Control of extraneous variables
           PSY 6217 – Research Design                          • Random assignment of subjects to groups
                                                               • Maintain at a constant value
                                                               • Counterbalancing

      Sources of Variability in Data                                       Error Variability
                                                            Ideally, error variability is distributed randomly across
Systematic Variability                                     conditions in an experiment
  Variability attributed to the effects of manipulations      Error variability is evaluated & controlled within the
  of an independent variable                                  statistical analysis of the data
  Treatment Variability                                    Confounded Variables
Error Variability                                             Extraneous variables that are correlated with
  Variability attributed to the effects of extraneous         manipulations of an independent variable
  variables                                                   Problem: Confounds confuse variation due to
   • Individual differences (subject variability)             treatments with variation due to these extraneous
   • Error variability of measures used (when                 variables
     reliability of a measure is less than 1.00)              This variability cannot be reliably isolated using
                                                              statistical procedures

        Between-Subjects Designs
                                                           Costs associated with matching
 Two-group versus parametric (multi-group)                   Time and expense related to finding matching subjects
designs                                                      Loss of potential subjects (unable to match)
 Problem of non-equivalent groups                            Loss of generalizability of results
                                                             Problems created by attrition after matching
 Matched-groups Designs
                                                             Loss of degrees of freedom in the statistical analysis
  Matched pairs (precision matching)
  Matched groups (frequency distribution matching)
                                                           Advantages associated with matching
                                                             Control of extraneous subject variables
                                                             Potential for increased sensitivity to the effects of
                                                             manipulated variables (when matching variables are
                                                             related to the dependent measure)

              Within-Subjects Designs                                  Advantages of Within-Subjects Designs

  “Ultimate” Match – each subject is matched                            Efficient in the recruitment and use of subjects
 with itself
                                                                        Good control of individual differences as an
  Each subject participates in every condition in                      extraneous variable
 the design
                                                                        Statistical advantage
  Individual differences (subjects) are now                               Potential for increased sensitivity to effects of
 treated as an additional factor in the statistical                       treatment
 analysis (Repeated Measures Designs)                                     Individual differences are no longer part of error
                                                                          variance, but are identified as variance attributed to
                                                                          the Subjects variable

Disadvantages of Within-Subjects Designs                                      Minimizing Carryover Effects
Time required for subject participation
Subject attrition                                                       Allow subject behavior to stabilize before
  Loss of data (e.g., from equipment failures) in one condition will   exposure to any data collection conditions
  require discarding all data for that subject                            Create room for adaptation or habituation to occur
Carryover effects                                                      Practice trials
  Practice                                                                Give enough practice before data collection begins
  Fatigue                                                                 to familiarize with task and procedures
  Habituation or sensitization to manipulations                         Create breaks to offset effects of boredom or
  Adaptation                                                           fatigue
  Contrast effects                                                      Counterbalance presentation of conditions
  Irreversible changes

                   Counterbalancing                                                 Block Randomization
                                                                       Each subject completes
  Complete counterbalancing                                            all four conditions          Subject   Order of Conditions
     All possible orders of conditions used                            4n subjects required for
  Partial counterbalancing                                             the design                     1       A    C     B     D
     Block Randomization                                                Controls effects of order
                                                                       in the sequence
     Latin Square Design                                                                              2       C    B     D     A
                                                                          Practice, boredom, etc.
     Randomization (requires many sequences)                           Does not control the
                                                                                                      3       B    D     A     C
                                                                       effects of unique effects
                                                                       of one condition on the
                                                                       following condition            4       D    A     C     B

                                                                                               Choice of Design
                   Latin Square Design
                                                                          Within-Subjects Designs
                                                                           Large individual differences: Subject variables are
 Each subject completes                                                    correlated with performance on the dependent measure
 all four conditions & 4n     Subject    Order of Conditions
                                                                           Economic use of subjects
 subjects required for the
                                                                           Interest in practice or order effects as manipulations
 design                         1        A        B       D          C
                                                                           requires use of a within-subjects design
  Controls effects of order
 in the sequence                2        B        C           A      D    Matched Groups Designs
  Controls the unique                                                      Need to control individual differences but carryover
 effects of context or          3        C        D           B      A     effects are a serious concern
 contrast created by                                                      Between-Subjects Designs
 experimental conditions        4        D        A           C      B     Participation in conditions requires extensive time
                                                                           Concern over carryover effects

                    Factorial Designs                                     Three Hypotheses Tested in a Two Factor
  Two or more independent variables                                        What is the effect of
  Test Hypothesis about Main Effect of each IV                             Noise Intensity?
 separately (as occurs in single factor designs)                              Main Effect of Noise
 Test Hypothesis about Interaction Effects                                                               Two Factor Experiment
                                                                           What is the effect of
                                                                           Noise Predictability?                        Noise Intensity
Single Factor Experiment:        Two Factor Experiment                        Main Effect of
     Noise Intensity                                                          Predictability           Predictability   Soft    Loud
                                                  Noise Intensity
                                                                           Interaction of Noise and
   Soft              Loud     Predictability       Soft           Loud     Predicability                Predictable     Grp 1   Grp 2     X
                               Predictable        Grp 1           Grp 2       Interaction Effect
                                                                              Cell Means
                                                                                                       Unpredictable    Grp 3   Grp 4     X
 Group 1            Group 2
                              Unpredictable       Grp 3           Grp 4
                                                                                                                          X       X

          Notation for Factorial Designs                                                    Factorial Designs

  Identifying the number                              Factor A             Between Subjects
 of factors in the design      Factor B           A1               A2         n x n design requires n x n independent groups
  Identifying the number                                                   Within Subjects (Repeated Measures)
                                    B1            A1B1            A2B1
 of levels for each factor
                                    B2                                        n x n design requires each subject to do n x n tasks
 in the design                                    A1B2            A2B2
    2 x 2 design                                                           Mixed Designs
    3 x 3 design
                                                   Factor A
                                                                              One or more factors manipulated within-subjects
    2 x 3 design                             A1          A2         A3        Other factors manipulated between subjects
  All designs are expected
                                B1        A1B1 A2B1 A3B1
 to be fully crossed
                                B2        A1B2 A2B2 A3B2
                                B3        A1B3 A2B3 A3B3

          Higher Order Designs

 Three or more Factors
 Test one main effect for each factor
 Test all possible combinations of two-way
 Test all possible combinations of higher-order
interactions (three-way, four-way, etc.)
 Problem in interpretation of higher-order


To top