Research Methods in Psychology

Document Sample
Research Methods in Psychology Powered By Docstoc
					Research Methods in Psychology

   Repeated Measures Designs
Repeated Measures Designs

 Each individual participates in each
  condition of the experiment
  • completes the DV with each condition
  • hence “repeated measures”
 Also called “within-subject” design
  • entire experiment is conducted “within” each
Repeated Measures Designs, continued

 Why Use a Repeated Measures Design?
  • no need to balance individual differences
    across conditions of experiment
     all participants are in each condition
  • fewer participants needed
  • convenient and efficient
  • more sensitive

 A sensitive experiment
  • can detect the effect of an independent
  • even if the effect is small
 Repeated measures designs are more
  sensitive than independent groups designs
  • “error variation” is reduced
      same people participate in each condition
      variability due to individual differences eliminated
Practice Effects

 Main disadvantage of repeated measures
  designs is practice effects
  • People change as they are tested repeatedly
      performance may improve over time
      people may become bored or tired over time
 Practice effects become a potential
  confounding variable if not controlled
Practice Effects, continued

 Example:
  • Suppose a researcher compares two different
    study methods, A and B
     Condition A: participants use a highlighter to mark
      key points while reading a text, then take a test on
      the material
     Condition B: participants read a text, then make up
      sample test questions and answers, then take a
      test on the material
Practice Effects, continued

  • Suppose
      all participants first experience Condition A and
       then Condition B
      results indicate test scores are higher in Condition
       A compared to Condition B
  • Is marking text with highlighter (A) better than
    writing sample questions/answers (B)?
      impossible to know
        • confounding of IV with order of presentation
        • practice effects (boredom, fatigue) may account for
          poorer performance in Condition B
Practice Effects, continued
 Practice effects must be balanced, or
  averaged, across conditions
  • Counterbalancing the order of conditions
    distributes practice effects equally across
      half of the participants do Condition A, then B
      the remaining participants to Condition B, then A
      Conditions A and B then have equivalent practice
      practice effects aren’t eliminated, but they are
       averaged across the conditions of the experiment
Counterbalancing Practice Effects

 Two types of repeated measures designs
  • Complete and Incomplete
  • purpose of each type of design is to
    counterbalance practice effects
  • each design uses different procedures for
    counterbalancing practice effects
Complete Design

 Practice effects are balanced within each
  participant in the complete repeated
  measures design
  • each participant experiences each condition
    several times, using different orders each time
  • a complete repeated measures design is used
      each condition is brief (e.g., simple judgments
       about stimuli)
Complete Design, continued

 Two methods for generating orders of
  • block randomization
  • ABBA counterbalancing
Complete Design, continued

 Block randomization
  • a block consists of all conditions (e.g., 4
    conditions: A, B, C, D)
  • generate a random order of the block (ACBD)
  • participant completes condition A, then C,
    then B, then D
  • generate a new random order for each time
    the participant completes the conditions of the
    experiment (e.g., DACB, CDBA, ADBD)
Complete Design, continued

 Block randomization
  • balances practice effects only when
    conditions are presented many times
  • practice effects are averaged across the many
    presentations of the conditions
  • practice effects are not balanced if conditions
    are presented only a few times to each
Complete Design, continued

 ABBA counterbalancing
  • used when conditions are presented only a
    few times to each participant
  • procedure: present one random sequence of
    conditions (e.g., DABC), then present the
    opposite of the sequence (CBAD)
  • each condition has the same amount of
    practice effects
Complete Design, continued

 ABBA counterbalancing
  • balance practice effects that are “linear”
      linear practice effects
        • participants change in the same way following each
          presentation of a condition
      nonlinear practice effects
        • participants change dramatically following the
          administration of a condition
        • example: participant experiences insight about how to
          complete an experimental task (“aha … now I get it”)
        • likely to use this insight in subsequent conditions
Complete Design, continued

    • Example of linear practice effects
         suppose participants gain “one unit” of practice
          with each administration (“trial”) of a condition
            • there are zero practice effects with the first administration

                Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6
   Condition       A      B       C       C       B       A
Practice Effects +0      +1      +2      +3      +4      +5

    Practice effects are balanced because total practice effects is +5
      for each condition:
                 A: 0 + 5       B: 1 + 4         C: 2 + 3
Complete Design, continued

    • Example of nonlinear practice effects:
         Suppose a participant figures out a method for
          completing the task on the third trial, and then uses
          the new method for subsequent trials

                Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6
   Condition       A      B       C       C       B       A
Practice Effects +0      +1      +5      +5      +5      +5

    Practice effects are not balanced across the conditions:
                 A: 0 + 5 = 5    B: 1 + 5 = 6    C: 5 + 5 = 10
Complete Design, continued

  • Nonlinear practice effects create a
     differences in scores on the DV may not be caused
      by the IV (conditions A, B, C)
     differences on DV may be due to different amounts
      of practice effects associated with each condition
  • ABBA counterbalancing should not be used
     when practice effects are likely to vary or change
      over time (i.e., nonlinear practice effects)
     use block randomization instead
Complete Design, continued

  • ABBA counterbalancing should not be used
    when anticipation effects can occur
     participants develop expectations about which
      condition will appear next in a sequence
     responses may be influenced by expectations
      rather than actual experience of each condition
     if anticipation effects are likely, use block
Incomplete Design

 Each participant experiences each
  condition of the experiment exactly once
  • complete design: more than once
 Practice effects are balanced across
  participants in the incomplete design
  • complete design: practice effects balanced
    within each subject
Incomplete Design, continued

 General rule for balancing practice effects
  • each condition (e.g., A, B, C) must appear in
    each ordinal position (1st, 2nd, 3rd) equally
  • if this rule is followed, practice effects
      will be balanced across conditions
      will not confound the experiment
Incomplete Design, continued

 Two techniques for balancing practice
  effects in an incomplete repeated
  measures design
  • all possible orders
  • selected orders
Incomplete Design, continued

 All possible orders
  • use when there are four or fewer conditions
  • two conditions (A, B) → two possible orders: AB, BA
      half of the participants would be randomly assigned to do
       condition A first, followed by B
      other half of participants would complete condition B first,
       followed by A
  • three conditions (A, B, C) → six possible orders:
      participants would be randomly assigned to one of the six
Incomplete Design, continued

  • four conditions (ABCD) → 24 possible orders
    (ABCD, ABDC, ACBD, ACDB, ADBC, etc.)
  • five conditions → 120 possible orders
  • six conditions → 720 possible orders
  • at least one participant must receive each
    order of the conditions
     therefore, all possible orders is used for
      experiments with four or fewer conditions of the IV
Incomplete Design, continued

 Selected orders
  • select particular orders of conditions to
    balance practice effects
  • two methods
      Latin Square
      random starting order with rotation
  • each condition appears in each ordinal
    position exactly once
  • each participant is randomly assigned to one
    of the orders of conditions
Incomplete Design, continued

  • Procedure for Latin Square
     randomly order the conditions of the experiment
      (e.g., ABCD)
     number the conditions (A = 1, B = 2, C = 3, D = 4)
     use this rule for generating the 1st order:
      1, 2, N, 3, N – 1, 4, N – 2, 5, N – 3, 6, etc.
           where N = last number of conditions
       • the first order of four conditions would be 1 2 4 3
Incomplete Design, continued

     to generate 2nd order of conditions add 1 to each
      number in the first order (1 2 4 3)
       • “N” represents the number of conditions (e.g., 4); we
         can’t use N + 1 because this would create a 5th condition
       • additional rule: N + 1 always is “1” -- the first condition
       • the second order of conditions is 2 3 1 4
     to generate 3rd order of conditions add 1 to each
      number in the second order (again, N + 1 = 1)
       • the third order of conditions is 3 4 2 1
     follow the same procedure for each subsequent
       • The number of orders is the same as the number of
         conditions (e.g., 4 conditions → 4 orders)
Incomplete Design, continued

        Match letters of conditions to their numbers to
         create the Latin Square

 1st           2nd 3rd      4th           1st    2nd
              3rd  4th
 2            4      3             A      B      D         C
 3            1      4             B      C      A         D
 3             4     2      1             C      D         B
 4             1     3      2             D      A         C
Incomplete Design, continued

  • Each condition appears in each ordinal
    position equally often, which balances
    practice effects
     For example, condition “A” appears in each ordinal
            1st 2nd 3rd        4th
            A   B      D       C
            B   C      A       D
            C   D      B       A
            D   A      C       B
Incomplete Design, continued

  • Another advantage of Latin Square
     each condition precedes and follows every other
      condition once (e.g., AB and BA, BC and CB)
          1st     2nd 3rd       4th
          A       B      D      C
          B       C      A      D
          C       D      B      A
          D       A      C      B

     this helps to control for potential order effects
Incomplete Design, continued

 Random starting order with rotation
  • generate a random order of conditions (e.g., ABCD)
  • rotate the sequence by moving each condition one position to
    the left each time
               1st     2nd     3rd     4th
               A       B       C       D
               B       C       D       A
               C       D       A       B
               D       A       B       C
       each condition appears in each ordinal position to balance
        practice effects
       unlike Latin Square, order of conditions is not balanced
Data Analysis
of Repeated Measures Designs
 Complete repeated measures designs
  require an additional step
  • Because participants complete each condition
    many times, a summary score (e.g., mean) is
    computed for each participant for each
  • this represents each participant’s average
    performance in each condition
Data Analysis, continued

  • Suppose
     two participants complete two conditions (A, B) of
      an experiment four times each
     the DV is their rating on a 1–5 scale
     assume IV (conditions A, B) represents two types
      of stimuli participants are asked to judge (e.g., size
      of the stimuli)
Data Analysis, continued

     Suppose the following data are observed in an
      ABBA design:
     Condition   Participant 1   Participant 2
          A              2               1
          B              4               4
          B              5               4
          A              1               1
          B              3               5
          A              1               2
          A              2               3
          B              5               5
Data Analysis, continued

     To analyze these data we first need to compute the average
      rating for each condition (A, B) for each participant:
     Condition      Participant 1    Participant 2
           A                 2                1
           B                 4                4
           B                 5                4
           A                 1                1
           B                 3                5
           A                 1                2
           A                 2                3
           B                 5                5
                 A: (2+1+1+2)/4 = 1.50        A: (1+1+2+3)/4 = 1.75
                 B: (4+5+3+5)/4 = 4.25        B: (4+4+5+5)/4 = 4.50
Data Analysis, continued

      Next calculate the mean for each condition across
       all participants
      In this example with two participants, the means
       for conditions A and B are
               Condition A     Condition B
  participant 1 1.50             4.25
  participant 2 1.75             4.50
                 mean 1.625               4.375

      Null hypothesis testing or confidence intervals
       would be used to determine whether this difference
       between means is reliable
The Problem of Differential Transfer

 Repeated measures designs should not
  be used when differential transfer is
  • occurs when the effects of one condition
    persist and affect participants’ experience of
    subsequent conditions
  • use independent groups design instead
  • assess whether differential transfer is a
    problem by comparing results for repeated
    measures design and random groups design
Comparison of Two Designs

 Differences between repeated measures design
  and independent groups design
  • Independent variable
      repeated measures: each participant experiences every
       condition of the IV
      independent groups: each participant experiences only one
       condition of the IV
  • What is balanced (averaged) across conditions to rule
    out alternative explanations for findings?
      repeated measures: practice effects
      independent groups: individual differences variables

Shared By: