VIEWS: 10 PAGES: 38 POSTED ON: 1/27/2012
Research Methods in Psychology Repeated Measures Designs Repeated Measures Designs Each individual participates in each condition of the experiment • completes the DV with each condition • hence “repeated measures” Also called “within-subject” design • entire experiment is conducted “within” each subject Repeated Measures Designs, continued Why Use a Repeated Measures Design? • no need to balance individual differences across conditions of experiment all participants are in each condition • fewer participants needed • convenient and efficient • more sensitive Sensitivity A sensitive experiment • can detect the effect of an independent variable • even if the effect is small Repeated measures designs are more sensitive than independent groups designs • “error variation” is reduced same people participate in each condition variability due to individual differences eliminated Practice Effects Main disadvantage of repeated measures designs is practice effects • People change as they are tested repeatedly performance may improve over time people may become bored or tired over time Practice effects become a potential confounding variable if not controlled Practice Effects, continued Example: • Suppose a researcher compares two different study methods, A and B Condition A: participants use a highlighter to mark key points while reading a text, then take a test on the material Condition B: participants read a text, then make up sample test questions and answers, then take a test on the material Practice Effects, continued • Suppose all participants first experience Condition A and then Condition B results indicate test scores are higher in Condition A compared to Condition B • Is marking text with highlighter (A) better than writing sample questions/answers (B)? impossible to know • confounding of IV with order of presentation • practice effects (boredom, fatigue) may account for poorer performance in Condition B Practice Effects, continued Practice effects must be balanced, or averaged, across conditions • Counterbalancing the order of conditions distributes practice effects equally across conditions half of the participants do Condition A, then B the remaining participants to Condition B, then A Conditions A and B then have equivalent practice effects practice effects aren’t eliminated, but they are averaged across the conditions of the experiment Counterbalancing Practice Effects Two types of repeated measures designs • Complete and Incomplete • purpose of each type of design is to counterbalance practice effects • each design uses different procedures for counterbalancing practice effects Complete Design Practice effects are balanced within each participant in the complete repeated measures design • each participant experiences each condition several times, using different orders each time • a complete repeated measures design is used when each condition is brief (e.g., simple judgments about stimuli) Complete Design, continued Two methods for generating orders of conditions • block randomization • ABBA counterbalancing Complete Design, continued Block randomization • a block consists of all conditions (e.g., 4 conditions: A, B, C, D) • generate a random order of the block (ACBD) • participant completes condition A, then C, then B, then D • generate a new random order for each time the participant completes the conditions of the experiment (e.g., DACB, CDBA, ADBD) Complete Design, continued Block randomization • balances practice effects only when conditions are presented many times • practice effects are averaged across the many presentations of the conditions • practice effects are not balanced if conditions are presented only a few times to each participant Complete Design, continued ABBA counterbalancing • used when conditions are presented only a few times to each participant • procedure: present one random sequence of conditions (e.g., DABC), then present the opposite of the sequence (CBAD) • each condition has the same amount of practice effects Complete Design, continued ABBA counterbalancing • balance practice effects that are “linear” linear practice effects • participants change in the same way following each presentation of a condition nonlinear practice effects • participants change dramatically following the administration of a condition • example: participant experiences insight about how to complete an experimental task (“aha … now I get it”) • likely to use this insight in subsequent conditions Complete Design, continued • Example of linear practice effects suppose participants gain “one unit” of practice with each administration (“trial”) of a condition • there are zero practice effects with the first administration Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6 Condition A B C C B A Practice Effects +0 +1 +2 +3 +4 +5 Practice effects are balanced because total practice effects is +5 for each condition: A: 0 + 5 B: 1 + 4 C: 2 + 3 Complete Design, continued • Example of nonlinear practice effects: Suppose a participant figures out a method for completing the task on the third trial, and then uses the new method for subsequent trials Trial 1 Trial 2 Trial 3 Trial 4 Trial 5 Trial 6 Condition A B C C B A Practice Effects +0 +1 +5 +5 +5 +5 Practice effects are not balanced across the conditions: A: 0 + 5 = 5 B: 1 + 5 = 6 C: 5 + 5 = 10 Complete Design, continued • Nonlinear practice effects create a confounding differences in scores on the DV may not be caused by the IV (conditions A, B, C) differences on DV may be due to different amounts of practice effects associated with each condition • ABBA counterbalancing should not be used when practice effects are likely to vary or change over time (i.e., nonlinear practice effects) use block randomization instead Complete Design, continued • ABBA counterbalancing should not be used when anticipation effects can occur participants develop expectations about which condition will appear next in a sequence responses may be influenced by expectations rather than actual experience of each condition if anticipation effects are likely, use block randomization Incomplete Design Each participant experiences each condition of the experiment exactly once • complete design: more than once Practice effects are balanced across participants in the incomplete design • complete design: practice effects balanced within each subject Incomplete Design, continued General rule for balancing practice effects • each condition (e.g., A, B, C) must appear in each ordinal position (1st, 2nd, 3rd) equally often • if this rule is followed, practice effects will be balanced across conditions will not confound the experiment Incomplete Design, continued Two techniques for balancing practice effects in an incomplete repeated measures design • all possible orders • selected orders Incomplete Design, continued All possible orders • use when there are four or fewer conditions • two conditions (A, B) → two possible orders: AB, BA half of the participants would be randomly assigned to do condition A first, followed by B other half of participants would complete condition B first, followed by A • three conditions (A, B, C) → six possible orders: ABC, ACB, BAC, BCA, CAB, CBA participants would be randomly assigned to one of the six orders Incomplete Design, continued • four conditions (ABCD) → 24 possible orders (ABCD, ABDC, ACBD, ACDB, ADBC, etc.) • five conditions → 120 possible orders • six conditions → 720 possible orders • at least one participant must receive each order of the conditions therefore, all possible orders is used for experiments with four or fewer conditions of the IV Incomplete Design, continued Selected orders • select particular orders of conditions to balance practice effects • two methods Latin Square random starting order with rotation • each condition appears in each ordinal position exactly once • each participant is randomly assigned to one of the orders of conditions Incomplete Design, continued • Procedure for Latin Square randomly order the conditions of the experiment (e.g., ABCD) number the conditions (A = 1, B = 2, C = 3, D = 4) use this rule for generating the 1st order: 1, 2, N, 3, N – 1, 4, N – 2, 5, N – 3, 6, etc. where N = last number of conditions • the first order of four conditions would be 1 2 4 3 Incomplete Design, continued to generate 2nd order of conditions add 1 to each number in the first order (1 2 4 3) • “N” represents the number of conditions (e.g., 4); we can’t use N + 1 because this would create a 5th condition • additional rule: N + 1 always is “1” -- the first condition • the second order of conditions is 2 3 1 4 to generate 3rd order of conditions add 1 to each number in the second order (again, N + 1 = 1) • the third order of conditions is 3 4 2 1 follow the same procedure for each subsequent order • The number of orders is the same as the number of conditions (e.g., 4 conditions → 4 orders) Incomplete Design, continued Match letters of conditions to their numbers to create the Latin Square 1st 2nd 3rd 4th 1st 2nd 3rd 4th 1 2 4 3 A B D C 2 3 1 4 B C A D 3 4 2 1 C D B A 4 1 3 2 D A C Incomplete Design, continued • Each condition appears in each ordinal position equally often, which balances practice effects For example, condition “A” appears in each ordinal position: 1st 2nd 3rd 4th A B D C B C A D C D B A D A C B Incomplete Design, continued • Another advantage of Latin Square each condition precedes and follows every other condition once (e.g., AB and BA, BC and CB) 1st 2nd 3rd 4th A B D C B C A D C D B A D A C B this helps to control for potential order effects Incomplete Design, continued Random starting order with rotation • generate a random order of conditions (e.g., ABCD) • rotate the sequence by moving each condition one position to the left each time 1st 2nd 3rd 4th A B C D B C D A C D A B D A B C each condition appears in each ordinal position to balance practice effects unlike Latin Square, order of conditions is not balanced Data Analysis of Repeated Measures Designs Complete repeated measures designs require an additional step • Because participants complete each condition many times, a summary score (e.g., mean) is computed for each participant for each condition • this represents each participant’s average performance in each condition Data Analysis, continued • Suppose two participants complete two conditions (A, B) of an experiment four times each the DV is their rating on a 1–5 scale assume IV (conditions A, B) represents two types of stimuli participants are asked to judge (e.g., size of the stimuli) Data Analysis, continued Suppose the following data are observed in an ABBA design: Condition Participant 1 Participant 2 A 2 1 B 4 4 B 5 4 A 1 1 B 3 5 A 1 2 A 2 3 B 5 5 Data Analysis, continued To analyze these data we first need to compute the average rating for each condition (A, B) for each participant: Condition Participant 1 Participant 2 A 2 1 B 4 4 B 5 4 A 1 1 B 3 5 A 1 2 A 2 3 B 5 5 A: (2+1+1+2)/4 = 1.50 A: (1+1+2+3)/4 = 1.75 B: (4+5+3+5)/4 = 4.25 B: (4+4+5+5)/4 = 4.50 Data Analysis, continued Next calculate the mean for each condition across all participants In this example with two participants, the means for conditions A and B are Condition A Condition B participant 1 1.50 4.25 participant 2 1.75 4.50 mean 1.625 4.375 Null hypothesis testing or confidence intervals would be used to determine whether this difference between means is reliable The Problem of Differential Transfer Repeated measures designs should not be used when differential transfer is possible • occurs when the effects of one condition persist and affect participants’ experience of subsequent conditions • use independent groups design instead • assess whether differential transfer is a problem by comparing results for repeated measures design and random groups design Comparison of Two Designs Differences between repeated measures design and independent groups design • Independent variable repeated measures: each participant experiences every condition of the IV independent groups: each participant experiences only one condition of the IV • What is balanced (averaged) across conditions to rule out alternative explanations for findings? repeated measures: practice effects independent groups: individual differences variables