Within subjs by dkkauwe

VIEWS: 4 PAGES: 52

									PY206: Statistical Methods for Psychology

Within-Subjects Designs

Within-subjects designs
 M&D, Chapter 11  Howell, Chapter 14  Last modified: 11/29/04

Slide 1

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Correlated errors

Slide 2

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Correlates errors
 If i1 and i2 are indeed correlated in the population, we have a problem
 Critical assumption of independence is violated  ANOVA is not robust to such violation

 Consider the following hypothetical data from Howell
Treatment
Subject
1 2 3 4 Mean

1
2 10 22 30 16

2
4 12 29 31 19

3
7 13 30 34 21

Mean
4.33 11.67 27 31.67 18.67
Luiz Pessoa, Brown University © 2004

Slide 3

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Correlated errors
 Treatment differences appear relatively small  Large variability within treatment  Differences among subjects produces most of the variability within treatments  We need to remove subject differences to have a better (and smaller) estimate of error  Basic intuition:

Yij  Yij  Yi*
Slide 4
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Partitioning total variability
 All we need to do is to partition SStotal such that we can remove SSbetween subj

 SSbetween subj can be thought of as a regular main effect
 Remaining variability can then be further partitioned
Total variation
Between subjects Within subjects Between treatments
Slide 5

Error

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Model for a = 2
 Our old full model was
Yij     j   ij

 We could write this in two parts (for two treatments)
Yi1    1   i1 Yi 2     2   i 2

 By subtracting Yi2 – Yi1 we have
Di     i

 where Di  Yi 2  Yi1;    2  1;  i   i 2   i1
Slide 6
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Model for a = 2
 Original model required two scores from each subject  New model requires one score per subject
 Each subject contributes only one observation of   Removes the dependency among the errors

Slide 7

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Model for a = 2
 The original null hypothesis was
H 0 : 1   2  0

 The new null hypothesis is H0 :   0

 New model should be evaluated relative to restricted one. Under the null hypothesis (restricted model)
Di  0   i

Slide 8

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Model for a = 2
Full: Di     i Restricted: 0   i

 Least-squares estimates are needed  For the full model, LS estimate of  is D

 To compare models
SSE ( F )   ( Di  D ) 2 SSE ( R )   ( Di  0) 2  ( Di ) 2
i i i

Slide 9

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Model for a = 2
 As usual
( SSE ( R)  SSE ( F )) /(df R  df F ) F SSE ( F ) / df F

( SSE ( R)  SSE ( F )) /[(n  (n  1)]) F SSE ( F ) /(n  1)

Slide 10

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Model for a = 2
 With some manipulation
nD 2 F 2 sD
2 sD 

Di2  nD 2  n 1

 Where s2D is an estimate of the population variance of the D scores
 F(1,n–1) can also be written as a t with n – 1 df

Slide 11

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Model for a = 2
 Dependent-sample t test

D t sD / n
 Summary: to take into account between-subject variability, we compare difference scores

Slide 12

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

When a > 2
 With more than two groups, analysis is far from straightforward

 Multivariate approaches are usually more adequate although the univariate mixed-model approach is more prevalent (simpler)  Simpler strategy is to treat the problem as a twofactor design: repeated measures design
 One factor represents experimental condition  Second factor represents subjects

Slide 13

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Two-factor design

Slide 14

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Models
 Full model has both treatment and subject effects
Yij     j   i   ij

 Null hypothesis is
H 0 : 1   2  ...   a  0

 So the restricted model has no treatment effect
Yij     i   ij

Slide 15

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Models
 We can estimate the treatment and subject effects from the data
 Averaging across subjects i  Averaging across treatments j

ˆ  j  Y* j  Y**

ˆ  i  Yi*  Y**

 This means that for the full model ˆ ˆ ˆ ˆ Yij     j   i
 Y**  (Y* j  Y** )  (Yi*  Y** )  Y* j  Yi*  Y**
Slide 16
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Models
 And for the restricted model, only subject effect exists
ˆ ˆ ˆ Yij     i  Yi*

 Degrees of freedom: Yij     j   i   ij
df F  na  (# independent pars)  na  [1  (a  1)  (n  1)]  (n  1)(a  1) df R  na  (# independent pars)  na  [1  (n  1)]  n(a  1)
df R  df F  a  1
Slide 17
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example: data

Slide 18

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example: Full model
 Column (condition) and row (subject) differences allowed

Slide 19

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example: Restricted model
 Row (subject) differences allowed, but not column (condition) differences
 Difference between conditions is treated as sampling error  If conditions do differ, full model would provide a better description (fit) to the data

Slide 20

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example: Errors for the full model

Slide 21

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example: Errors for the restricted model

Slide 22

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Error: Full model
 If we remember that

ˆ ˆ ˆ ˆ Yij     j   i  Y* j  Yi*  Y**
 Then
n n

ˆ SSE ( F )   (Yij  Yij ) 2
i 1 j 1 n n

  (Yij  Y* j  Yi*  Y** ) 2
i 1 j 1

interaction  Thus

SSE ( F )  SS AS
Slide 23
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Error: restricted model
 For the restricted model
ˆ SSE ( R)   (Yij  Yij ) 2
i 1 j 1 n n n n

  (Yij  Y* j  Yi*  Y** ) 2   n(Y* j  Y** ) 2
i 1 j 1 j 1

a

interaction

treatment

 Thus
SSE ( R )  SS AS  SS A

Slide 24

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

F ratio
 Thus
SSE ( R)  SSE ( F )  ( SS AS  SS A )  SS AS  SS A

 We can then use usual ratio for the condition effect:
MSA MSA F  MSE MSAS

Slide 25

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Idea
 Why should this ratio inform us about the null hypothesis?

 MSAS: index of the extent to which the A effect varies from subject to subject
 Variability indicates that differences in means are less consistent across individuals

 Ratio reflects the average magnitude of condition differences relative to the inconsistency of those differences across subjects  F would be large to the extent that there were consistent condition differences from subject to subject
Slide 26
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Variability

Total variation
Between subjects Within subjects Between treatments

Error

AS

Slide 27

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Degrees of freedom

an – 1
n–1 a–1 n(a – 1) (n – 1)(a – 1)

Slide 28

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Assumption
 Homegeneity of treatment-difference variances
 Differences Yl – Ym must have the same population variance for every pair of levels

 The variance of the difference is given by
2 2 2  Y Y   Y   Y  2cov(Yl , Ym )
l m l m

2 2   Yl   Ym  2 lm Yl  Ym

Slide 29

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Assumption
 Assumption is met when population covariance matrix has compound symmetry
 CS: If an only if all the variances are equal to each other and all the covariances are equal to each other

Slide 30

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Assumption
 When compound symmetry holds
2  Y Y   2   2  2   2 2 (1   )
l m

 That is, variance of the difference does not depend on the particular levels l and m  Compound symmetry implies that homogeneity of treatment-difference variances assumption holds  Technically, CS is more than needed for assumption to hold
 Population covariance matrix must meet sphericity
Slide 31
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example

Slide 32

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example
 Sample variances are: 187.96, 200.51, 177.96, 217.86
 Rather similar; differences likely reflect sampling variability

 However, correlations vary considerably: 0.47 to 0.85

Slide 33

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example
 Correlations lead to lack of homogeneity in the variances of the differences between pairs of levels

 When the homogeneity of difference-variances does not hold, ANOVA is not appropriate
 Type I error can be as high as 0.10-0.15 (not 0.05).

Slide 34

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Adjusted tests
 Theoretical work by Box indicates that adjustments can be made if the degree of departure from homogeneity is determined
 Correction factor   1 is applied to both numerator and denominator df

ˆ  Greenhouse-Geisser correction 
 Properly controls Type I error, but conservative

 Huynh-Feldt correction    ˆ  
 Tends to overestimate  (set to max of 1.0)  Increase in power, but slight increase in Type I error rate
Slide 35
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Adjusted tests
 “Unadjusted mixed-model test should never be used because it is extremely sensitive to the sphericity assumption”
 For a nominal  of 0.01 actual level can be as high as 0.06

 “Use  because on occasion  will fail to properly ˆ  control for Type I error rate”  Comparisons between means are strongly dependent on sphericity, and are more sensitive than test for main effect
 Contrast  of 0.05 can be as high as 0.70
Slide 36
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Advantages of within-subjects designs
 Fewer number of subjects
 n subjects generate na data points  na subjects needed in between-subjects design

 Increased power
 Individual differences are explicitly modeled
They do not end up in the error term

 Similar to analysis of covariance

Slide 37

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Number of subjects
 Consider a two-group case (good approximation)
N B (1   ) NW  2
  is the population correlation between scores at the two levels of the within-subjects design  Even when  is zero, half as many subjects are needed  Term (1 – ) reflects the benefit of using each subjects as her own control

Slide 38

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Number of subjects

0.0 0.3 0.5 0.7 between within 128 64 128 45 128 32 128 20

 When compound symmetry holds, for a levels
N B (1   ) NW  a
Slide 39
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Comparison between designs
 Carryover effects can severely bias treatment effects
 Drug wash-out  Training, learning, habituation, etc

 Within-subjects design appropriate when effects are (likely) temporary

 Between/within designs might be answering slightly different questions
 Within: subject experiences manipulation in the context of other manipulations  Between: individual subject only experiences one manipulation
Slide 40
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

One between- and one within-subjects variable
 Example from Howell
 3 animal groups  6 treatments (drug application over 6 “intervals”)

Slide 41

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Data

Slide 42

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Overall partitioning
Total variation Between subjects Within subjects Intervals

Groups

Ss w/in groups

IxG

Error (I x Ss w/in groups)

Slide 43

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Between-subjects partitioning
 Partitioning includes the “main effect” of Groups and remaining variability due to Ss within groups

 Latter can be used as the error relative to which the effect of groups is tested: errorbetween

Slide 44

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

ANOVA table

Slide 45

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Within-subjects partitioning
 Interval is a within-subjects term  Interaction with Groups is also a within-subjects term  Remaining variability is sometimes denoted errorwithin
 Can also be thought as I x Ss within groups

 Actual calculations of sum of squares proceed in the usual manner

Slide 46

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Overall partitioning
 SStotal is first partitioned into SSbetween and SSw/in subj  Each term is then further subdivided

Slide 47

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Overall partitioning
SS w/in subj  SS total  SSbetween subj
Error

SSSs w/in group  SSbetween subj  SSgroups SSI  Ss w/in group  SS w/in subj  SSintervals  SS IG

Slide 48

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Assumptions
 Sphericity is often tested via a test by Mauchly
 Although routinely printed out, not recommended because of its extreme lack of robustness

 “Because tests of sphericity are likely to have serious problems when we need them the most, it has been suggested that we always use the correction for degrees of freedom”  In this case:
ˆ   0.66    0.87
Slide 49
Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Assumptions
 When one is interested in within-subjects simple effects (e.g., Interval for the control group), the adequacy of df adjustments as performed via Greenhouse-Geisser or Huynh-Feldt corrections is very questionable  One approach is to simply perform separate repeatedmeasures analysis for each of the groups
Total variation

Between subjects

Within subjects

Between treatments

Error
Luiz Pessoa, Brown University © 2004

Slide 50

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example

Slide 51

Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology

Within-Subjects Designs

Example
 What are we doing?  Basically, using unpooled errors instead of the pooled error used in the complete analysis
 (2685 + 3477 + 1871) / 3 = 2678 pooled

 For between-subjects simple effects (e.g., Groups at Interval 1), procedure is less straightforward
 Consult Howell

Slide 52

Luiz Pessoa, Brown University © 2004


								
To top