Multivariate Analysis of Variance (MANOVA)

Multivariate Analysis of Variance (MANOVA) • Let's revisit ANOVA to set the conceptual background for MANOVA – mean differences between groups of a factor are tested via the F-test • F = MSbg / MSwg – MSbg = SSbg / dfbg – MSwg = SSwg / dfwg • if no treatment effect F approximately equals 1 • if there is a treatment effect F is large ANOVA Basics continued • let’s do a little example with the following data set Experimental Control Group Group 2 6 3 7 1 5 Treatment Mean 2 6 Sums of Squares 2 2 Overall Mean 4 Total SS 28 ANOVA Basics continued • Calculating SSwg and SSbg – SSwg is the pooled within-group variability • so simply add the SS for each group (=4) – SSbg is the difference the mean of each group and the grand mean • this quantity is then multiplied by n (=24) • Calculating MSwg and MSbg • MSwg = 4 / 4 = 1 • MSbg = 24 / 1 = 24 • Calculate F – F = 24 / 1 = 24 • translate this into a p-value (=.008) – divide the SS values by degrees of freedom Multivariate Analysis of Variance (MANOVA) • Obviously MANOVA is an extension of ANOVA • 2 or more correlated DVs analyzed together • DVs form a “set” • this is yet another form of the general linear model – Y1 + Y2 + Y3 + ... + Yn = X1 + X2 + X3 + ... + Xn • each individual gets a new composite DV – where the Y-values are continuous variables – and the X-values are discrete variables • or dummy-coded variables if we have multiple groups/conditions for specific discrete variables Multivariate Analysis of Variance (MANOVA) • Why not do separate ANOVAs? – MANOVA is sometimes powerful because we have taken this correlation among DVs into account • i.e., better deals with overlapping variance – partial redundancy among DVs • does it by creating a linear composite of the individual DVs • helps protect against Type I error – however, we could also do the same analyses using ANOVA if – used a more conservative alpha level – DVs are not “too highly correlated” Multivariate Analysis of Variance (MANOVA) • Examples – Clinical • clients randomly assigned to psychodynamic, CBT, or no therapy • we have DV difference scores for depression, life satisfaction, and physical health – Educational • effect of parent support on child achievement • achievement measured through grades & standardized tests Multivariate Analysis of Variance (MANOVA) – Developmental • gender differences in social coping during transition to junior high • coping assessed with a variety of measures – # friends in social network, # after-school activities, self-ratings of being liked, etc. – in all examples, we have more than 1 DV – similarities/differences to repeated measures ANOVA • similar in that we expect correlations between these DVs • different in that we have different measures for the DV – although this is not absolutely necessary Multivariate Analysis of Variance (MANOVA) • MANOVA and null hypothesis significance testing (NHST) – t-test • 1 =  2 – ANOVA • 1 =  2 =  3 =  n – MANOVA • 1 =  2 =  3 =  n for DV 1 AND • 1 =  2 =  3 =  n for DV 2 AND • 1 =  2 =  3 =  n for DV 3 AND • .... – the alternative hypothesis is that there is at least 1 differences (across groups) in at least 1 of the DVs or in the DV composite Multivariate Analysis of Variance (MANOVA) • Testing the null – In ANOVA, variance is partitioned into: • SS total = SS between + SS within • so if SS between is much larger than SS within the null is probably not correct – Similar approach in MANOVA • however, SS (which are scalars) are replaced by – sums of squares and cross-product (SSCP) matrices • because we need to take correlations (covariances) of the DVs into account – we use determinants to get a summary index of variance in these matrices • like mean square in ANOVA Multivariate Analysis of Variance (MANOVA) • Omnibus tests – Wilk’s  • • • • most popular test ratio of determinanterror to determinanttotal represents percent variance not explained 1 -  is an index of variance explained – 2, a measure of effect size – analogous to R2 in regression • want  to be small, and this value is transformed into an approximate F for hypothesis-testing Multivariate Analysis of Variance (MANOVA) • Other tests include – Pillai’s Trace – Hotelling’s Trace – Roy’s Largest (or Characteristic) Root • Statistical power – Pillai’s > Wilks > Hotelling’s > Roy’s • Robustness to violations of assumptions – Pillai’s most robust • In general, however, all 4 omnibus tests will agree Multivariate Analysis of Variance (MANOVA) • Follow-up analyses – for which DVs are there differences? • results from univariate ANOVAs are provided – like doing an ANOVA for each DV • you should use a more conservative alpha level • individual test  = experimenterwise  / number of DVs – ANCOVA, controlling for all other DVs – specific contrasts • simple, polynomial – posthoc comparison procedures • Tukey’s, Bonferroni, etc. – conduct a discriminant function analysis (DFA) Multivariate Analysis of Variance (MANOVA) • Assumptions and practical issues of MANOVA – participants’ scores on each DV are statistically independent from other participants’ scores on each DV • i.e., the errors are statistically independent – evaluate intraclass correlation coefficient – multivariate normality • each DV has a normal distribution • all linear combinations of DVs have normal distributions • all “groups” must have normal distributions • and, after saying all of this, MANOVA is fairly robust to violations from normality Multivariate Analysis of Variance (MANOVA) – homogeneity of variance/covariance • variance for each DV is the same across groups/conditions • and, covariance between each pair of DVs is the same across groups/conditions (pooled for the error term) • statistically evaluate this using Box’s M test and/or Bartlett’s chi-square and/or Levene’s test – problem: all are sensitive to small violations • if assumption is not satisfied... – continue on and don’t worry about it, or – use posthoc methods such as Games-Howell, Tamhane’s T2, Dunnett’s C, etc. – these correct for violation of this assumption • use a conservative alpha (.001) to determine this Multivariate Analysis of Variance (MANOVA) – linearity among DVs • linear combinations for the DVs are used to maximize the separation (i.e., differences) between groups – homogeneity of regression • specific to MANCOVA • the relationship between the DVs and the covariate(s) must be the same across groups/conditions • if not, this implies that the DV-covariate relationship is not consistent across the groups of your IV • you have an IV-covariate interaction: yikes!!! Multivariate Analysis of Variance (MANOVA) • Kinds of research questions – main effects for the IVs – interactions among the IVs – importance of DVs part I • a different linear combination of DVs is created for each “effect” (i.e., for each main effect and interaction) • therefore, different DVs may be differentially influenced by type of effect – importance of DVs part II • analogous to hierarchical or sequential regression • DVs are organized in terms of their importance • Roy-Bargman stepdown analysis Multivariate Analysis of Variance (MANOVA) – effect sizes • 2, can exceed 1 because DVs are correlated • partial 2, which adjusts for the overestimation – incorporated with more sophisticated designs • inclusion of covariates for the MANCOVA • choose your covariates wisely – correlations with DVs not with the IVs – including a time element • Repeated-measures MANOVA • having multiple DVs measured on multiple occasions Multivariate Analysis of Variance (MANOVA) • Some final thoughts – Back to the type of research questions that MANOVA answers • multiple univariate questions – when DVs have low correlations • intrinsically multivariate questions – when DVs have “high” correlations ( .60 ) – used when the principal concern is how the DVs differ/change with respect to the “new” variable Multivariate Analysis of Variance (MANOVA) – select your DVs carefully • there is a danger in include too many DVs • one “bad” variable can drastically alter your results – sample size • MANOVA does require larger sample sizes than ANOVA • have to factor in number of DVs • Scott’s rules of thumb – 10 participants per DV, and – minimum of 10 in each condition/group combination