Fitting Learning Curves_ Nonlinear Mixed Effects Modeling

Document Sample
Fitting Learning Curves_ Nonlinear Mixed Effects Modeling Powered By Docstoc
					Nonlinear Mixed Effects Modeling
Traditional (Unordered ANOVA):
    Morris Water Maze data

             90
             80                             Group 1
             70       *                     Group 2
   Latency




             60
                          *   *
             50
             40
             30
             20
             10
              0
                  1   2   3   4 5 6 7       8   9 10
                              Testing Day
Standard methods of analyzing learning
curves from repeated measures designs
• Repeated measures ANOVA
  – Ignores the ordinal characteristics of the day/session
    variable.
     • Problems: Fails to identify function, may lack power.
  – Can be followed by linear and quadratic contrasts.
     • Problem: poor extrapolation.
• Two-stage approach
  – Fit nonlinear curve to each subject, identify best-fitting
    parameter values for each subject, and analyze these
    parameter values as a function of group.
  – Problem: No sensitivity to differences in uncertainty in
    these estimates as a function of small samples,
    variability in data, missing values.
          Poor Extrapolation for Linear/Quadratic Fits

           80

           60
Latency




           40

           20


                 2       4     6   8   10   12   14
          - 20
                     Day/Session
          - 40
  Two-stage approach to nonlinear fits


• Fit nonlinear function to each subject’s data
  – Must identify nonlinear function because we no
    longer assume linearity.
• Learning curves often well approximated by
  exponential function
    Two-stage approach to nonlinear fits

    • Two-parameter exponential:
                               Bx
                y  Ae
      – A indicates the intercept at x=0
      – B is the slope.
    • For two-stage approach, the A and B values
      are estimated for each subject and then
      subjected to ANOVA by condition.


          Groups’ and Subjects’
Intercepts (Left) and Slopes (Right) Vary
  A method that estimates parameter
      values at multiple levels

 • Best guess at A and B values for each group
   and each subject.
   – Two levels of estimation:
                             B g x g
            y g  Ag e
y s[g ]  Ag  As 
                                 B g B s x s[ g]
                     e
  Nonlinear mixed effects modeling

• A multilevel modeling approach
  – Uses all of the data to estimate parameter values at both
    levels (fixed effect of group and random effect of
    subject).
  – Thus, when there is little or uncertain data from a
    subject:
     • The subject will have less influence on group estimates, and
     • The subject’s estimated values (fixed plus random effects) will
       be more strongly determined by group estimates.
         NLME: Is it worth it?

• What does a “proper” analysis buy us for
  real data?
  – How does the traditional method (ANOVA
    followed by day-wise tests), linear fits, and
    nonlinear fits compare in terms of Type I error,
    Type II error, and discriminating true from
    false positives
  – Answer: We used Monte Carlo simulations to
    address this question (Young, Clark, Goffus, &
    Hoane, 2009).
       Monte Carlo Methods

• Generate many, many data sets with known
  population characteristics.
• Run analyses on these data sets.
• Determine the degree to which each
  analysis recovers the true population
  characteristics.
Example of Real Data
Example of simulated data
Discriminating True Positives from
  False Positives using SDT’s d’
                                   d’
             Slope
Intercepts   effect   Unordered   Linear   Nonlinear
Equal          0
Equal         0.5       0.41      0.43       0.46
Equal          1        1.18      1.30       1.42
Equal          2        2.58      2.92       3.08
Uneq <90       0        0.77      3.29       3.34
Uneq <90      0.5       1.03      3.48       3.59
Uneq <90       1        1.64      3.48       3.60
Uneq <90       2        2.86      3.90       4.40
Uneq >90       0        0.29      2.36       2.04
Uneq >90      0.5       0.64      2.45       2.12
Uneq >90       1        1.38      2.76       2.45
Uneq >90       2        2.79      3.63       3.41
How big is that SDT difference?
                      1.0
                              lme

                                             nlme

                      0.8
   P(True Positive)




                      0.6

                                                                d’ = 0
                                Unordered
                                 ANOVA
                      0.4




                      0.2




                        0.0         0.2       0.4       0.6              0.8   1.0
                                            P(False Positive)
       Intermediate Summary

• Unordered mixed effects modeling of group
  differences as a function of day was
  significantly underpowered.
  – Consistently weak d’.
  – Only good for detecting large effects.
• Linear and Nonlinear mixed effects
  modeling were similar in power.
  – LME slightly better when running into
    truncation issue.
           Balancing Fit with Complexity: Which
           Approach Models True Relationship?
           3300
                                                                                   unordered
                                                                                   lme
           3250
                                                                                   nlme

           3200
Mean BIC




           3150


           3100


           3050


           3000
                  -0.1   -0.2   -0.3    -0.1   -0.2   -0.3    -0.1   -0.2   -0.3
                                        Base Learning Rate

                     Equal Intercepts   Unequal Intercepts   Unequal Intercepts
                                             (< 90)               (> 90)
               NLME in R
• library(nlme)
• mydata$group<-as.factor(mydata$group)
  – If you coded group as a number.
• mydatag<-groupedData(latency~1| subject,
  data=mydata)
   A and B constant across groups
    A and B vary across subjects
myr<-                          random.effects(myr)
 nlme(latency~A*2^(B*day),     ID       A    B
 fixed=A+B~1,                   5 -39.94         -0.149
                               49 -36.00         -0.040
 random=pdDiag(A+B ~ 1),       48 -24.76         -0.191
                                8 -18.64         -0.235
 start=c(A=90, B= -.2),        …

 data=mydatag)

Fixed effects: A + B ~ 1
   Val    SE      t    p
A 75.52 3.07 24.63         0
B -0.144 0.03 -5.74        0
   A and B constant across groups
      A varies across subjects
myr<-nlme(latency~A*2^(B*day),   random.effects(myr)
  fixed=A+B~1,
                                 ID        A
  random=pdDiag(A ~ 1),           5 -48.43
                                 49 -38.85
  start=c(A=90, B= -.2),         48 -39.83
  data=mydatag)                   8 -39.23
                                 …

Fixed effects: A + B ~ 1
   Val    SE      t    p
A 67.24 4.53 14.85         0
B -0.059 0.01 -6.33        0
  A constant, B varies across groups
    A and B vary across subjects
myr<-nlme(latency~A*2^(B*day),
                                           random.effects(myr)
  fixed=c(A~1, B~Group),
  random=pdDiag(A+B ~ 1),                  ID       A    B
                                            5 -29.41         -0.023
  start=c(A=90, B= -.2, rep(0,5),
                                           49 -25.70         -0.058
  data=mydatag)                            48 -15.73         -0.025
                                            8 -11.05         -0.033
                                           …
                Value SE       t p-value
A               76.15 2.84 26.75 0.0000
B.(Intercept)   -0.06 0.04 -1.65 0.0997
B.GROUP2        -0.10 0.05 -1.89 0.0603
B.GROUP3        -0.10 0.05 -2.08 0.0387
B.GROUP4         0.03 0.05 0.53 0.5982
B.GROUP5        -0.47 0.10 -4.75 0.0000
A, B constant   A const, B varies
across groups    across groups




          shrinkage
      Learning LME and NLME
• Best resource (imho) for lmer is Gelman, A., & Hill, J.
  (2007). Data analysis using regression and
  multilevel/hierarchical models.
• Best resource for nlme is Pinheiro & Bates (2002). Mixed
  Effects Models in S and S-Plus.
   – Provides theory, examples, and demonstrates method in R and S-
     Plus.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:4/16/2012
language:English
pages:27