Multilevel Modeling

Document Sample

```					Multilevel Modeling
1.   Overview

2.   Application #1:       Growth Modeling

Break

3.   Application # 2:      Individuals Nested
Within Groups

4.   Questions?
Overview
1.   What is multilevel modeling?
2.   Examples of multilevel data structures
3.   Brief history
4.   Current applications
5.   Why multilevel modeling?
6.   What types of studies use multilevel
modeling?
7.   Computer Programs (HLM 6
SAS Mixed
8.   Resources
Multilevel Question
   What effects do the following variables

School Size
Classroom Climate
Student Gender
What is Multilevel or Hierarchical
Linear Modeling?

Nested Data Structures
Several Types of Nesting

   1. Individuals Nested Within Groups
Individuals Undivided

Unit of Analysis = Individuals
Individuals Nested Within
Groups

Unit of Analysis = Individuals + Classes
… and Further Nested

Unit of Analysis = Individuals + Classes + Schools
Examples of Multilevel Data
Structures
   Neighborhoods are nested within
communities

   Families are nested within
neighborhoods

   Children are nested within families
Examples of Multilevel Data
Structures
   Schools are nested within districts

   Classes are nested within schools

   Students are nested within classes
Multilevel Data Structures
Level 4 District (l)

Level 3 School (k)

Level 2 Class (j)

Level 1 Student (i)
2nd Type of Nesting

   Repeated Measures Nested Within
Individuals

Focus = Change or Growth
Time Points Nested Within
Individuals
Repeated Measures Nested
Within Individuals
Carlos
Day               Energy Level
Monday = 0        98
Tuesday = 1       90
Wednes. = 2       85
Thursday = 3      72
Friday   =4       70
Repeated Measures Nested
Within Individuals
100

90

80

70
ENERGY

60
0     1   2   3   4   5

DAY
Repeated Measures Nested
Within Individuals
100

90

80

70
ENERGY

60                             Rsq = 0.9641
0     1   2   3   4   5

DAY
Changes for 5 Individuals

Change s in Ene rgy Le ve l Ove r the We e k

100.00

75.00
Energy Level

50.00

25.00

0
0   1.00       2.00       3.00        4.00

Tim e
3rd Type of Nesting
(similar to the 2nd)
   Repeated Measures Nested Within
Individuals

Focus is not on change

Focus in on relationships between
variables within an individual
Repeated Measures Nested
Within Individuals
Carlos
Day      Hours of Sleep Energy Level
Monday       9              98
Tuesday      8              90
Wednesday    8              85
Thursday     6              72
Friday       7              70
Repeated Measures Nested Within
Individuals (Not Change)

100

90

80

70
ENERGY

60
5.5    6.0   6.5   7.0   7.5   8.0   8.5   9.0   9.5

HOURS
Repeated Measures Nested Within
Individuals (Not Change)

100

90

80

70
ENERGY

60
5.5   6.0   6.5   7.0   7.5   8.0   8.5   9.0   9.5

HOURS
Repeated Measures Nested
Within Individuals
Re pe ate d M e as ure s Ne s te d Within Individuals (3 Individuals )

100.00

75.00
Energy Level

50.00

25.00

0
2.00         4.50         7.00         9.50      12.00

Hours of Sle e p
Repeated Measures Within
Persons

Level 2 Student (i)

Level 1 Repeated Measures
Over Time (t)
Nested Data

   Data nested within a group tend to
be more alike than data from
individuals selected at random.

   Nature of group dynamics will tend
to exert an effect on individuals.
Nested Data
   Intraclass correlation (ICC) provides
a measure of the clustering and
dependence of the data

0 (very independent) to 1.0 (very
dependent)
Details discussed later
Brief History
of Multilevel Modeling

Robinson, W. S. (1950). Ecological
correlations and the behavior of
individuals. Sociological Review, 15, 351-
357.

Burstein, Leigh (1976). The use of data from
groups for inferences about individuals in
educational research. Doctoral
Dissertation, Stanford University.
Table 1
Frequency of HLM application evidenced in Scholarly Journals
Journal                     1999   2000   2001       2002   2003   Total by journal
American Educational Research Journal                  3       5      4         3      ?           ~15
Child Development                                      3       2      6         5      13          29
Cognition and Instruction                              1       0      0         0      0            1
Contemporary Educational Psychology                    0       0      0         0      0            0
Developmental Psychology                               2       1      2         5      7           17
Educational Evaluation and Policy Analysis             2       1      5         2      2           12
Educational Technology, Research and Development       0       0      0         0      0            0
Journal of Applied Psychology                          1       1      5         7      6           20
Journal of Counseling Psychology                       0       2      1         0      0            3
Journal of Educational Computing Research              0       0      0         0      0            0
Journal of Educational Psychology                      1       2      3         6      1           13
Journal of Educational Research                        2       0      3         3      5           13
Journal of Experimental Child Psychology               0       0      0         0      0            0
Journal of Experimental Education                      0       0      0         0      1            1
Journal of Personality and Social Psychology           4       4      6         5      13          32
Journal of Reading Behavior/Literacy Research          0       0      0         0      0            0
Journal of Research in Mathematics Education           0       0      0         0      0            0
Reading Research Quarterly                             0       0      0         1      0            1
Sociology of Education                                 1       2      5         2      1           11
Total by Year                                          20     20     40         39     49         ~168
Multilevel Articles
Frequency of Studies Employing HLM in Education or Related Journals

50
Total for 19 Journals Reviewed
Journal of Personality and Social Psychology
Child Development
Journal of Educational Research
Frequency

25

0
1999                  2000                    2001         2002              2003
Year
Some Current Applications
of Multilevel Modeling

   Growth Curve Analysis
   Value Added Modeling of Teacher
and School Effects
   Meta-Analysis
Multilevel Modeling Seems
New But….
Extension of General Linear Modeling

Simple Linear Regression
Multiple Linear Regression
ANOVA
ANCOVA
Repeated Measures ANOVA
Multilevel Modeling
   Our focus will be on observed
variables (not Latent Variables as in
Structural Equation Modeling)
Why Multilevel Modeling
vs.

1.   Individual level analysis (ignore
group)

2.   Group level analysis (aggregate
data and ignore individuals)
Problems with

1.   Individual level analysis (ignore group)

Violation of independence of data
standard errors (standard errors are
smaller than they should be).
Problems with

1.   Group level analysis
(aggregate data and ignore individuals)

Aggregation bias = the meaning of a
variable at Level-1 (e.g., individual level
SES) may not be the same as the
meaning at Level-2 (e.g., school level
SES)
Multilevel Approach

   2 or more levels can be considered
simultaneously
   Can analyze within- and between-
group variability
What Types of Studies Use
Multilevel Modeling?

Quantitative

Experimental
*Nonexperimental
(Survey, Observational)
How Many Levels Are
Usually Examined?

2 or 3 levels very common

15 students x 10 classes x 10 schools
= 1,500
Types of Outcomes

   Continuous Scale (Achievement,
Attitudes)
   Binary (pass/fail)
   Categorical with 3 + categories
Software to do Multilevel
Modeling
SPSS Users
2 SAV Files: Level 1
Level 2

(Raudenbush, Bryk, Cheong, & Congdon,
2004)
HLM 6
Software to do Multilevel
Modeling
SAS Users

Proc Mixed
Resources
(Sample…see handouts for more complete list)

   Books
   Hierarchical Linear Models: Applications and
Data Analysis Methods, 2nd ed.
Raudenbush & Bryk, 2002.

   Introducing Multilevel Modeling.
Kreft & DeLeeum, 1998.
   Journals
 Educational and Psychological Measurement

 Journal of Educational and Behavioral
Sciences
Resources (cont)
(Sample…see handouts for more complete list)

   Software
 HLM6

 SAS (NLMIXED and PROC MIXED)

 MLwiN

   Journal Articles
   See Handouts for various methodological and
applied articles
   Data Sets
 NAEP Data

 NELS:88; High School and Beyond
Self-Check 1
   A teacher with 1 classroom of 24
students used weekly curriculum-
based measurements to monitor
reading over a 14 week period. The
teacher was interested in individual
students’ rates of change and
differences in change by male and
female students.
Self-Check 1
   How would you classify this
situation?

(a) not multilevel
(b) 2-level
(c) 3-level
Self-Check 2
   A researcher randomly selected 50
elementary schools and randomly
selected 30 teachers within each
school. The researcher was
interested in the relationships
between 2 predictors (school size
and teachers’ years experience at
their current school) and teachers’
job satisfaction.
Self-Check 2
   How would you classify this
situation?

(a) not multilevel
(b) 2-level
(c) 3-level
Self-Check 3
   60 undergraduates from the research participant
pool volunteered for a study that used written
vignettes to manipulate the interactional style
(warm, not warm) of a professor interacting with
a student. 30 randomly assigned students read
the vignette depicting warmth and 30 randomly
assigned students read the vignette depicting a
lack of warmth. After reading the vignette
students used a questionnaire to rate the
likeability of the professor.
Self-Check 3
   How would you classify this
situation?
(Select ONLY one)

(a) not multilevel
(b) 2-level
(c) 3-level
Growth Curve Modeling

   Studying the growth in reading
achievement over a two year period

   Studying changes in student
attitudes over the middle school
years
Research Questions
   What is the form of change for an
individual during the study?
Research Questions
   What is an individual’s initial status
on the outcome of interest?
Research Questions
   How much does an individual change
during the course of the study?

Rise        Rise
b
Run
Run
Research Questions
   What is the average initial status of
the participants?
Research Questions
   What is the average change of the
participants?
Research Questions
   To what extent do participants vary
in their initial status?
Research Questions
   To what extent do participants vary
in their growth?
Research Questions
   To what extent does initial status
relate to growth?
Research Questions
   To what extent is initial status
related to predictors of interest?
Research Questions
   To what extent is growth related to
predictors of interest?
Design Issues
   How many waves a data collection
are needed?
   >2
   Depends on complexity of growth curve
Design Issues
   Can there be different numbers of
observations for different
participants?

Examples
   Missing data
   Planned missingness
Design Issues
   Can the time between observations
vary from participant to participant?

Example: Students observed
   1, 3, 5, & 7 months
   1, 2, 4, & 8 months
   2, 4, 6, & 8 months
Design Issues
   How many participants are needed?

   More is better
   Power analyses
   > 30 rule of thumb
Design Issues
   How should participants be sampled?

   What you have learned about sampling
still applies
Design Issues
   What is the value of random
assignment?

   What you have leaned about random
assignment still applies
Design Issues
   How should the outcome be
measured?

   What you have learned about
measurement still applies
Example
   Context description

A researcher was interested in changes in
verbal fluency of 4th grade students,
and differences in the changes between
boys and girls.
ID   Gender        Time______

t0    t4    t7

1      0      20     30   30
2      0      40     44   49
3      0      45     40   60
4      0      50     55   59
5      0      42     48   53
6      1      45     52   61
7      1      39     55   63
8      1      46     58   68
9      1      44     49   59
Example
   Level-1 model specification

Yfluency   0   1 * (Time)  error1
Example
   Level-2 model specification

 0  G00  G01 * (Gender )  error2
 1  G10  G11 * (Gender )
Example
   Combined Model

Yfluency  G00  G01 * (Gender )  G10 * (Time) 
G11 * (Gender ) * (Time)  error2  error1
Example
   SAS program

proc mixed covtest;
class gender;
model score = time gender time*gender/s;
random intercept / sub=student s;
Example
     SAS output – variance estimates

Covariance Parameter Estimates

Standard   Z
Cov Parm     Subject   Estimate     Error  Value     Pr Z

Intercept    Student   62.5125    35.9682    1.74   0.0411
Residual               14.1173     4.9912    2.83   0.0023
Example
     SAS output – fixed effects
Solution for Fixed Effects

Standard
Effect        Gender   Estimate    Error   DF t Value   Pr > |t|

Intercept               39.8103   3.7975    7   10.48   <.0001
time                     1.5077   0.3295   16    4.58   0.0003
Gender           F       5.7090   5.6962   16    1.00   0.3311
Gender           M            0        .    .     .      .
time*Gender      F       1.0692   0.4943   16    2.16   0.0460
time*Gender      M            0        .    .     .      .
Example
       Graph – fixed effects

100.00
GE NDE R = 0
GE NDE R = 1

75.00
SCORE

50.00

25.00

0
0   2.50    5.00   7.50   10.00

TIM E
Example
   Conclusions

   Fourth grade girl’s verbal fluency is
increasing at a faster rate than boy’s.
Persons Nested in Contexts

   Studying attitudes of teachers who
are nested in schools

   Studying achievement for students
who are nested in classrooms that
are nested in schools
Research Questions
   How much variation occurs within and
among groups?

   To what extent do teacher attitudes vary
within schools?

   To what extent does the average teacher
attitude vary among schools?
Research Questions
   What is the relationship among selected
within group factors and an outcome?

   To what extent do teacher attitudes vary
within schools as function of years
experience?

   To what extent does student achievement
vary within schools as a function of SES?
Research Questions
   What is the relationship among selected
between group factors and an outcome?

   To what extent do teacher attitudes vary
across schools as function of principal

   To what extent does student math
achievement vary across schools as a function
Research Questions
   To what extent is the relationship among
selected within group factors and an
outcome moderated by a between group
factor?

   To what extent does the within schools
relationship between student achievement and
SES depend on the school adopted
curriculum?
Design Issues

   Consider a design where students are
nested in schools

   How should schools should be sampled?

   How should students be sampled within
schools?
Design Issues

   Consider a design where students are
nested in schools

   How many schools should be sampled?

   How many students should be sampled per
school?
Design Issues

   What kind of outcomes can be
considered?

   Continuous
   Binary
   Count
   Ordinal
Design Issues
   How will level-1 variables be
conceptualized and measured?

   SES

   How will level-2 variables be
conceptualized and measured?

   SES
Terminology
   Individual growth trajectory – individual growth
curve model
 A model describing the change process for an
individual
   Intercept
 Predicted value of an individual’s status at
some fixed point
 The intercept cold represent the status at the
beginning of a study
   Slope
 The average amount of change in the
outcome for every 1 unit change in time
Intercept & Slope Illustration

25

20

15
Score

10
Rise
Rise       b
5
Run
Run
0
0   1   2    3     4      5     6         7     8   9   10
Time
intercept

35

30

25

20
Score

15

10

5

0
0          1                2                3    4

Time
HLM
   Hierarchical Linear Model
   The hierarchical or nested structure of
the data
   For growth curve models, the repeated
measures are nested within each
individual
Levels in Multilevel Models
   Level 1 = time-series data nested
within an individual

Y   0   1 * (Time)  error
Levels in Multilevel Models
   Level 2 = model that attempts to
explain the variation in the level 1
parameters

 0  G00  G01 * (Sessions)  error
 1  G10  G11 * (Sessions)  error
More terminology
   Fixed coefficient
   A regression coefficient that does not
vary across individuals
   Random coefficient
   A regression coefficient that does vary
across individuals
More terminology
   Balanced design
   Equal number of observations per unit
   Unbalanced design
   Unequal number of observation per unit
   Unconditional model
   Simplest level 2 model; no predictors of the
level 1 parameters (e.g., intercept and slope)
   Conditional model
   Level 2 model contains predictors of level 1
parameters
Estimation Methods
   Empirical Bayes (EB) estimate
   “optimal composite of an estimate
based on the data from that individual
and an estimate based on data from
other similar individuals” (Bryk,
Raudenbush, & Condon, 1994, p.4)
Estimation Methods
   Expectation-maximization (EM)
algorithm
   An iterative numerical algorithm for
producing maximum likelihood
estimates of variance covariance
components for unbalanced data.

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 11 posted: 11/17/2011 language: English pages: 96
How are you planning on using Docstoc?