# No Slide Title

Document Sample

```					               PART 3

BIO656--Multilevel Models   1
Term 4, 2006
NEED TO INCORPORATE
ALL UNCERTAINTIES
• The Z versus t distribution is the basic example
• Want to produce a CI for a population mean
• Assume a Gaussian sampling distribution,:

• Z is t with a large df
• t3 is the most different from Z for t-distributions with
a finite variance
BIO656--Multilevel Models               2
Term 4, 2006
BIO656--Multilevel Models   3
Term 4, 2006
ACCOUNTING FOR
(explaining)
UNEXPLAINED VARIABILITY

• Including regressors can explain (account for) some
of unexplained variability
• Doing so is always a trade-off in that you need to use
degrees of freedom to do the explaining
• Going too far--adding too many regressors-- inflates
residual variability
• In MLMs there is variance at various levels that can
potentially be taken into account

BIO656--Multilevel Models         4
Term 4, 2006
TEACHER EXPECTANCY
(data are in “Datasets” )

Data are from a Raudenbush & Bryk meta-analysis of
19 studies (see Cooper and Hedges,1994)
Effect sizek = distance between treatment and control group
means measured in population standard
deviation units
SEk = the standard error of the effect size
Weeksk = estimated weeks of teacher-student contact
prior to expectancy induction

BIO656--Multilevel Models        5
Term 4, 2006
TEACHER EXPECTANCY (continued)
• Each study consisted of either telling teachers
that a student had great potential or not
• All students received a pre-test and a post-test
• Teachers evaluated progress
• A positive effect size indicates that the teachers
rated students who were “likely to improve” as
having improved more than the control group
• A negative slope on “Weeks” indicates that the
more a teacher got to know a student before the
experiment,the less the influence of the
expectancy intervention

BIO656--Multilevel Models           6
Term 4, 2006
BIO656--Multilevel Models   7
Term 4, 2006
BIO656--Multilevel Models   8
Term 4, 2006
ESTIMATING 2
(when all k2 = 2 )
• Compute residuals, rk = (Yk – modelk)
• Compute the Mean Squared Error:
(n – df)MSE =  r2k
•Then, compute:
2 = (MSE - 2 )
• modelk is either just the intercept or
intercept +  weeksk
• 2 decreases if MSE decreases

BIO656--Multilevel Models   9
Term 4, 2006
A CASE STUDY
ON VARIANCE ACCOUNTING

BIO656--Multilevel Models   10
Term 4, 2006
DIABETES CONTROL STUDY
Percentage Variance at the
Patient, Physician, and Clinic Levels

• Patrick J. O’Connor MD MPH,
• Gestur Davidson PhD
• A. Lauren Crain PhD
• Leif I. Solberg MD
• Robin R. Whitebird PhD
• Thomas A. Louis PhD

HealthPartners Research Foundation
University of Minnesota
Johns Hopkins Bloomberg SPH

BIO656--Multilevel Models      11
Term 4, 2006
Conceptual Model:
What Affects Diabetes Care?

A Nested Hierarchy
• Health Plan
• Medical Group
• Clinic
• Physicians
• Patients
• Interactions across all levels

BIO656--Multilevel Models     12
Term 4, 2006
OUTCOME MEASURE:
A1c (HbA1c)
Glycohemoglobin, Glycated hemoglobin
• Known as: Hemoglobin A1c
• Used to monitor diabetes and to aid in treatment decisions
• Should be assayed at first diagnosis of diabetes
and then 2 to 4 times per year
• Requires a blood sample
• Normal values between 4% and 6%

BIO656--Multilevel Models          13
Term 4, 2006
STUDY SITE
Health Partners Medical Group, MN

• 175,000 adults receiving care at 19 clinics in 1995

• Medical group centrally administered and clinics have
common guidelines, formulary, and culture

• So, there may be less variance at clinic or physician
level than in other contexts

BIO656--Multilevel Models              14
Term 4, 2006
STUDY DESIGN

• Analysis of 2,463 adults with diabetes
mellitus in 1994
• Follow-up A1c data in 1995, 1996, 997
• Patients nested within providers and
clinics

BIO656--Multilevel Models     15
Term 4, 2006
STUDY PARTICIPANTS

• 2,463 adults with DM in same clinic and
with same primary care physician each
year from 1995-1997
• DM identification:
Sensitivity = 0.91
Pred. Pos. Val. = 0.94
• To be included in analysis must have
had at least one A1c test each year
• A1c test rates ranged 80-87% per year
1995-1997

BIO656--Multilevel Models        16
Term 4, 2006
Number of Eligible Adults with diabetes
in 1995 cohort

Year       Enrolled                With A1c
Test
1995       5,432                   4,339

1996       4,835                   3,941

1997       4,451                   3,767

BIO656--Multilevel Models              17
Term 4, 2006
Analytic Sample

•   19 Clinics
•   41 Physicians
•   2,463 Patients
•   3 years of time

BIO656--Multilevel Models   18
Term 4, 2006
MODEL & ANALYSIS
• Multilevel multivariate hierarchical linear models (using
MLwiN) to estimate variance components at each level
(time, patient, physician, clinic)

• Analyzed for A1c in each year, and change in A1c
across years

BIO656--Multilevel Models            19
Term 4, 2006
STUDY POTENTIAL

• There is substantial variance in A1c and in
change in A1c across all levels of the
hierarchy
• Some of the A1c variance is a “roll-up” from
lower levels
• To develop rational improvement strategies,
one must understand where the variance
resides and if some can be explained

BIO656--Multilevel Models         20
Term 4, 2006
HYPOTHESES
• After control for patient and physician variance,
there will be no clinically significant variance in
A1c change at the clinic level

• At each level that has significant variance in A1c
change, we may be able to identify key variables
that are related to the variance

BIO656--Multilevel Models                21
Term 4, 2006
Characteristics of study participants
who had/(did not have)  1 A1c tests
during the 36-month study period
A1c Measurement
Status
Comparison           Measured    Not Meas.         P-val
Variables
% Female                      53%           53%     0.73

Average Age                   59.5          60.4    0.10

Average Charlson              1.73          2.07    0.01

Female Physician              28%           25%     0.23

Average                       42.2          43.0    0.01
Physician Age
% Fam. Med.                   33%           35%     0.39
BIO656--Multilevel Models                  22
Term 4, 2006
Characteristics of participants who
were/(were not) assigned to
a primary care physician
Physician
Assignment Status
Comparison        Assigned      Not   P-val
Variables                    Assigned
% Female                 47%              47%    0.60

Average Age              60.4             55.3   0.01

Average                  1.75             1.02   0.01
Charlson
A1c Value                8.29             8.06   0.01

BIO656--Multilevel Models                 23
Term 4, 2006
BIO656--Multilevel Models   24
Term 4, 2006
BIO656--Multilevel Models   25
Term 4, 2006
BIO656--Multilevel Models   26
Term 4, 2006
PERCENT OF VARIANCE ACCOUNTING
1995 A1c Level
“Vanilla” model & with Covariates

Vanilla                      Full Model
Model
Clinic                   1.9%                         2.7%

Physician                 2.8%                        1.4%

Patient                 95.4%                        96.0%

BIO656--Multilevel Models                27
Term 4, 2006
Covariates for 1995
A1c Level (R2 = 0.14)
Variable           Coefficient                  SE
Pt Age < 65         0.025                       0.010
Insulin Use         0.159                       0.010
Sulfonyl Use        0.106                       0.010
Phy. Specialty     -0.015                       0.014
Phy. Gender        -0.016                       0.057
Pt Comorbid=2        0.025                      0.012

BIO656--Multilevel Models           28
Term 4, 2006
BEWARE OF SELECTION EFFECTS

• The slope on insulin is 0.159 which is > 0

• Does this mean that insulin is bad for diabetics?

• Or, does it represent an association between
A1c level and the decision to treat?

Hint: It’s the association/selection

BIO656--Multilevel Models          29
Term 4, 2006
PERCENT OF VARIANCE ACCOUNTING
(1997-1995) A1c Change
“Vanilla” model & with Covariates
Vanilla                      Full Model
Model
Clinic             < 0.1%                       < 0.1%

Physician             0.7%                       0.8%

Patient            99.3%*                       99.2%*

BIO656--Multilevel Models                30
Term 4, 2006
Covariates for 1997-1995
A1c change (R2 = 0.11)

Variable              Coefficient                 SE
Pt Age < 65            0.093                      0.044
Drug Intensity        -0.418                      0.056
Doc Specialty          0.057                      0.104
Pts Per Doc           -0.002                      0.002
Doc Age               -0.001                      0.004
Pt Comorbidity        -0.014                      0.018

BIO656--Multilevel Models           31
Term 4, 2006
ANALYZING CHANGE
REMOVES THE SELECTION EFFECT

• The slope on insulin is now –0.418 which is < 0

• Warning: such a simple analysis will not always
sort things out

 “Causal Analysis” is needed

BIO656--Multilevel Models            32
Term 4, 2006
RESULTS
• Models with limited set of covariates explained
about 14% of variance in A1c levels in 1995
• Models with covariates explained about 35% of
variance in change in A1c from 1995-97
• Over 90% of variance was at patient level or
related to physician-patient interaction
• Little variance at physician level
• Little variance at the clinic level

BIO656--Multilevel Models           33
Term 4, 2006
Factors associated with change in A1c

• Time: A1c got better each year
• Older patients had more improvement
• Comorbidity was related (complex)
• Drug Intensification (by drug class) was the
variable that was strongest predictor
• Unidentified Patient Factors are likely

BIO656--Multilevel Models       34
Term 4, 2006
Other Clinical Domains

• A1c Test Rates
• LDL Test Rates
• Eye Exam Rates

• Generally similar results, with the majority
of variance at patient level; much less
variance at physician and clinic levels

BIO656--Multilevel Models    35
Term 4, 2006
STUDY LIMITATIONS

• Relatively homogeneous medical group,
may reduce variance at clinic and doc level
• Clinic systems already in place
• Selection effects
• Paucity of covariates at clinic and provider
levels

BIO656--Multilevel Models    36
Term 4, 2006
DISCUSSION
• Interventions may be made at any level,
not just levels with significant variance
• However, a great deal of recent attention is
directed to clinic systems
• Patient behavior and provider behavior and
doctor-patient interaction need more attention
• Factors that impact drug intensification may
be key

BIO656--Multilevel Models           37
Term 4, 2006
Future Directions

•   Larger set of medical groups and clinics
•   More covariates at each level
•   Model selection effects
•   Estimate power at various levels
•   Strategies to handle missing data
•   Assess other clinical domains

BIO656--Multilevel Models     38
Term 4, 2006

```
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
 views: 2 posted: 7/27/2012 language: pages: 38
How are you planning on using Docstoc?