PROFILING PRACTICES:
HOW MUCH DO WE KNOW? Sharon-Lise Normand
Quality Measures
Continuous
Single
A
Multiple
B
Binary
SL Normand AcademyHealth 2006
C
D
1
SINGLE QUALITY MEASURE
(CONTINUOUS)
Practice 1 Practice i Practice I
Patient 1
Patient n1
Patient 1 Patient nI
Single quality measure
2
SL Normand AcademyHealth 2006
SINGLE QUALITY MEASURE
(CONTINUOUS)
Variation in outcomes: • Between-practice: 2P • Within-practice (among patients): 2W • Total variation: 2 = 2P + 2W
Intra-Practice Correlation: P = 2P/(2P + 2W ) Proportion of total variation in outcome due to the practice. Note: 2W = 2(1 - P)
3
SL Normand AcademyHealth 2006
SINGLE QUALITY MEASURE
(CONTINUOUS) How many patients per practice, n? • Reliability:
– Degree to which repeated measurements will yield the same results. – Why? Users know that the practice measure has a minimum level of reliability (R*). – n = R*(1 - P)/(P (1-R*))
4
SL Normand AcademyHealth 2006
Number Patients Per Practice to Achieve R*. Intra-Practice Correlation Coefficient R*
0.02 0.05 0.1 0.2
.70 .75 .80 .85
115 147 196 278
45 57 76 108
21 27 36 51
10 12 16 23
5
SL Normand AcademyHealth 2006
WHAT IS OUR EXPERIENCE?
(CONTINUOUS) Intra-Practice Correlation Coefficients: Not very much! • Outpatient visit rates – 0.04 (Hofer et al., 1999). • Hemoglobin A1c – 0.18 (Greenfield et al., 2002).
6
SL Normand AcademyHealth 2006
MULTIPLE QUALITY MEASURES
(CONTINUOUS)
Practice 1 Practice i Practice I
Patient 1
Patient n1
Patient 1 Patient nI
Multiple quality measures
7
SL Normand AcademyHealth 2006
MULTIPLE QUALITY MEASURES
(CONTINUOUS)
Variation in outcomes: • Between-practice: 2P • Among-subjects (within practice): 2S • Within-subject (among measures): 2W • Total variation: 2 = 2P + 2S + 2W
Intra-Practice Correlation: P = 2P/(2P + 2S + 2W ) Proportion of total variation in outcomes due to the practice.
8
SL Normand AcademyHealth 2006
MULTIPLE QUALITY MEASURES
(CONTINUOUS) Intra-Subject Correlation: S = (2P+ 2S)/(2P + 2S + 2W) Correlation for outcomes within subjects.
– Assume more similarity among outcomes measured on same subject than among measures attributed to same practice, e.g.,
S P
9
SL Normand AcademyHealth 2006
Number Patient-Measures Per Practice to Achieve R*. Intra-Practice Correlation Coefficient R*
0.01 .70 .75 .80
.85
0.02 115 147 196
278
0.05 45 57 76
108
0.1 21 27 36
51
10
231 297 396
561
SL Normand AcademyHealth 2006
WHAT IS OUR EXPERIENCE?
(CONTINUOUS)
Intra-Practice Correlation Coefficients:
Nothing!
11
SL Normand AcademyHealth 2006
SINGLE QUALITY MEASURE
(BINARY)
Variation in rates: • Between-practice: 2P • Within-practice (among patients): ? • Total variation: 2 = 2P + ? • Some suggested ? = (3.14/3)2 so that P = 2P/(2P + (3.14/3)2) • Others suggestions
12
SL Normand AcademyHealth 2006
SINGLE QUALITY MEASURE
(BINARY)
Variance Components Inappropriate: • Use the median odds ratio: exp(2 2P) -1(0.75) • Median odds ratio between patients treated by practices having higher quality and practices having lower quality. • If = 1, no practice variation; otherwise it is > 1.0.
13
SL Normand AcademyHealth 2006
REMARKS
• Very little experience/knowledge about magnitude of P and S • Formula/concepts assumed continuous variables
– More difficulty when measures are binary – Need to link median OR to sample size calculation
• Need information about these variance components/correlation coefficients; otherwise usefulness of information questionable.
14
SL Normand AcademyHealth 2006