The Effects of Health Plan Performance Measurement on Quality of Care for Medicare Beneficiaries
Presented by Kate Bundorf Co-authors: Laurence Baker and Kavita Choudhry
AcademyHealth Annual Research Conference
June 27, 2006
This project received support from the National Institute on Aging (AG023557 ) and the Agency for Healthcare Research and Quality (KO2 HS11668).
Background
• Health care provider “report cards” are viewed as a tool to improve quality of care.
– Allow consumers to make tradeoffs between cost and quality when choosing among providers. – Create incentives for providers to improve quality of care.
• Medicare has been at the forefront of efforts to disseminate information about provider quality. • NCQA has been the key driver of initiatives to measure health plan quality. • The Medicare program began requiring Medicare managed care plans to submit Medicare HEDIS data as of January 1997
Have Health Plan Report Cards Improved Quality of Care?
• Do consumers use the information?
– Health plan report cards have had a small effect on health plan enrollment in the commercial population (Chernew, et. al. 2001; Beaulieu, 2002; Scanlon et. al, 2002; Wedig and Tai-Seale, 2002; Jin and Sorenson, 2005). – Health plan report cards in the Medicare program have shifted enrollment from lower to higher quality plans; but – Enrollment shifts have been driven by measures of consumer satisfaction rather than measures of processes of care (Dafny and Dranove 2006).
• Do providers use the information?
– Hospitals responded to the implementation of mandatory report cards even in the absence of evidence that patients were using the information (Dranove, Kessler, et. al. 2003). – HEDIS performance indicators are widely used by health plans to implement and monitor quality improvement programs (Perry 2000; Scanlon, Darby, et. al. 2001).
Have Health Plan Report Cards Improved Quality of Care?
• Performance on the indicators among plans participating in HEDIS has improved over time (NCQA 2002; Lied and Sheingold 2001; Trivedi, Zaslavsky, et. al. 2005). • These trends are not necessarily evidence of a causal effect:
– No control group – Improvements in performance could be driven by improvements in measurement – Changes in utilization could be due to changing plan enrollment
• After controlling for these issues in the commercial population, performance improvements are less dramatic (Bundorf and Baker 2006).
Study Design
• Compare utilization of measured services between Medicare managed care enrollees and other beneficiaries before and after the implementation of mandatory quality reporting. • Use managed care market share in the beneficiaries’ county to control for both selection of beneficiaries across sectors and spillovers from managed care to the FFS sector.
Data Sources
• 1993-1999 Medicare Current Beneficiary Survey (MCBS)
– Annual survey of approximately 12,000 Medicare beneficiaries conducted by CMS. – Study sample includes non-institutionalized beneficiaries >=65 residing in a metropolitan area.
• 1993-1999 Medicare Managed Care Market Penetration Report from CMS.
Performance Indicators
Study Variable
Mammogram (Women 6569)
HEDIS Performance Indicator
The percentage of women 52 – 69 who were enrolled in a health plan that had a mammogram during the measurement year or the year prior to the measurement year. The percentage of members age 35 and older hospitalized and discharged from the hospital after surviving a heart attack who received a prescription for a betablocker
The percentage of older adults who received a flu shot. The percentage of diabetics receiving eye exams.
MCBS Definition
Respondent self report of having had a mammogram within the past year among women 65-69.
Beta Blocker for Heart Attack Patients
Identification of utilization of a beta blocker during the survey year among those self reporting that they had ever been told they had a myocardial infarction, including men and women 65 and over.
Respondent self report of having received a flu shot. Respondent self report of having had an eye exam in the past year among beneficiaries indicating they had ever been told they have diabetes.
Flu shot Eye Exam for Diabetics
Utilization of performance indicators among Medicare managed care enrollees
Mammogram
0.80 0.70 0.60
0.80 0.70 0.60
Beta Blocker
Utilization Rate
Utilization Rate
0.50 0.40 0.30 0.20 0.10 0.00 1993 1994 1995 1996 1997 1998 1999
0.50 0.40 0.30 0.20 0.10 0.00 1993 1994 1995 1996 1997 1998 1999
Medicare Managed Care Enrollees
Medicare Managed Care Enrollees
Flu Shots
0.80 0.70
Eye Exam for Diabetics
0.80 0.70 0.60
Utilization Rate
0.60
Utilization Rate
0.50 0.40 0.30 0.20 0.10 0.00 1993 1994 1995 1996 1997 1998 1999
0.50 0.40 0.30 0.20 0.10 0.00 1993 1994 1995 1996 1997 1998 1999
Medicare Managed Care Enrollees
Medicare Managed Care Enrollees
Utilization of performance indicators relative to other beneficiaries
Mammogram
0.80 0.70 0.60
0.80 0.70 0.60
Beta Blocker
Utilization Rate
Utilization Rate
0.50 0.40 0.30 0.20 0.10 0.00 1993 1994 1995 1996 1997 1998 1999
0.50 0.40 0.30 0.20 0.10 0.00 1993 1994 1995 1996 1997 1998 Other Insurance 1999
Medicare Managed Care Enrollees
Other Insurance
Medicare Managed Care Enrollees
Flu Shots
0.80 0.70 0.60
0.80 0.70 0.60
Eye Exam for Diabetics
Utilization Rate
1993 1994 1995 1996 1997 1998 1999
Utilization Rate
0.50 0.40 0.30 0.20 0.10 0.00
0.50 0.40 0.30 0.20 0.10 0.00 1993 1994 1995 1996 1997 1998 Other Insurance 1999
Medicare Managed Care Enrollees
Other Insurance
Medicare Managed Care Enrollees
Model Estimation
Model 1: Unadjusted difference-in-difference estimate
Compare the difference between beneficiaries enrolled in managed care plans and those enrolled in FFS before and after the implementation of mandatory quality reporting for Medicare managed care plans (controlling for time trends common to all beneficiaries).
Model 2: Control for selection based on observable characteristics
Controls include age, sex, education, marital status, race, ethnicity, self-reported history of 16 conditions, self reported health status, number of ADLs, BMI indicators, smoking status, and county fixed effects.
Model 3: Control for selection and spillover effects using managed care market share variables
• Spillover and selection effects: managed care share
•
•
Selection effects: managed care share*HMO enrollee
Difference between pre- and post-intervention periods in spillover and selection effects: Interaction of each variable with an indicator of the intervention period
Results for Mammogram
Table 5: Multivariate Models of Utilization of Performance Indicators Mammogram (2) 0.075 [0.051] -0.086 [0.057]
MMC Enrollee MMC Enrollee*Reporting Period MMC Share MMC Share * MMC Enrollee MMC Share * Reporting Period MMC Share*MMC Enrollee*Reporting Period
(1) 0.122 [0.051]* -0.114 [0.058]*
Observations 4271 R-squared 0.01 Test of MMC Share + MMC Share*HMO=0 (p-value) Test of MMC Share*Report + MMC Share*HMO*Report=0 (p-value) Robust standard errors in brackets + significant at 10%; * significant at 5%; ** significant at 1%
4160 0.18
(3) -0.088 [0.085] -0.037 [0.112] -0.237 [0.297] 0.679 [0.311]* 0.048 [0.154] -0.255 [0.359] 4126 0.18 0.25 0.60
Results for Beta Blocker
Table 5: Multivariate Models of Utilization of Performance Indicators Beta Blocker (2) -0.026 [0.033] 0.018 [0.040]
MMC Enrollee MMC Enrollee*Reporting Period MMC Share MMC Share * MMC Enrollee MMC Share * Reporting Period MMC Share*MMC Enrollee*Reporting Period
(1) -0.017 [0.030] 0.026 [0.039]
Observations 6387 R-squared 0.02 Test of MMC Share + MMC Share*HMO=0 (p-value) Test of MMC Share*Report + MMC Share*HMO*Report=0 (p-value) Robust standard errors in brackets + significant at 10%; * significant at 5%; ** significant at 1%
6148 0.18
(3) -0.032 [0.062] -0.007 [0.084] -0.294 [0.229] 0.127 [0.217] 0.299 [0.141]* -0.059 [0.276] 6110 0.18 0.64 0.35
Results for Flu Shot
Table 5: Multivariate Models of Utilization of Performance Indicators Flu Shot (2) 0.037 [0.014]** 0.009 [0.021]
MMC Enrollee MMC Enrollee*Reporting Period MMC Share MMC Share * MMC Enrollee MMC Share * Reporting Period MMC Share*MMC Enrollee*Reporting Period
(1) 0.037 [0.020]+ 0.012 [0.023]
Observations 39789 R-squared 0.02 Test of MMC Share + MMC Share*HMO=0 (p-value) Test of MMC Share*Report + MMC Share*HMO*Report=0 (p-value) Robust standard errors in brackets + significant at 10%; * significant at 5%; ** significant at 1%
38326 0.12
(3) 0.011 [0.030] 0.018 [0.040] 0.058 [0.113] 0.077 [0.107] -0.064 [0.061] 0.008 [0.154] 38041 0.12 0.33 0.70
Results for Eye Exams for Diabetics
Table 6: Post-Reporting Period Analysis of Eye Exams for Diabetics
MMC Enrollee MMC Share MMC Share * MMC Enrollee Constant Observations R-squared Robust standard errors in brackets * significant at 5%; ** significant at 1%
(1) -0.006 [0.024]
(2) -0.011 [0.027]
0.681 [0.018]** 3237 0
0.698 [0.093]** 3156 0.16
(3) -0.046 [0.060] 0.56 [0.872] 0.134 [0.207] 0.579 [0.210]** 3156 0.16
Summary of Results
• Rates of utilization of the performance indicators we studied did not increase more rapidly among Medicare HMO enrollees than beneficiaries enrolled in traditional Medicare subsequent to the implementation of mandatory quality reporting. • The lack of a positive effect of quality reporting on utilization of performance measures among HMO enrollees is not sensitive to controlling for either selection of enrollees across the sectors or spillovers from quality reporting activities to the FFS sector. • Utilization rates of beta blockers increased more rapidly for FFS beneficiaries in markets with high levels of managed care penetration in the post-reporting period.
Implications
• The implementation of mandatory health plan quality measurement and reporting did not increase utilization of performance measures among Medicare managed care enrollees for the services we studied.
– Report cards did not create strong enough incentives for plans to improve their performance.
– Quality improvement programs were in place in HMOs serving the commercial population prior to the implementation of mandatory participation by the Medicare program.
– Indicators we studied are not those that were likely to have shown improvement.
• Quality reporting by HMOs may have had positive spillover effects on the Medicare FFS population.
Research Question
• Did the implementation of mandatory health plan participation in standardized performance measurement and reporting improve quality of care among enrollees in Medicare Managed Care plans?
Health Plan Performance Measurement in Medicare
• In 1996, HCFA contracted with NCQA to develop HEDIS measures for Medicare managed care plans. • HCFA began requiring plans to submit Medicare HEDIS data as of January 1997. • In 1998, CMS launched www.Medicare.gov. • In 1999, HEDIS measures became available on the site.
Model Estimation
Model 1: Unadjusted difference-in-difference estimate
yi ,m,t a 1H i ,m,t 2 I t * H i ,m,t Z t i ,m,t
where i indexes individuals, m indexes counties, t indexes years (1993-1999) Y is a binary indicator of utilization of service; H is an indicator of enrollment in a Medicare managed care plan;
I is an indicator of whether mandatory quality reporting had been implemented during the time period; and Z includes year fixed effects
Model 2: Control for selection based on observable characteristics
Controls include individuals characteristics (age, sex, education, marital status, race, ethnicity, selfreported history of 16 conditions, self reported health status, number of ADLs, BMI indicators, and smoking status) and county fixed effects.
Model 3: Control for selection and spillover effects using market share variables
Identifying Selection and Spillover Effects (Model 3)
yi ,m,t a 1H i ,m,t 2 I t * H i ,m,t 3 S m,t 4 S m,t * H i ,m,t 5 S m,t * I t 6 S m,t * H i ,m,t * I t X i ,m,t Am Z t i ,m,t
Coefficient B3
B4 B5
Variable Managed Care Share
Managed Care Share*HMO Enrollee Managed Care Share*Intervention Period
Effect Spillover and Selection in the preintervention period
Selection in the pre-intervention period Spillover and selection due to quality reporting intervention
B6
B3-(- B4)
Managed Care Share*HMO Enrollee*Intervention Period
Selection due to quality reporting intervention
Spillover from HMO enrollment onto FFS beneficiaries (pre-intervention)
B5-(- B6)
Spillover from HMO enrollment onto FFS beneficiaries due to quality reporting