Assessment of Leadership Performance by jdw44147


More Info
									    AHQA – 2008 Annual Meeting

Hospital Leadership Quality
Building A Technical Assistance Program For
      Hospital Quality Improvement
      Hospital Leadership Collaborative
             February 27, 2008

                                              Health Benchmarks Inc
     1. Overview of HLQAT project
     2. Origins – Leadership Survey (Short)
     3. HLQAT Scope & Phases
     4. Testing the Model
     5. Roles of the QIOs
     6. Anticipated value of the project

      1. HLQAT Project (2008-2010 & beyond)
     Integrate activities of the HLC with the CMS 9th SOW to drive
       hospital quality improvement.
      Develop and test HLQAT as a tool for measuring domains of
       hospital leadership that drive change.
      Develop tools to provide support to the QIOs in giving technical
       assistance to low-performing hospitals under the CMS 9th SOW.
      Apply measures of hospital quality performance to evaluate
       effectiveness of technical assistance protocols and test HLQAT as an
       agent of change.

Key Project Staff
     University of Iowa:  Barry R. Greene, PhD, Tom Vaughn,
      PhD, Samuel Levey, PhD , Duncan Moore, FACHE
     Brandeis University: Chris Tompkins, PhD
     HBI: Judy Chen, Ph.D., Josh Marehbian, MPH
     OFMQ: Dale Bratzler, DO, MPH, Shannon Archer RN, CPHQ
     Premier-CareScience (Penn-LDI): Eugene Kroch, PhD
     HSAG: Andrea Silvey, PhD
     Dot.Comments: Chris Hatcher, MHA
     CMS: Ninth Scope of Work
Oversight (and collaboration)
     AHA – Steve Mayfield
     IHI – James Conway (via IHI Impact Network)

2. Origins – Leadership Survey (2005-2006)*
     Identify how hospital leadership is involved in quality
     Link survey results to hospital quality outcomes (Quality
     Share findings to promote a strategic approach to quality
      improvement in hospitals based on empirical findings.
     Collaborators:
       Centers for Medicare and Medicaid Services
       Univ. of Iowa College of Public Heath
       Leonard Davis Institute of Univ. of Pennsylvania
       Hospital Associations from 9 states
          *Vaughn et al., “Engagement of Leadership in quality Improvement Initiatives:
          Results from the Executive Quality Improvement Survey,” J Pat Safety, 2006.
          *Kroch et al., “Hospital Boards and Quality Dashboards,” J Pat Safety, 2006.
    Leadership Survey
      18-question survey distributed via internet in early 2005
       to 1,380 hospitals in 9 states: AZ, CO, IL, IA, MD, NJ,
       NY, PA, and WI.

      438 usable hospital responses (rate = 32%)

      CEOs (55%), QI execs (25%), CMO/CNO (13%)

      Examines hospital QI drivers and impediments, reporting
       methods, board and physician participation in QI, and
       senior executive incentives.

    Descriptive Findings
     24% of boards interact with the medical staff “a great
        amount” in setting hospital quality strategy.
       27% of boards spend more than one fourth of their time on
        quality issues.
       66% of hospitals base some type of executive compensation
        on measurable Quality Improvement.
       BUT only 13% of hospitals tie quality improvement to
        executive base compensation packages
       80% of responding hospitals use a formal quality
        performance measurement “dashboard” for reporting to their

                  Linking Leadership Survey to
                 CareScience Quality Index (Qx)
     Measures the risk-adjusted overall rate of adverse outcomes
       Mortality
       Morbidity
       Complications
     Uses the Corporate Hospital Rating Project* utility weights to
      construct an index (Qx)
     Responses were matched to Qx derived from H.A. “all-patient”
      data and MedPAR 2004 data
      *Pauly MV, Brailer DJ, and Kroch EA. Measuring Hospital Outcomes from a Buyer’s
      Perspective (1996). American Journal of Medical Quality. 11 (3): 112-122.

                                             Quality Index Distribution
                                         A Comparison across Samples
                                                                        2747 hospitals in 19 alll-payer states
                                                                        mean=100.3; stdev=6.8
              12                                                        1378 hospitals in 9 survey states
                                                                        mean=100.8; stdev=6.47
                           Low Quality (bottom third)
              10                                                        438 hospitals in matched sample
                                                                        mean=100.7; stdev = 6.72
Percent (%)

                                                                        High Quality (top third)



                      69       77       85       92     100     108        115       123       131          138
                                                        Quality Index

          Better outcomes found in hospitals where...
     1.   the board spends >25% of time on quality issues (p =
     2.   the board receives a formal quality performance
          measurement report (p=0.005);
     3.   there is a high level of interaction between the board and the
          medical staff on quality strategy (p=0.021);
     4.   the senior executives’ compensation is based in part on QI
          performance (p=0.008);
     5.   the CEO is identified as the person with the greatest impact
          on QI (p=0.01), especially when so identified by the QI
          executive (p<0.001).

                                    Strong Quality Leadership
                                          NB: numbers above bars are case counts

          120                          n=92

                   Four Criteria for "selected" hospitals:
Q Index

           80                   > 25% of board time on quality
                         formal quality measurement report to board
                            quality-based incentive compensation
           60   great deal of board interation with medical staff for QI strategy



                                     Selected                                       Others
                                     Relationship between Qx and Most Influential Position
                                                                  N= 438
                               3.5                                                              CMO/QI Exec
Ratio of High/Low Performers

                                                            Hospitals where the CMO/QI exec identifies
                                                            the CEO/Pres as the most influential person
                               2.5                          are about three times more likely to be in high
                                                            performance group (p-value < 0.001).

                               1.0                                                 0.82

                                             CMO/QI Exec                              CEO/COO
                12                                              Respondent
       3. Scope of the HLQAT Project

      Baseline links between hospital leadership’s involvement
       in quality improvement and hospital performance
      Interventions to be carried out at designated hospitals
          Based on outcomes/process gaps
          Based on HLQAT-identified opportunities for change

      Follow-up analyses will measure changes in performance

      Follow-up HLQAT will assess change of hospital
       leadership’s involvement in quality improvement
          Relative to comparison group

     HLQAT Project Phases
     •Phase I – Preparation for Technical Assistance
        • HLQAT validation and assessment
        • Identify low performers April to Sept 2008
        • Develop TA protocols
     •Phase II – Implementation of Technical Assistance
        • TAP training               Sept 2008 to Dec 2009
        • Interventions

     •Phase III – Empirical Evaluation
        • Hospital Performance evaluation
        • HLQAT re-evaluation       Jan 2010 and beyond

     HLQAT Activity/Intervention Flow

    HLQAT Assessment Domains
        Path to Quality Improvement


            *Silvey et al., “Components Essential to Achieving ‘High
16          Performer’ Status,” HSAG, 2005
        4. Testing the Model
     1. ASSERTION: Technical Assistance Protocols (TAP) can
        strengthen Hospital Leadership (HL)
     2. ASSERTION: The dimensions of HL are significant determinants
        adherence to Clinical Processes (CP).
     3. THUS: HL improvements can be tested for their effects on CP.
     4. ASSERTION: CP are hypothesized to be significantly related to
        Clinical Outcomes (CO)*.
     5. THUS: Causal chain to test:

                       TAP→ HL→ CP →CO
           *In a subsequent phase of the project the test is for the extent to which
           specific CP improvements are related to targeted Clinical Outcomes.
     Empirical Method
     1. Score hospital performance on a combination of CP and
        CO consistent with CMS priorities.
     2. Scoring measures criteria:
          Workable for QIO technical assistance
          Relevant to quasi-experimental method (Regression
           Discontinuity Design)
     3. Choose threshold values such that:
          Hospitals scoring below a chosen threshold will be
           formally assigned to receive QIO technical assistance
          Hospitals scoring above a threshold will be assigned as
           control hospitals, and will not receive QIO technical
          Option: Choose both low and high thresholds, (i.e.,divide the hospitals into
           three (versus two) groups), and allow for technical assistance to be delivered to
18         hospitals falling between the thresholds
     Performance Metrics

      Clinical Process (CP) measures
        JC-CMS-HQA core measures
        CMS – SCIP measures
        Other (stroke, asthma, etc.)
      Clinical Outcomes (CO)
        Mortality
        Complication rates
        ALOS
        Readmissions
        AHRQ PSIs

      Surgical Complications Methodology

      A combination of a data-driven and literature-based approach is taken
        to identify about 30 procedures to include in the methodology
           Complications for each procedure are classified
           A risk-adjustment methodology accounts for patient factors
                Age
                Gender
                Type of procedure
                Comorbidity level (Charlson Index)
                Disease burden
                Other patient factors

       Surgical Complications Methodology

      STEP I: Identify the relevant procedures
        a. Inpatient data are aggregated across all hospitals
        b. Frequencies of all procedures are obtained and procedures
             are grouped by clinical team
        c.   The most commonly performed procedures are identified
        d.   Primary and non-primary diagnosis codes related to these
             patients are identified
        e.   Literature is searched for complications
        f.   For each procedure, a comprehensive list of complications is
             developed, driven by literature, data, and expert review

     Surgical Complications Methodology

     Step II: Classification of complications and comorbidities

     ICD-9                                                  ICD-9
                     ICD-9 Code Description                                 ICD-9 Description
       Dx                                                     Dx
     99812   Hematoma complicating a procedure                      Asthma, unspecified type, without
                                                                    mention of status asthmaticus
     99811   Hemorrhage complicating a procedure
                                                           78057    Other and unspecified sleep apnea
     99859   Other postoperative infection
                                                                    Nondependent tobacco use
     5990    Urinary tract infection, site not specified   3051
             Other specified complications of
     99889                                                          Depressive disorder, not
             procedures not elsewhere classified           311
                                                                    elsewhere classified
             Respiratory complications, not
     9973                                                  2749     Gout, unspecified
             elsewhere classified
                                                           V1582    Personal history of tobacco use
             Accidental puncture or laceration during
             a procedure, not elsewhere classified         6173     Endometriosis of pelvic peritoneum

       Surgical Complications Methodology

      STEP III: Run Model on MedPAR Claims Data
        a. Identify procedures during our measurement period
        b. Identify relevant complications during the 1-30 day period
          post surgery
           a. Inpatient and outpatient diagnoses
        c. Crude complication rates calculated
        d. Identify comorbid conditions before and after procedure
           (Charlson Index)
        e. Predicted (risk-adjusted complication rates calculated)
        f. Rates and benchmarks reported

     Regression Discontinuity Design

     Specific Hypotheses
     1.   H1: Hospitals receiving QIO interventions based on the TAP
          will significantly improve in clinical quality (measured by CP and
          CO) relative to control hospitals.

     2.   H2: Hospitals receiving TAP interventions will significantly
          improve in domain scores related to hospital structures
          (measured by HLQAT) relative to controls.

     3.   H3: Prior improvements in hospital structures are associated
          with improved clinical quality (measured by CP and CO).

     5. Roles of the QIOs
        Under 9th SOW QIOs are the obvious means to helping
         low-performing hospitals.
        Coordination, engagement, and support across QIOs for
         an national effort will enhance the likelihood of success.
        Input during validation phase of the HLQAT
        TAP design based on the HLQAT domains
        HLQAT and TAP training (CMS support contracts)
        TAP implementation (CMS support contracts)

         CMS 9th SOW
      QIOs are to focus on these main themes (Section C.6):
         C.6.1. Beneficiary Protection
         C.6.2. Patient Safety
         C.6.3. Prevention.

      “QIOs will be required to offer help to specific hospitals that have not
       recently performed well on important quality measures.”
      “The requirements of the Patient Safety Theme are designed to address areas
       of patient harm for which there is evidence of how to improve safety by
       improving health care processes and systems.”
      Required activities include:
        “Administer and collect results of the Healthcare Leadership and
          Quality Assessment Tool (HLQAT)….”
                                   Source: CMS web site

     QIO Interventions
      “Utilize these tools for providing tailored and structured
       interventions to identified participant groups that will ultimately
       assist them in reaching the Achievable Benchmarks of Care….The
       QIO shall administer, collect and utilize the results by the 18th
       month of the effective date of contract and then readminister
       between months 18 and 35.”
      Under QIO support contract CMS will provide training on
       the HLQAT – via series of meetings (not yet finalized)
      HLQAT domains
        Culture – knowledge seeking, goal setting, communication,
         collaboration, support
        Policy & operations – roles & responsibilities, monitoring,
         manamgement strategy, incentives, resource allocation

      Hawaii Hospital Incentive Program
  In 2001, the Hawaii Medical Service Association (Blue Cross
     and Blue Shield of Hawaii) initiated the Hospital Quality
     Service Recognition Program.
      Designed to provide financial bonuses to facilities based on performance across
       several measures
  Hospitals were eligible to receive payments based on their
     total reimbursement from the plan
                           5-Year sum was paid based on Combined
      Portion of their eligible Trends for All HQSR Hospitals their performance across measures

              Risk Adjusted Figures

                                                                 Surgery Length of
                                                                 Maternity Length
                                                                 of Stay

32                                    Time (2002-2006)
       Facility, QIO, and Health Plan Collaboration

      HLQAT will be integrated in the Hawaii hospital incentive
       program. Participation will be tied to bonuses.

      Hospitals will implement the HLQAT tool at different levels
       within the facilities.

      HBI will measure baseline and follow-up outcomes and

      QIOs will work with hospitals to implement interventions.

     6. Anticipated Value of the Project
     1. Creation of an accessible, electronic version of the HLQAT,
       and benchmark data, to support hospitals as they assess themselves
       in terms of their shared culture and commitment to improved

     2. Development of a technical assistance toolkit (TAP) for QIOs
        and other groups working with hospitals that can be used to match
        interventions with specific gaps in perceived commitment to a
        culture of excellence, or with shortfalls in organizational capacity
        to effect change.

     3. Improved quality of care, as measured by the metrics described
        in the proposal, in hospitals receiving consultation.

         Questions – Comments

      Building A Technical Assistance Program For
            Hospital Quality Improvement

    Eugene A. Kroch, PhD –
Josh Marehbian, MPH –

                                                    Health Benchmarks Inc

To top