Your Federal Quarterly Tax Payments are due April 15th Get Help Now >>

2005 IQLM Conference by NIQe3C


									            2005 IQLM Conference

                 Indicators Workgroup
                     April 29, 2005

Department of Health and Human Services
IQLM Thanks The Quality Indicators
Workgroup Members

Workgroup Co-Leaders:          Workgroup Members
• Lee Hilborne, MD, MPH        • Raj Behal, MD (UHC)
  (UCLA/RAND)                  • Lucia Berte, MA (Consultant)
• Frederick Meier, MD (HFHS)   • Robert Dufour, MD (VAMC)
                               • Diane Feeney, BSN, MSc (NQF)
CDC Co-liaisons/Indicator
  Reviewers:                   • Linda Hanold, MHSA (JCAHO)
• Shahram Shahangian, PhD      • Alan Hoffman, MD (NCQA)
• Susan Snyder, PhD            • Robert Pendrak, MD (Inservco
                                 Insurance Services)
• Pamela Thompson, MS
                               • Paddy Sundararajan, PhD (Quest
CDC Supervision:               • Richard Zarbo, MD, DMD (HFHS)
• Joe Boone, PhD
• Devery Howerton, PhD         Former Workgroup Members:
                               • Merilyn Francis, MPP (NQF)
                               • David Sundwall, MD (ACLA)
Consider the following question
• If Congress wanted to quickly know about
  America’s laboratories, what information
  should they have?
   – Laboratory medicine’s contribution to the nation’s
   – What is important to the nation should be
     important to its laboratories

• If IQLM goes off by itself, the relevance will
  be lost
But Aren’t Laboratory Tests
Relevant To Clinical Monitoring?
• Consider 24 common clinical outpatient
  conditions with explicit evidence for a specific
  course of evaluation or treatment
• In how many of these common conditions are
  laboratory tests part of the diagnosis or
   – Involved in Diagnosis:                50%
   – Involved in Treatment Monitoring:     38%
   – Involved in Diagnosis or Treatment:   63%
• No other specialty touches so many clinical
  situations that frequently impact patients
The Indicators Group Was Given
A Specific Agenda
• Define a core indicator set for laboratory practice
   – IOM quality domains
   – Cover total testing process
   – Practice settings
• Be judicial in selection
   – Perhaps 3-5 domains, 3-5 measures per domain
• How should indicators be used?
   – Surveillance for quality across organizations
   – Identification of best practices for awards
   – Ability to monitor through the IQLM Network
• Review and incorporate existing evidence
   – Identify evidence gaps related to core indicator set
   – Determine strategies to fill the gaps
Indicators Fit Into The Institute
                                       Develop Network of
Awards and Grants                      Laboratories & Partners

                     Awards      Network

                     National   Laboratory
                     Report     Indicators
Identify Issues                       Indicators to Benchmark
And Best Practices                    And Monitor Progress
Our Job Was To Provide Initial
Direction Regarding Indicators
Quality Indicator Characteristics
Should Be Broad and Defensible
• Evidence-based, practice-tested measures of
  IOM health care domains
• Associated with identifiable health care
  quality problems
• Objective metrics that can be implemented in
  a standardized and comparable format
  – Employ standard methods for evaluation
• Reliable means of external and internal
  evaluation of quality performance over time
• Address a wide range of laboratory tests and
  testing sites
We Considered How To Prioritize
Indicator Efforts
• What is important?
• Where should we focus?
• In what testing venues will indicators
  be applicable?
• How can we avoid reinventing the
• How much can we do at the beginning
  without biting off too much?
Some Important Concepts
•   Link to National Quality Report
•   Define scope of analysis
•   Information is actionable
•   Data efforts are doable (practicality)
    – Feasible
    – Affordable
• Submissions should be auditable
While All Phases Of The Total
Testing Process Are Important…

Pre-Analytic    Analytic
   Phase         Phase         Phase

      We Concluded Focusing On
   Pre And Post Analytic Phases Will
       Have The Greatest Impact
IQLM, and Indicators Should
Acknowledge All Stakeholders

•   Laboratory professionals
•   Accrediting organizations
•   Standards developers
•   Administrators
•   Diagnostic industry
•   Physicians and others providing direct patient care
•   Patients and public
•   Policy makers
•   Payers and purchasers
•   Delivery systems
•   Proficiency testing (?)
Indicators Must Be Sensitive To
Different Testing Venues
•   Hospital Laboratories
•   Physician Office Laboratories
•   Reference Laboratories
•   Public Health Laboratories

• And Consider Different Perspectives
    – What the laboratory needs to know
    – What laboratory users need to know
    – What patients and consumers need to know to
      select a laboratory
Look To Partners Who Have
Considered Questions Before
• College of American Pathologists
   – Q-Probes and Q-Tracks
• National Committee on Quality Assurance
   – HEDIS
• Joint Commission on Accreditation of Health Care Organizations
   – ORYX
   – National Patient Safety Goals
• Veterans Administration
• Agency for Healthcare Research and Quality
   – National Healthcare Quality Report
   – Natl Quality Measures Clearinghouse/Ntl Guideline Clearinghouse
• US Preventive Services Task Force
• Centers for Disease Control and Prevention (e.g., MMWR)
• Published work (Medline, data mining, others)
We Reviewed The Literature And
Found Many Potential Indicators
Table 1. Laboratory-Based Q uality Indicators Grouped by the Stage of the Total Testing Process, Mapped into the IO M Domains, and Scored Against Desireable Attributes for Q uality Measures
Indicators                                                                                       IO M Domain                                                  Evaluation Criteria
                                                                   Safety Effectiveness Equity Patient centeredness Timeliness Efficiency Importance Scientific Acceptability    Feasibility Usefulness
Informed decision making upon test ordering                                                               
Screening for hyperlipedemia (lipid profile)                                             
Screening for colorectal cancer (fecal occult blood test)                                
Screening for cervical cancer (Pap test, HPV test)                                       
Screening for infectious disease (chlamydia, HIV, HCV, etc.)                            
Screening for antimicrobial susceptibility                                                                                        
Screening for malnutrition (hospitalized elederly)                                                                                  
Prenatal/Neonatal screening (blood typing, AFP, HIV, PKU, etc.)                         
Diabetes screening/management (HbA1c, LDL-cholesterol, etc.)                             
Blood type and screen for scheduled surgeries                                                                                     
Blood utilization and wastage indicators                                                                                          
Blood culture (management of pneumonia)                                                                                           
Monitoring of oral anticoagulant therapy (PT )                                                                         
T herapeutic drug smonitoring                                                                                          
T est order accuracy                                                                                                               
Patient (wristband) identification                                                                                                 
Patient preparation for specimen collection                                                                                         
T iming of specimen collection                                                                                                    
Complications, discomfort, and satisfaction with phlebotomy                                               
Specimen integrity/quantity                                                                                                        
Blood culture contamination                                                                                                        
Urine culture contamination                                                                                                        
Specimen preparation for analysis                                                                                                  
Specimen transportation                                                                                                            
Accuracy of specimen identification (labeling)                                                                                     
Condition for specimen storage                                                                                                     
QC performance                                                                                                                      
Proficiency testing/Performance evaluation                                       
Frozen section diagnostic discordance and false negative rate                                                                      
Cervicovaginal cytology and cervical biopsy results correlation                                                                    
Autopsy-detected diagnostic errors                                               
Laboratory result availability                                                                                                      
Amended reports                                                                                                                    
Laboratory/Clinical information system                                                                                            
Adequacy of reporting content                                                                            
Critical values reporting                                                                                                         
T urnaround time (and clinician's/patient's satisfaction with it)                                                        
Result interpretation by clinician/patient                                                                                        
Clinical/Preventive action (noting results in medical record)                                                                     
Laboratory safety practice indicators                                                                                              
Customer (clinician/patient) satisfaction with laboratory services                                                                
Competency of testing personnel                                                                                                   
Availability (shortness) of testing personnel                                                                                     
Then Focused Down Using Some
Guiding Principles
• A limited number of indicators relevant to the
  quality agenda
   – No more than about 8-12
   – If there are too many, the process won’t be
     deemed credible
• Should cover various laboratory venues
• If at all possible, use existing, validated
   – Develop new or test obscure indicators only if
     absolutely necessary
We Identified Two Main Indicator

• Systems Indicators
  – Involve interactions between the
    laboratory and laboratory customers
• Laboratory Quality Indicators
  – Primary impact is on the provision of care
    by the laboratory
  – Specific total testing process subsets
     • Pre, intra and post analytic components
The Following List Encompasses
The Highest Priority Items
•   Diabetes monitoring                           (system)
•   Hyperlipidemia screening                      (system)
•   Patient identification                    (preanalytic)
•   Test order accuracy                       (preanalytic)
•   Blood culture contamination               (preanalytic)
•   Adequacy of specimen information          (preanalytic)
•   Accuracy of Point of Care Testing            (analytic)
•   Cervical Cytology/Biopsy Correlation         (analytic)
•   Critical value reporting                 (postanalytic)
•   Turnaround time                        (infrastructure)
•   Clinician satisfaction                 (infrastructure)
•   Clinician follow up                  (system/general)
IQLM’s Initial List Closely Parallels That
Recommended by Dr. Howanitz last year
      Measure                 Discipline             Phase           Frequency
Customer Satisfaction              All            Pre, Intra, Post       Annual

Turnaround Time                 Chemistry,        Pre, Intra, Post       Monthly
Patient Identification             All             Preanalytical         Monthly

Specimen Acceptability          Chemistry,         Preanalytical         Monthly
Proficiency Testing                All              Analytical       6-20 specimens/
                                                                     analyte per year
Critical Value Reporting           All            Postanalytical         Monthly

Blood Product Wastage      Transfusion Medicine   Postanalytical         Monthly

Blood Culture                  Microbiology        Preanalytical         Monthly

                                                            Presented at CAP 04
There Are Some Key Dimensions
That Remain To Be Addressed

• We’ve covered issues of underuse and
  access in several measures
• We need to consider a focus on
  – Overuse (appropriateness)
  – Misuse
We Started By Reviewing The
Twelve Selected Indicators
• Summarize and describe the evidence so
  practical healthcare decisions are feasible
• Use defined recognized methods for reviews
  and desirable quality measure attributes
• Maintain a practical perspective consistent
  with the constraints imposed by the limited
  availability and quality of evidence, producing
  a comprehensive, objective and reproducible
Indicators Were Evaluated Using
A Structured Process
• Formulate the problem
  – Conceptual framework
• Search strategy
• Quality and validity assessment of studies
  – Inclusion criteria
• Collect and analyze findings
  – Evaluation criteria/template
• Interpret findings and present results
• Update findings

                   Cochrane Handbook for Systematic Reviews of Interventions, 3/05
Evaluation Was Approached With
This Conceptual Framework
                                                      Mortality, DALY,
                                                       QALY, Cost
    Properties    Indicator

  Problem                        Intermediate         Health
Specification                     Outcomes           Outcomes

Issue The
Addresses        Interventions
                                 Actions To     Associated
                                  Change        w/ desired
                                 Outcome          Health
Evaluation Criteria Followed
Four Primary Dimensions
•   Importance
•   Scientific acceptability
•   Feasibility
•   Usefulness
Evaluation Criteria: Importance
• Health importance
  – Ability to meaningfully impact populations
  – Measures an important quality aspect(s)
     • Common: high prevalence/incidence
     • Impact: serious impact on health outcomes
• Potential for improvement
  – Need supported by quality variation or
    substandard quality
  – Literature or expert opinion support (e.g.,
    effective interventions)
Evaluation Criteria:
Scientific Acceptability
• Strength of evidence based on peer reviewed
  – Quality problem is explicitly defined
  – Indicator links specifically to the problem
  – Indicators must be reliable and valid
     • Findings are consistent among raters
     • Accurately measures desired attributes
• Other sources for future consideration
  – Professional organizations
  – ? Expert opinion
Evaluation Criteria: Feasibility
• Data definitions are sufficiently clear
   – Abstraction tools can be developed
   – Data abstraction quality easily standardized
       • Manual: Trained individuals for consistency
       • Electronic: Clear data fields easily extractable
• Ability to broadly implement indicators
   – Across multiple similar laboratories
   – Across different laboratory types
• Benefits of measurement exceed financial and
  administrative burdens
   – Burdens: need to collect new data, abstraction time, analysis
     time, health impact of erroneous results
   – Benefits: health improvement, reduced rework, reduced
Evaluation Criteria: Usefulness
• Relevant
  – Stakeholder(s) find the indicator useful
  – Acceptance by laboratories, clinicians and other
  – Relevance extends to the healthcare system
    (beyond the laboratory)
• Opportunity to impact health system
  – Interventions within stakeholders’ sphere of
  – Actionable findings to guide organizational
    decisionmaking and inform public policy
Here’s An Example of Indicator

                  % of specimens
                  w/ inaccurate or
                  inadequate info

 Specimens w/                         •Reporting/treatment
                                       error or delay
  inaccurate or                                               Mortality
                                      •Patient satisfaction
inadequate info                                                Cost
                                      •Cost of Error

                   rejection policy
                  •CPOE system
                  •Label bar coding
Accuracy And Adequacy Of
Specimen Information
• Definition: Percent of specimens sent to
  laboratory with inaccurate or inadequate
  – No label, illegible, no patient information
  – No tissue source or clinical information when
• Population: Specimens sent to laboratory
• Providers: Locations where specimens are
• IOM domains: Safety, timeliness, efficiency
The Degree Of Support For
Criteria Domains Is Variable
   Current information supporting evaluation
    criteria appears adequate

   Current information supporting
    evaluation criteria is equivocal/uncertain

   Current information supporting evaluation
    criteria is limited or does not exist
Accuracy And Adequacy Of Specimen
Information: Importance= 
• Prevalence/incidence: 
  – Surgical specimens: 1.2-2.2% (CAP)
  – Clinical lab specimens 0.015-0.030% (CAP)
• Impact: 
  – Limited data available regarding impact on
• Potential for improvement 
  – Variation suggests opportunities
  – Monitoring and feedback insufficient
Accuracy And Adequacy Of Specimen
Information: Acceptability= 
• Strength of evidence: 
  – Quality problem defined: 
  – Indicator linked to quality problem:     

  – Professional society acceptance: 
• Reliability and validity: 
  – Validity: No explicit information:   

  – Reliability: CAP studies suggest
    consistency: 
Accuracy And Adequacy Of Specimen
Information: Feasibility= 
• Clear data definitions: 
  – Standardization of data definitions remains an
  – JCAHO has developed standard definitions
• Implementable: 
  – Many studies have been done in multiple
• Reasonable Cost:
  – No data specifically documenting cost to study
  – Multiple studies suggest cost is not excessive
Accuracy And Adequacy Of Specimen
Information: Usefulness= 
• Relevance to users, stakeholders: 
  – Useful for performance improvement
  – Not explicitly on any national metric
• Operationalizable:
  – No specific data on operationalizing this metric
  – Findings suggest strategies can be developed
    improve performance
     • Specimen rejection policy dissemination and enforcement
     • Bar codes on specimen labels
What Do We Currently Know About
The First Indictors Evaluated?


            =Pending Review

   Diabetes monitoring
   Hyperlipidemia screening                                                      
   Patient identification                                                      
   Test order accuracy/appropriateness                                           
   Blood culture contamination                                                   
   Adequacy of specimen information                   
   Accuracy of Point of Care Testing
   Cervical Cytology/Biopsy Correlation
   Critical value reporting                 
   Turnaround time
   Clinician satisfaction                   
   Clinician follow up
The Workgroup Raised As Many
Challenges As Solutions
• Laboratory indicators are limited
• Strength of evidence linking laboratory indicators to
  health outcomes is weak
   – Lack of evidence for health outcomes
• Literature does not directly address many defined
  quality problems or review questions
• Lack of standard definitions limits comparability of
• Quality of evidence, generalizability and applicability
  not evaluated
• Many laboratory indicators have limited relevance to
  national health priorities
Some General Themes Emerge
• System indicators likely are better supported
  by the evidence on health outcomes
  – Selected because of their impact on patient
  – Results are linked to evidence supported specific
    care interventions
• Laboratory indicators are less frequently
  supported by ties to health outcomes
  – Most of the work has been done by CAP
  – Definitions for many are clear or could be
  – Linked to intermediate outcomes but links to
    health outcomes are generally inferential
Issues For The New IQLM To
• Solidify a definition of quality as it relates to
  laboratory practice
• Assure selected/future indicators map to
  laboratory quality definitions
• Focus on test utilization: overuse, underuse
  and misuse of testing services
• Consider whether intermediate outcomes
  should be sufficient
   – Identify direct and indirect ways to link
     intermediate processes to health outcomes
   – Accept intermediate outcomes as final outcomes
     for most laboratory services
Logical Next Steps for Indicators
Once IQLM Direction Is Clear
• Begin a broader discussion of indicators
  with key stakeholders
• Link indicators to awards and network
• Determine the extent to which indicator
  validation will be studied
• Select one or two for more in depth

To top