Enhancing Validity of Physicians Economic Profiles I

Document Sample
Enhancing Validity of Physicians Economic Profiles I Powered By Docstoc
					ENHANCING VALIDITY OF
PHYSICIANS’ ECONOMIC
      PROFILES


       Bill Thomas
  University of Southern Maine
Alternative Session Title



 8 Ways to
 Screw Up
 Identification of
 Cost Efficient
 Providers
                     “Whoa! Watch where that thing
                     lands.. we’ll probably need it.”
Types of economic profiling

   PCP Profiling
    • Assumes gate-keeper HMO model
   Specialist Profiling
    • Assumes care can be partitioned into
      episodes, and episodes can be attributed to
      specific physicians
     Who does economic profiling of
    physicians?

   Health plans! Lots & lots of health plans!!
    • National plans (Aetna, CIGNA, Humana, United,
      Wellpoint) and regional plans (too many to list)
       • Symmetry Health Data Systems, the leading vendor of
         episode grouper software used in specialist profiling,
         says it has 400 health plan clients with combined
         membership of almost 200 million people
       • Other episode grouper software vendors include
         Thomson Medstat (Medstat Episode Grouper), and the
         Cave Grouper
    • Why? Pressure from employers, concerns about
      cost control and quality
Who does economic profiling of
physicians?

   Physician organizations
    • Why? Monitor productivity, enhance
      profitability
Data source for economic profiling

   Health plan claims databases
    • Claims have patient ID, provider ID, dates of
        service, Dx codes, Px codes
    •   Types of claims
         • Inpatient
         • Outpatient
         • Professional
         • Pharmacy
    • Are pharmacy claims always available?
How are physicians’ scores calculated? steps
in economic profiling:


 1.   Claims are partitioned into episodes by
      episode grouper software
 2.   Actual cost of each episode is calculated
 3.   Expected cost of each episode is calculated
 4.   Responsibility for each episode is attributed
      to a physician
 5.   Sums of actual costs and expected costs are
      calculated for each physician
 6.   Physician score is calculated – e.g., ratio of
      actual to expected costs
What is the physician score called?

   This is a VERY contentious subject!!
    • Health plans tend to call it efficiency, but
        health economists and many physicians do
        not like this terminology AT ALL.
    •   Well, how about…….relative resource use?
    •   Too clumsy? How about cost of care?
    •   Doesn’t work either? What about costliness?
    •   No? Let’s just use cost efficiency.
What are we trying to do in economic
profiling?

   We are trying to rank physicians in terms
    of cost efficiency performance.
   All physicians?
    • No. We compare physicians within specialty.
    • Why? So we’re comparing physicians who are
      treating similar types of cases.
What are we looking for?
We’ve looking for accurate measures so we can correctly
classify physicians on cost efficiency performance.




                     A           B             C




   0           0.5               1                 1.5   2
                         Physician O/E Ratio
  Can we screw up the performance
  measurement process?

Yep! It’s easy. We’ve got
  at least 8 ways to screw
  it up.
We can:
8. Fail to deal with distorting
    effects of outliers.
7. Incorrectly attribute episodes
    to physicians
6. Fail to control adequately for
    case mix differences         “One of the nicest evenings I’ve spent at
                                    the Wilson’s…and then you had to go
                                    and do that on the rug!”
Can we screw up the performance measurement
process?

  We also can:
       5. Specify wrong comparison group by defining
           physician specialty incorrectly
       4. Fail to adequately risk adjust to control for
           effects of severity and comorbidities
       3. Use an incomplete source of claims data
       2. Use episode samples that are too small!!
       1. Use the wrong cost efficiency metric.
8. How do cost outlier episodes
affect cost efficiency scores?
   Physician has 25                              Effect of A Single Cost Outlier on Physician's
                                                                   Average Cost
    episodes that
    average $185                           $250



   26th episode costs                     $225

    $953, and increases

                            Average Cost
    average to $215.                       $200



   If 26th episode is an                  $175

    outlier and cost is
    truncated to $390,                     $150
                                                   Average Cost of 25 Average Cost of 26 Average Cost of 26
                                                       Episodes           Episodes       Episodes -- Outlier
    average is reduced                                                                       Truncated

    to $193
8. How can we deal with cost outlier
episodes?

   Truncating (Winsorizing) costs to some
    specified level
     • Advantage: retains episodes for profiles
     • Disadvantage: a couple of additional
       calculations are required
   Eliminating (trimming) them from sample
     • Advantage: it’s easy
     • Disadvantage: reduces number of episodes
       available for profiles
8. Comparison of Two Types of Outlier
Methods

             Percent Error in Predicting Most Cost Efficient Third of
                                   Physicians

           30%
 Percent




           20%

           10%

           0%
                 Limit 2%   Limit 5%   Limit 10%    Limit 2%    Limit 5%   Limit 10%

                      Outliers Truncated                    OuliersTrimmed

                                   Cardiology      Family Practice
    7. How do you attribute responsibility for
    episodes to physicians?


   Examples of attribution rules in current use:
    • Attribute to physician with maximum number of
        face to face encounters in episode
    •   Attribute to physician who has maximum
        percentage of professional fees in episode
    •   Attribute to physicians who represent at least 20%
        of episode professional and prescribing costs
7. How do you attribute responsibility for
episodes to physicians?

   Alternative rules for episode attribution:
    • Attribution to a single physician
       • Maximum number of face to face encounters
       • Maximum professional fees
       • Maximum professional and prescribing fees
    • Attribution to one or more physicians
       • Encounters, professional fees, or professional and
        prescribing fees exceeding specified percentage of
        episode total – e.g., 20%, 30%, 50%
7. Performance of 3 Different
Episode Attribution Rules
                Percent Error in Predicting Most Cost Efficient
                             Third of Physicians

          30%
          25%
          20%
Percent




          15%
          10%
          5%
          0%
                    At Leat 20%         At Least 30%          At Least 50%

                                       Attribution Rule
                                  Cardiology    Family Practice
6. Controlling for case mix
differences among physicians

   Indirect standardization
    •   Every episode has both an actual and an expected
        costs
    •   Physician cost efficiency is calculated as a function of
        total actual and total expected costs
   Example:
    •   A physician’s average actual episode cost is $673 and
        average expected episode cost is $705
    •   Ratio of observed (actual) to expected cost (O/E
        Ratio) is 0.95.
6. Controlling for case mix
differences among physicians

   Direct Standardization (“market basket’)
    • An episode-type frequency distribution is
        developed for physician’s specialty
    •   Physician cost efficiency is calculated as a
        function of average cost for each episode type
        and specialty’s frequency distribution
6. Example of Direct Standardization
Calculations

                                            Specialty     Doctor's      Doctor Cost x
Episode Type
                                             Percent    Average Cost   Specialty Percent


Hyperlipidemia                                 6%          $525              $32

Ischemic Heart Disease                        17%          $1,293            $220

Valvular Disorder                              5%          $1,151            $58

Minor Conduction Disorder                     15%          $488              $73

Benign Hypertension                            4%          $963              $39

Cardiovascular Disease Signs and Symptoms     33%          $381              $126

Pulmonary Disease signs and Symptoms           9%          $521              $47

Non-Specific Diagnoses or Conditions          11%          $128              $14

Total                                        100%                            $607
6. Does one case mix method yield more
accurate results than the other?

   With large enough samples of episodes, the
    two methods should yield identical results
   With small samples, indirect standardization
    may be more accurate
   Research comparing the two methodologies in
    the context of economic profiling has not been
    done
5. How do you define specialty for
economic profiling?

1.   Based on provider file credentials
2.   Based on types of episodes treated by
     physicians in specialty to be defined
3.   Combination of 1 & 2
5. How does defining a physician’s
specialty affect cost efficiency scores?

                           Change in Cost Efficiency Score: Defining Specialty
                              Based on 98th Percentile of ETGs Managed

                          1.5
 Cost Efficiency Score




                         1.25




                           1
                                      A                 B                     C
                                                    Prov ide r

          Score - All of Provider's ETGs         Score --98th Percentile of Specialty ETGs
4. After claims are grouped into episodes, is
further risk adjustment necessary?


      Thomas (HSR April 2006) says no.
      But MedStat says yes, and now
       Symmetry is saying yes too.
      Is Thomas wrong?
       • Yes and no.
       • Person level risk adjusters (e.g., ERGs,
         DCGs, ACGs) don’t explain cost variation
         within episode types
4. After claims are grouped into episodes, is
further risk adjustment necessary?


      Is Thomas wrong?
       • But MedStat and Symmetry now have risk
         adjusters specific to individual episode types
      How significantly does risk adjustment
       affect physician’s cost efficiency scores?
       • We don’t know. Research hasn’t been done
         on this question.
3. Effects of Missing Pharmacy Claims on
Physicians’ Cost Efficiency Scores
                       Pharmacy Cost As A Percentage of Episode Total Cost: Selected MPCs


                                                                                Percentage of Total Episode Cost
                                                                   Total         Represented By Pharmacy Cost
                                                                  Number
MPC                   Major Practice Category Name
                                                                  ETGs in
                                                                              Minimum                   Maximum
                                                                   MPC                      Weighted
                                                                                ETG                       ETG
                                                                                            Average
                                                                              Percent                    Percent

 4    Psychiatry                                                     13          3.5          53.5         75.8

 2    Endocrinology                                                  26          1.0          47.8         72.2

 6    Neurology                                                      27          0.2          35.7         74.6

 9    Otolaryngology                                                 31          0.0          35.3         56.7

10    Pulmonology                                                    23          0.0          34.1         59.2

 8    Cardiology                                                     37          0.5          28.0         57.8


21    Late Effects, Environmental Trauma, and Poisonings             3           7.3          25.9         36.4

17    Dermatology                                                    40          1.4          25.3         65.2
3. Effects of Missing Pharmacy Claims on
Physicians’ Cost Efficiency Scores

                                 Consistency of Physicians' Cost Efficiency Scores
                                        With and Without Pharmacy Claims
 Average Weighted Kappa




                          1.00

                          0.75

                          0.50

                          0.25

                          0.00
                                    Cardiology   Family Practice General Surgery   Nuerology
2. Are episode sample sizes really a big
issue?

   YES. It turns out that episode sample
    sizes are the most important determinant
    of cost efficiency score validity.
   Why? It all relates to the Central Limit
    Theorem.
2. Why episode sample size is important?

   Suppose we have 3
    physicians: A, B, and
    C                               A       B                 C

   Physician A’s true
    cost efficiency score
    is 0.6; B’s is 1.0, and
    C’s is 1.4
   Can we correctly          0   0.5           1                 1.5   2
    classify these                      Physician O/E Ratio
    physicians using
    episode data?
2. How Many Episodes Should Be
Required for Profiling?
                                       Cardiologist Episodes: Mean Costs of Samples of
                                                         Different Sizes


                            300
  Frequency of Occurrence




                            250
                            200

                            150

                            100
                             50

                              0
                                  $0   $500     $1,000   $1,500   $2,000    $2,500   $3,000    $3,500    $4,000
                                                             Sample Mean Cost

                              Population of Episodes     Samples of 10     Samples of 30      Samples of 50
2. How Many Episodes Should Be
Required for Profiling?

        Distribution of Sample Means for Samples of 10
           and 20 Episodes: Physicians A, B, and C


                                              20 episode samples
                                              10 episode samples




 0.00          0.50          1.00          1.50           2.00
2. How Many Episodes Should Be
Required for Profiling?
       Distribution of Sample Means for Samples of 50
               Episodes: Physicians A, B, and C




0.00          0.50          1.00          1.50          2.00
2. So, Why Not Require Large Sample
Sizes for Profiles?

                       Percent of Physicians Satisfying Minimum Episode
                                         Requirements

                       30%
  Percent Satisfying




                       25%
    Requirement




                       20%
                       15%
                       10%
                       5%
                       0%
                             0   10       20    30    40     50    60     70    80        90   100
                                      Minimum Number of Episodes Required for Profiling
2. How Can Number of Episodes per
Physician Be Increased?

   Extend period over which episodes are
    accumulated
   Profile physician groups instead of
    individual physicians
   Construct multi-health-plan consolidated
    databases
1. What’s the best cost efficiency metric?


     For direct standardization, it’s case-mix
      adjusted cost
     For indirect standardization, the metric
      commonly used is ratio of observed
      (actual) to expected cost, or O/E Ratio
     But O/E Ratio does not control for
      sample size differences among
      physicians
1. What’s the best cost efficiency metric?

   An alternative to O/E Ratio is the
   Standardized Cost Difference (SCD),
   defined as:             ˆ
                        y -y
                       Zk     k   k

                               / Nk


   Where yk is physician’s average actual cost, ỹk is
   physician’s average expected cost, σ is standard
   deviation of expected costs for the specialty, and
   Nk is physician’s number of episodes.
1. Comparing Cost Efficient Physicians
Identified by O/E Ratio and by SCD

                                                                                      Mean
  Top 25%                     Number of               O/E Ratio
                Provider ID               O/E Ratio               SCD     SCD Rank   Number
 Identified                   Episodes                  Rank
                                                                                     Episodes


                 2941960        172         0.69         12       -2.58      58

                 8194783         21         0.72         16       -3.00      54

By O/E Ratio     0849331        119         0.78         23       -2.62      56
                                                                                       102
But Not SCD
                 2865352         90         0.85         32       -4.22      41

                 2383441         34         0.86         36       -3.46      46

                 2891077        175         0.87         39       -4.25      40

                 4480266        442          0.9         47       -6.36      21

                 2690254        412         0.92         55       -5.15      29

 By SCD But      2206036        190          0.9         48       -5.04      30
                                                                                       274
Not O/E Ratio
                 2931901        195         0.89         44       -4.86      34

                 2610921        169         0.89         45       -4.58      37

                 1986733        235         0.91         50       -4.26      39
Are there any important research
questions here?

   Episode groupers are a useful tool to
    measure physician efficiency but should
    be used with care
   Findings presented in this session have
    not been replicated by other
    researchers
   Every one of the 10 issues discussed
    represents an important research
    question
Are these issues important to
health plans?




         “And now that’s the last of that.”

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:40
posted:4/12/2008
language:English
pages:40