Measuring Technical Progress in Real Time

Document Sample
Measuring Technical Progress in Real Time Powered By Docstoc
					Measuring Technical Progress and
  Potential Impact in Real Time
                 Presented at the
  American Evaluation Association Conference 2009
                        by
           Jerald Hage, Wilbur Hadden
              Center for Innovation
             University of Maryland
                Gretchen Jordan
          Sandia National Laboratories
 Measuring Technical Progress and Potential
            Impact in Real Time

Why be concerned about this issue?

 Papers and patents occur to late for effective
  interventions for research management
 Metrics provide opportunities for expanding the
  number of social and economic benefits
 Without better and more timely measures, there will
  be a considerable backlash against the research
  stimulus package
 Need to shift focus from productivity to innovation
 Measuring Technical Progress and Potential
            Impact in Real Time

Current studies :
 NSF study of 6 morbidities at two Institutes
  of NIH (200 projects) (focus of this talk)
 NSF study of 5 research areas in 6 national
  laboratories N = 72 projects
Past studies:
 Scientific and technological progress at
  Sandia National Laboratories (22 projects)
 New products at STAR at NOAA (5 projects)
    Review of current cost-benefit studies
              of health research



Too much emphasis on mortality rates

Too much emphasis on economic returns

Too much emphasis on long time periods

Little consideration of the deconstruction of
 the streams of research
              Correcting the Problems

 View health care as a process:
   - prevention
   - diagnosis
   - treatment
   - post-treatment
 Develop various social measures (e.g. reduction in
  home care, improved quality of life, etc.)
 Focus on specific research projects as completed
 Isolate core ideas in streams of research
A Framework for Understanding the Impacts of
              Health Research

   Global Research                                                     Health
                                                                                       Healthcare          (Population)
                                                                      Industry
                                                                                     Appropriateness,         Health

                                            Interactions/Feedback
                           Knowledge Pool                              Other           Access, etc.           Status
  Health Research                                                    Industries                                and
                                                                                       Prevention,
  •Biomedical                                                                                                Function
                                                                                   Diagnosis, Treatment,
  •Clinical                                                         Government      and Post Treatment
  •Health services
  •Population and                                                    Research                               Well Being
  Public health                                                       Agenda                                   And
                                                                                     Determinants of
  •Cross cutting                                                                                            Economic
                                                                                      health (social,
  research                                                             Public                               Prosperity
                                                                                   environmental, etc.)
                                                                    Information,
                                                                       Groups
      Research
      Capacity                                                                             Our focus for measures
                                                                                             of potential impact
Modified from CAHS Report on ROI for Health research
available at http://www.cahs-acss.ca/e/publications/

                                                                                                                          6
    J. Hage October 2009
         Measuring Health Outcomes as
              (percent change in)

Prevention
• effectiveness of prevention intervention (decline in
  incidence of morbidity)
• severity of incidence of morbidity

Intake and Assessments
• the speed of diagnosis (reduction in the number of
   tests and their duration)
• the accuracy of diagnosis (reduction in false
   positives or negatives)
• accuracy of prognosis (duration and quality of life,
   etc.)
      Measuring Social/Economic Benefits

Prevention
• Value of illness days saved from decline in morbidity
  incidence
• Value of reduction in cost of treatments for less
  severe morbidity incidence

Intake and Assessments
• Reduction in the costs of tests for diagnosis
• Reduction in costs of false positives or negatives
• Reduction in the costs of futile interventions
 Measuring Social/Economic Benefits: Example One


In a clinical trial of a new staining technique for earlier
   diagnosis of metastasis, the treatment group had
   their sentinel lymph node removed.
The treatment group had a much better survival rate
   after four years and with fewer new tumors
Using the treatment groups experience, one takes the
   number of new patients in the next year and
   computes the percentage with metastasis that would
   be found.
Measuring Social/Economic Benefits: Example One

Example: diagnostic technique in melanoma

Formula for social value:
new patients X % metastasis X % mortality = Life years*
53,120                15.8%    22.5%         1,888
* only in fourth year

Formula for economic value:
Life years     X   Value of one year     = Economic value
1888                 $45,000                  $85 million

Does not include the value of reduction in the number of tumors
 Measuring Social/Economic Benefits: Example One

Assumptions:
• The percentage of sentinel nodes where metastasis
  has occurred in the study is the same as in the
  general population.
• No confounding differences between the populations
  of Australia (the majority of the patients), the U.S. or
  Europe.
Reference: Morton et al. September 28, 2006
  “Sentinel-Note Biopsy or Nodal Observation in
  Melanoma” New England Journal of Medicine, 355,
  13: 1307-1317
Measuring Social/Economic Benefits: Example
                    Two

In a clinical trial of new patients with Alheizmer’s
   Disease those with depression were given anti-
   depressant drug
The treatment group had a slower on set of Alheizmer’s
   disease by on average about three months
Using the treatment groups’ experience, one takes the
   number of new patients in the next year and
   computes the savings in home care
 Measuring Social/Economic Benefits: Example
                     Two

Metric for Prevention: Alheizmer’s Disease
Formula for social value:
new patients X % depression X Reduction in hours

454,000                  28%               520 per case
Or 2,754,237 days of home care
Formula for economic value:
Value per        Value of
Hour            Total hours
$10               $66 million
Does not include the value of reduced stress
Measuring Social/Economic Benefits: Example
                    Two
Assumptions:
• The percentage of depression among new patients is the same as
   existed in the study, which to us seems to be an underestimation.
• For new patients only home care is used, which again is a
   conservative estimate.
• Only 40 hours per week of home care for each of the three months or
   13 weeks because the new patients do not need constant watching.
• An average wage of only $10.
• The assumption of an improvement of three months is based on the
   conservative estimate of the findings from five drugs that have been
   approved by the FDA for Alzheimer’s which reduced symptoms for six
   to 12 months for about 50 percent of the population, i.e. 50 % of six
   months or three months.
Reference: Lu et al., June 16, 2009 “Donepezil delays progression of AD
   in MCI subjects with depressive symptoms” Neurology, 72, 24, 2115-
   21
  Measuring Technical Progress in Real Time

Next Steps:
1 Apply all health outcome and social/economic
  benefit metrics in six morbidity areas
2 Determine what proportion of the funding was
  completed by NIH
3 Determine the stream of development of ideas that
  led to the outcome
    Measuring Technical Progress in Real Time

Please contact us if you have questions,
  suggestions, or opportunities to collaborate.
• Jerald Hage: Email HAGE@socy.umd.edu,
  phone 301-405-6437.
• Wilbur Hadden: Email
  whadden@socy.umd.edu, phone 310-405-
  9746
• Gretchen Jordan: Email
  gbjorda@sandia.gov, phone 505-844-9075

				
DOCUMENT INFO
Categories:
Tags:
Stats:
views:0
posted:6/19/2012
language:
pages:16