Simpson slides

Lost in Translation: How Do We Increase the Return on Investment from Research? Lisa Simpson, MB, BCh, MPH, FAAP 18th Annual National Maternal and Child Health Leadership Conference May 16, 2005 Outline Some fundamental assumptions The translation imperative What do we know about what promotes – or impedes - research translation? Unique issues in MCH practice Measuring Impact – the return on investment Strategies for MCH practice Some Fundamental Assumptions Research translation will result in more informed and more effective policies and practice Policymaking encompasses at least – Public/legislative – Administrative/systems – Clinical/practice Use is NOT in of itself impact This is not a new problem! The Case of Scurvy 1601: Lancaster shows that lemon juice supplement eliminates scurvy among sailors 1747: Lind shows that citrus juice supplement eliminates scurvy 1795: (194 years after discovery) British Navy implements citrus juice supplement Source: Mosteller, Science l981;221:881 The Case for Prenatal Corticosteroids 1969: Liggins original research 1974-1995: 17 trials conducted 1989: first systematic review 1990’s: continued underuse between 20-30% 1995: NIH consensus conference 1999: Leviton et al. trial shows increased use from 33% to 58% (passive dissemination) or 68% (active dissemination) Hanney et al, Social Science & Medicine, 2004 Original research Negative results 18% Dickersin, 1987 variable Balas, 2002 Kumar, 1992 Submission 46% Koren, 1989 Negative results 0.5 year Acceptance 0.6 year Kumar, 1992 Publication 35% Lack of numbers Balas, 1995 17:14 0.3 year Poyer, 1982 Expert opinion Bibliographic databases 50% Poynard, 1985 6. 0 - 13.0 years Antman, 1992 Reviews, guidelines, textbook 9.3 years Inconsistent indexing Implementation It takes 17 years to turn 14 per cent of original research to the benefit of patient care The Translation Imperative Growing chorus nationally and internationally Focus of World Health Report 2004 EBM movement is itself generating pressure and a call for EBPP: evidence based policy & practice Governmental accountability: GPRA Growing recognition of the scope and severity of the healthcare quality challenges Doubling the budget of the National Institutes of Health Investments in Children: NIH Budget Overall doubling between FY 1998 & 2003 Pediatric spending increased at an average annual rate of 12.8% Proportion of NIH budget devoted to pediatric portfolio from 12.3% to 11.3% Total pediatric portfolio = $3.2 billion (FY 2004) 2/3 of portfolio in just 5 of 22 IC’s: NICHD, NIMH, NIDDK, NHLBI, and NCI Gitterman, Health Affairs, 2004 Evolution of the Social Contract Public funding in exchange for high quality research What is the return on investment? What is the utility of research? Who is using the research and what difference does it make? Adapted from Rich, 2002 The Two Translational Blocks in the Clinical Research Continuum Translational Blocks •Lack of Willing Participants •Regulatory Burden •Fragmented Infrastructure •Incompatible Databases •Lack of Qualified Investigators 1 Translation From Basic Science To Human Studies •Career Disincentives •Practice Limitations •High Research Costs •Lack of Funding 2 Translation of New Knowledge Into Clinical Practice And Health Decision Making Basic Biomedical Research Clinical Science And Knowledge Improved Health Clinical Research Continuum IOM Clinical Research Roundtable, Sung et al, JAMA, 2003 The Two Translational Blocks in the Clinical Research Continuum Translational Blocks •Lack of Willing Participants •Regulatory Burden •Fragmented Infrastructure •Incompatible Databases •Lack of Qualified Investigators 1 Translation From Basic Science To Human Studies •Career Disincentives •Practice Limitations •High Research Costs •Lack of Funding 2 Translation of New Knowledge Into Clinical Practice And Health Decision Making Basic Biomedical Research Clinical Science And Knowledge Improved Health Clinical Research Continuum IOM Clinical Research Roundtable, Sung et al, JAMA, 2003 The Two Translational Blocks in the Clinical Research Continuum Translational Blocks •Lack of Willing Participants •Regulatory Burden •Fragmented Infrastructure •Incompatible Databases •Lack of Qualified Investigators 1 Translation From Basic Science To Human Studies •Career Disincentives •Practice Limitations •High Research Costs •Lack of Funding 2 Translation of New Knowledge Into Clinical Practice And Health Decision Making Basic Biomedical Research Clinical Science And Knowledge Improved Health Clinical Research Continuum IOM Clinical Research Roundtable, Sung et al, JAMA, 2003 Outline Some fundamental assumptions The translation imperative What do we know about what promotes – or impedes - research translation? Unique issues in MCH practice Measuring Impact – the return on investment Strategies for MCH practice Rogers’ Diffusion S-Curve Laggards Late majority Adoption Early adopters Early majority Innovators Time Hierarchy of Technology Adopters Category Innovators Characteristics Venturesome • Cosmopolite • Dispersed contacts • High tolerance for uncertainty • % of Adopters 2.5 Early adopters Well respected opinion leader • Well integrated in social system • Judicious & successful use • 13.5 Early majority Late majority Deliberate • Highly interconnected peer system • 34 34 Skeptical • Responsive to economic necessity or social norms • Low tolerance for uncertainty • Laggards Traditional, relatively isolated • Precarious economic situation, suspicious • 16 Wise Adaptation of Rogers’ S-Curve Pop.A Pop. B Laggards Late majority Adoption Early adopters Innovators Early majority Pop. C Time The Dynamics of Innovation Diffusion The Innovation/New knowledge (message) Communication Channels (medium & messenger) Social context Diffusion of Innovations The Innovation/New knowledge 1. Relative advantage • Ability to judge if benefits outweigh risks. • Interplay between interests of patient, clinician & system • Does not occur when controversy exists about findings – e.g. mammography. LESSONS: • Understand end user • Recognize impact of change • Consider business case • Use testimonials & success stories Diffusion of Innovations The Innovation/New knowledge 1. Relative advantage 2. Trialibility • Ability to test out new knowledge/innovation without total commitment & with minimal investment • Reduces uncertainty about risks & benefits LESSONS: • Use small tests of change • Try out in one program • Focus on knowledge & innovations that can be tested Diffusion of Innovations The Innovation/New knowledge 1. Relative advantage 2. Trialibility 3. Observability • Ability to watch others applying knowledge or using the innovation • Better evidence of improved decisionmaking, increased functionality, and better outcomes LESSONS: • Bring groups together to share experiences • Use charismatic opinion leaders to demonstrate success • Use “viral marketing” Diffusion of Innovations The Innovation/New knowledge 1. 2. 3. 4. Relative advantage Trialibility Observability Compatibility • Compatible with values, past experiences and needs of user •Addresses an issue which users agree is a problem LESSONS: • Look for current behaviors/practices that are similar to the one being introduced • Innovations that reduce hassles are more likely to be successful Diffusion of Innovations Communication Channels (medium) • Internet & medical literature and new online journals with discussion portals • Web based affinity groups (NICHQ collaboratives) • Direct to consumer information • Patient mediated clinician/system change • Personal contact • “Piggy-backing” LESSONS: • Methods to inform vs. persuading are different • The more complex the message, the more face-to-face is important • Match channel to audience & innovation • Use “connectors” or “information brokers” Diffusion of Innovations Social context • Powerful driver of behavior – “ that’s how I learned to do it” • Code of silence among physicians still hindering quality and safety movement • Important to choose the right network for diffusion – e.g. of practice guidelines • Many barriers at the financing/regulatory level LESSONS: • Leverage existing social & professional networks • Leadership, a commitment to excellence and an organizational culture of creativity essential • Identify the system barriers to adoption Factors that influence policy application of research Pertinence [relevance] Ideological acceptability Practicality Issue complexity Time urgency Power and interest group politics Method of transmission: messenger and message frame Adapted from Davis & Howden-Chapman, 1996 Policymakers Views on Research Use 24 studies with 2041 policymaker interviews Facilitators: – – – – – – Personal contact (54%) Timeliness & relevance (54%) Summary & recommendations included (46%) Good quality (25%) Confirmed current policy or endorsed self interest (25%) Community pressure (17%) Innvaer et al, J Health Serv Res Policy, 2002 Policymakers Views on Research Use Barriers: – Lack of personal contact (46%) – Lack of timeliness or relevance (38%) – Mutual mistrust and reciprocal naivete (33%) – Power & budget struggles (29%) – Poor quality (25%) – Political instability (21%) Innvaer et al, J Health Serv Res Policy, 2002 Policymakers on Researchers They take forever to answer even the simplest question(s). Then they’re late with the results. What is this stuff they write? Who can understand it? They work in their “ivory towers” and produce impractical or irrelevant ideas and recommendations. They’re always hedging. I can’t get a straight answer. They don’t answer the question I thought I asked They take little responsibility for the implications of their findings. Researchers on Policymakers They don’t ask researchable questions. They don’t accept uncertainty. They don’t accept that reducing uncertainty costs more money. They don’t appreciate the influence of “publish or perish” on my life. They want unrealistic turn-around for results. They expect me to drop everything and deliver results for policy input. They want “bottom line” answers to take them off the policy hook. They can’t be trusted with my results - may misinterpret or misuse my results. From ‘Push’ to ‘Push, Pull and Partner’ Implications: attention to dissemination + receptor capacity ongoing interaction between the two processes ‘linkage and exchange’ knowledge brokers/boundary spanners (Lomas, CHSRF) Information Brokers Government agencies, e.g. AHRQ, MCHB Foundations, e.g. Commonwealth Fund Research and policy centers – University, not-for-profit & for-profit EIS Constituent organizations, e.g. AMCHP, ASTHO Change Agents “an individual who influences clients’ innovation decisions in a direction deemed desirable by the change agency” – Rogers, 2003 Academic detailing: – Almost 60,000 pharmaceutical sales representatives – one for every 14 doctors!! Strategies to Promote Clinical Translation Multiple reviews in last ten years Prevailing wisdom – Multifaceted interventions more effective – Passive dissemination (e.g. CME ineffective) Most recent review >200 studies Mean absolute improvement of only 10% Wide variation in effect size (-1% to 34%) Multifaceted interventions not significantly more effective than single faceted ones – Passive dissemination produced modest, but consistently positive improvements Shojania & Grimshaw, Health Affairs, 2005 – – – – Outline Some fundamental assumptions The translation imperative What do we know about what promotes – or impedes - research translation? Unique issues in MCH practice Measuring Impact – the return on investment Strategies for MCH practice Stakeholders in Children’s Healthcare Quality Clinical Policy Decisionmakers children, teens, families, practitioners Health Care Systems Decisionmakers health plans, hospitals, CHC’s, LHDs, MCH Programs, schools Public Policy Decisionmakers State, Federal, Voluntary Types of Decisions Public policy Do we reimburse for ADHD related assessments? Systems policy How do we promote better ADHD identification and management? Clinical policy (“EBM”) Which children should be managed with which strategies? Unique Barriers in MCH Practice Dominance of private sector, market based approach: knowledge as competitive advantage Skepticism as to the role of government Prevailing view of children as responsibility of parents Fragmented systems of care for children Heavy reliance on state action for children’s health & wellbeing multiple & diffuse receptor sites Unique Barriers in MCH Practice Multiple levels of policy decisionmaking – 50 Medicaid program – 50 Title V programs – 35 Stand alone SCHIP programs Multiple sites of care – – – – Clinical settings Health departments Schools Early intervention programs Unique Barriers in MCH Practice Under resourced providers – Child health providers: lowest paid, undercapitalized Under investment in translation infrastructure – Improvement collaboratives – Information technology applications Return on investment (ROI) more difficult Exploring the ROI for Quality in Children’s Health Care A business case for health care improvement exists if… – the investing entity realizes a financial return on investment in a reasonable time frame & using a reasonable discount rate – “bankable dollars”, reduction in losses, or avoided costs are realized – the investing entity believes an important indirect effect on organizational functioning and sustainability would accrue in a reasonable time frame (e.g. increased market share, staff retention) Leatherman et al, Health Affairs, 2003 Barriers to the Business Case for Quality in Health Care Inability of consumers to perceive quality differences – quality information even less available in child health Displacement of payoffs in time & sector – insurance churning; benefit reaped in schools Disconnect between consumers & payers Failure to pay for quality while paying for defects –Task-based fee schedules with no incentive for quality Uneven access to information among clinicians –Evidence base for children’s health less robust Modified from Leatherman et al, Health Affairs, 2003 Outline Some fundamental assumptions The translation imperative What do we know about what promotes – or impedes - research translation? Unique issues in MCH practice Measuring Impact – the return on investment Strategies for MCH practice Government likes to begin things – to declare grand new programs and causes and national objectives. But good beginnings are not the measure of success. Government should be results-oriented – guided not by process but guided by performance. There comes a time when every program must be judged either a success or a failure. George W. Bush, January 2000 Program Results Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)? Does the program (including program partners) achieve its annual performance goals? Does the performance of this program compare favorably to other programs with similar purpose and goals? Do comprehensive, independent evaluations of this program indicate that the program is effective and achieving results? Hierarchy of Research Impact Level 4 Level 3 Improves access, outcomes, quality Improves delivery and practice Level 2 Level 1 Improves process and policies Improves other research Stryer, Tunis & Clancy, 1999 Levels of Impact: the Case of ADHD Level 1: Impact on knowledge base future research Level 2: Impact on policies, change agents Level 3: Impact on clinical practice Level 4: Impact on patient outcomes Evidence Report on the Management of ADHD Findings led to new AAP guidelines AAP/NICHQ developed QI program in use by 13 AAP chapters TBD NICHQ: Assessing Impact 73% of HPHC patients with persistent asthma now have up to date asthma management plans 90% of sampled patients with asthma have severity classified. 85% of sampled patients with persistent asthma have appropriate medications prescribed. 95% of these patients have asthma management plans. PPOC, Boston Outline Some fundamental assumptions The translation imperative What do we know about what promotes – or impedes - research translation? Unique issues in MCH practice Measuring Impact – the return on investment Strategies for MCH practice Tools & Strategies Syntheses – Toolkits – Change packages Training Collaborative learning – Distance learning System wide partnerships Building Blocks for a Translation System in MCH Consensus Methods – Use inspired strategic research – Translation strategies – Knowledge mapping (synthesis) Information Infrastructure – Information tools & libraries CAHMI/Family Voices Data Resource Center – http://www.cshcndata.org/DesktopDefault.aspx – http://www.nschdata.org/ AHRQ Child Health Toolbox – http://www.ahrq.gov/chtoolbx/ ACT For Health (Healthy People 2010) – http://www.actforhealth.org/ Modified from Gruman, CFAH, www.cfah.org Building Blocks for a Translation System in MCH – Ongoing communication between researchers Annual CHSR meeting PAS meeting – Accurate & current information for the public CMWF chartbooks CYSHCN chartbook & website Modified from Gruman, CFAH, www.cfah.org Building Blocks for a Translation System in MCH Drivers of a translation system – Demand for improvement – Incentives for translation System guidance & accountability – Coordination across agencies – Broadly recognized leadership – Accountability & training Child Health Improvement Corps Modified from Gruman, CFAH, www.cfah.org