Demystifying the Federal Grant Review Process

Demystifying Federal Grant Review Process 1 The Panel Francis Chesley, MD Enola Proctor, PhD Ming Tai-Seale, PhD Kenneth Wells, MD 2 The Agenda A Typical Case of Grant-Making (M) Review criteria and reviewer expectations (E) Finding the Idea, Writing the Proposal, Having Patience (K) Views from AHRQ & E-Submission (F) Panel Questions and Answers 3 A Somewhat Typical Experience Ming 4 5 Summary Statement “Pink Sheet” 6 Priority Score How is the summary priority score calculated? Some are un-scored What is the fundable range? 7 Adjectives Used in Review 1.0-1.5 1.6-2.0 2.1-2.5 2.6-3.5 3.6-5.0 Outstanding Excellent Very Good Good Acceptable 8 Study Section Members Disciplines Anthropology Biostatistics Economics Epidemiology Health services research Medicine Nursing Organizational Theory Sociology Social work Methodological Orientations Quantitative Qualitative Mixed 9 Reading the Pink Sheet Expect the language to be Frank, and Not overly enthusiastic Talk to an experienced grant-maker Resubmit unless you see “fatally flawed” Do NOT resubmit right away 10 Take a Vacation … 11 12 Federal grants: review criteria and reviewer expectations Enola Proctor Center for Mental Health Services Research Washington University in St. Louis 13 Reviewer charge Assess likelihood that proposed research will have substantial impact on pursuit of NIH research goals: Advance understanding of Biological systems Improve control of disease Enhance health 14 5 Review Criteria Significance Approach Innovation Investigator Environment 15 Significance Is the research important? Is study compelling? Is the dependent variable important? To NIH objectives? To nation’s public health? To priority populations? Does the study extend boundaries of current knowledge? Theory, research Does the study address priorities of the funding agency? Implications: Can the findings inform care? 16 Application is responsive to priority issues Agency reports Bridging Science and Services Surgeon General’s report IOM: quality Agency partnership: “The Road Ahead…” Agency web site Program announcements Reports (Director’s report; Council minutes) 17 Approach: Aims Is the research question fully specified? Aims are manageable, but reflects long term agenda Incorporates relevant concepts, variables Does the approach ensure a good knowledge pay-off? Extends boundaries of current knowledge Informs theory (conceptual framework) 18 Approach: Methods Setting and context Collaborative arrangements feasible data collection plan (sufficient subjects, reasonable procedures, good measures) Manageable: Sufficient time for methodology and products? Person power: enough, and the right people 19 Approach: Methods, cont’d Analytic model = fully specified rival explanations controlled through design or measurement of relevant variables Procedures protect data quality Appropriate methods of analysis Sufficient power 20 Innovation Original and innovative aims Novel concepts, methods, approaches Challenges existing thinking or approaches Advances new methods or technologies 21 Investigator Sufficient experience to direct the project Well trained Productive Proposed project builds on background work investigator publications preliminary studies Proposed work will make “next logical” contribution to knowledge Is surrounded by a team who can ensure success Range of substantive and methodological expertise Multidisciplinary 22 Environment Environment contributes to project success Project capitalizes on environment, its people, and resources Collaborations Evidence of organizational support 23 Overall review All criteria are considered when assigning overall score Application does not have to be equally strong in all categories Major issue: will proposed research have major scientific impact? 24 Additional criteria, per NIH policy Adequacy of plans to include: Both genders Members of minority groups Children Subject recruitment and retention plans Budget Is it reasonable, over duration of project? Proposed protections Humans, animals, environment 25 Don’t make reviewers’ job harder than it is! Well written application Easy on the eye (white space) Appropriate to funding mechanism Fully explicated details don’t try to “obscure” weaknesses Consistent use of terms Content clearly organized and labeled 26 Finding the Idea, Writing the Proposal, Having Patience Ken Wells 27 Overall Strategies Think First, Second, and Third Get Feedback at all Stages (3 times) Develop and Follow Timeline for Submit Plan on 3 Major Rewrites (It’s Never Over) Talk to Agency Staff Early, 3 Times After a Good Idea and Opportunity, It’s Methods, Methods, Methods 28 Conceptual Framework Find a relevant basic or social science theory to organize: Hypotheses Variable selection Analysis plan 29 Conceptual Framework—Cont. Framework—Cont. 30 Finding Ideas Make a list of 5 project ideas that you would want to work on for years Prioritize and sketch out 1-3 ideas Get feedback from senior researchers, clinicians, or patients and community members 31 Shaping the Idea Based on literature, agency priorities, identify next steps suitable to your stage of development and institution capabilities Identify data sources needed Identify special opportunities SENIOR CONSULT 32 Develop the Design Clarify settings and subjects Develop Partnership – What model of collaboration? Consider diversity goals CONSULT with a statistician – Hypotheses – Sampling – Design 33 Developing the Design (cont.) Develop rough ballpark for budget SENIOR HELP NEEDED Discuss concept with funding agency, based on aim, opportunity, design, and likely budget 34 First Draft Develop Conceptual Framework Identify Main Variables Draft Interventions Develop Pilot Data GET SENIOR CONSULT AND BEG FOR $ Write Aims, Background, Design (Pretend you’re almost done!) 35 First Draft Statistician Consult to help outline main analysis and develop Power Calculations to determine: Is the Study Affordable? If Yes, PROCEED TO PARK PLACE If Not, Go Back to Start (JAIL) 36 Second Draft Develop Operations plans team organization, data collection, timeline Detailed Budget--GET HELP Statistical Consult: Detailed Analysis Plan Review assumptions Don’t delegate blindly They are YOUR hypotheses; Modify design, scope, budget as needed Human Subjects Section: Consult with IRB, Mentors Plop revised draft together 37 Third Draft Meet all agency requirements Develop budget justification Highlight “value added,” pilot data, fit of aims, method, and analysis to model AGENCY & SENIOR REVIEW 38 It’s Not Over Yet Take the feedback seriously Revise the whole proposal if needed Common problems: Not feasible (budget too large or scope too broad ); No pilot data; Aims not specific; Background has literature but no synthesis, framework, value-added unclear; Methods are under-developed (alternatives not considered; analysis not tied to hypotheses; design flaws: wrong sample for aim, causal inference poor; limitations 39 Expect a second submission: Responding to Pink Sheets Rage is OK for colleagues, family, friends Reviewers are both looking for problems and trying to help--Don’t take it personally Get advice from agency staff ASAP Think before revising GET SENIOR HELP Take your significant other to dinner 40 Demystifying the Federal Grant Review Process Francis D. Chesley, Jr., M.D. Director, Office of Extramural Research, Education and Priority Populations June 26, 2006 41 Federal Perspective Research Priorities and Program Priorities Application Process Importance of Concept Papers Common Pitfalls Peer Review Process Electronic Grant Application! 42 Applicant Responsibilities Know PHS Form 398 and 424 R&R Know the Funding Agency and Staff Know Agency Research Priorities Know the Grant Mechanisms Know the Grant Process and Key Changes Understand Agency Research Budget 43 AHRQ Grant Application Submission, Review, and Award Process Development, Receipt, and Assignment of Applications CSR sends computerized notice of assignment to applicant CSR computes percentile ratings SRAs prepare Summary Statements Study Section Review for Scientific Merit Applicant Investigator develops and submits grant application to NIH/CSR CSR assigns application to NIH Institute or AHRQ CSR Assignment Office AHRQ assigns application to Initial Review Group DSR Referral Officer assigns to appropriate AHRQ Program IRG members review and evaluate IRG reviews and assigns priority scores or designates noncompetitive CSR sends computerized notice of review results to applicant DSR informs program that summary statement is available PO mails summary statements to investigators NIH Councils (Duals) Site visit made if necessary Site visit report AHRQ Review for Program Relevance and Funding Determinations Award Negotiation and Issuance PO reviews and prepares/sends recommendation memo to DSR/GMS DSR schedules funding meeting Applications selected for funding Signed “paylists” received by GMO Final review and negotiations Congressional liaison notified Award issued Public Affairs notified Award received by institution Investigator begins work (30 days) 44 AHRQ E-grants Transition! Electronic Grant Application Receipt Use of the SF424 (R&R) Grant Application AHRQ will require electronic submission of all competing grant applications via Grants.gov using the SF 424 Research and Related (R&R) application. ________________________________________ Paper No More, Use 424 (R&R) Electronic submission is here to stay! 45 OLD NEW 46 AHRQ’s Transition Strategy AHRQ will transition by individual grant mechanism ALL applications will require electronic submission through Grants.gov on the SF424 (R&R) form Mechanisms not yet transitioned will continue to require submission on PHS 398 on paper AHRQ will announce plans to transition mechanisms in the NIH Guide for Grants and Contracts and on Grants.gov 47 AHRQ Transitions to Electronic Grant Application Submission AHRQ transitioning to electronic grant submission through Grants.gov Grants.gov - Web portal that serves as the single access point for all Federal grant programs. Grants.gov provides the interface for 26 agencies to announce $350 billion in annual grant awards and for all grant applicants to find and submit applications to those funding announcements. www.grants.gov Simultaneously transitioning from the PHS 398 to the SF 424 (R&R) application 48 AHRQ’s Transition Plan Type of Grant Small Conference Grants Dissertations Large Conference Grants Small Research Grant Program Research Project Grant Program Research Career Development Awards Individual National Research Service Awards Institutional National Research Service Awards Mechanism R13 R36 R13 R03 R01 / R18 K02 / K08 F31 / F32 T32 Submission Date December 20, 2005 April 10, 2006 June 1, 2006 July 24, 2006 February 1, 2007 July, 2007 August, 2007 September, 2007 49 Getting Started One time registrations for Grants.gov (http://grants.gov) and eRA Commons (era.nih.gov/commons) systems must be completed before application submission. For up to date general information on electronic submission, the SF 424 (R&R), and Grants.gov, visit the AHRQ Electronic Submission of Grant Applications Web Site: http://www.ahrq.gov/path/egrants.htm 50 “Opportunity is missed by most because it is dressed in overalls and looks like work.” Thomas Edison 51 Contact Information AHRQ WEBSITE www.ahrq.gov Francis D. Chesley, Jr., M.D. (301) 427-1521 Francis.Chesley@ahrq.hhs.gov 52 Panel Discussion How to cope with working environment variations Finding the right mentor Building your research team Recruiting “cold reviewers” How long does it take Idea → submission → grant received The fairytale The horror story 53 The Fairytale 54 The Horror Story 55 Questions & Answers Panel 56 Resources Instructions on how to prepare your application http://grants.nih.gov/grants/funding/phs398/section_1.html Contact information Ming: mtaiseale@srph.tamhsc.edu Enola: ekp@wustl.edu Ken: kwells@ucla.edu Francis: Francis.Chesley@ahrq.hhs.gov 57 58