Peer Review of NIH Research Grant Applications - PowerPoint

Document Sample
Peer Review of NIH Research Grant Applications - PowerPoint Powered By Docstoc
					Peer Review of NIH
  Research Grant
   Applications
      Center for Scientific Review

      National Institute of
      Diabetes and Digestive and
      Kidney Diseases

      National Institutes of Health
    U. S. Department of Health and Human Services
                                               The
                                             Secretary

                                        Deputy Secretary




                                                                              Health Resources
 Administration for     Administration on                Food and Drug
                                                                                and Services
Children and Families       Aging                        Administration
                                                                               Administration
       (ACF)                (AoA)                            (FDA)
                                                                                  (HRSA)




                           Agency for
Health Care Financing                                      Indian Health      National Institutes
                        Health Care Policy
   Administration                                             Services            of Health
                          and Research
       (HCFA)                                                  (IHS)                (NIH)
                            (AHCPR)




     Centers for        Agency for Toxic             Substance Abuse and
                                                                              Program Support
   Disease Control      Substances and               Mental Health Services
                                                                                   Center
   and Prevention       Disease Registry                Administration
                                                                                   (PSC)
       (CDC)               (ATSDR)                        (SAMHSA)
National Institutes of Health

                    Much of the
                    biomedical research
                    in the united states
                    is supported by the
                    federal government,
                    primarily the
                    National Institutes
                    of Health (NIH)
                        National Institutes of Health
                                                                   Office of the Director




                                                                                    National Institute                                 National Institute
                         National Institute          National Institute
 National Institute                                                                  of Arthritis and            National Cancer        of Child Health
                         on Alcohol Abuse             of Allergy and
     on Aging                                                                        Musculoskeletal                Institute             and Human
                          and Alcoholism            Infectious Diseases
                                                                                    and Skin Diseases                                    Development




National Institute on    National Institute          National Institute
                                                                                                                 National Institute
Deafness and Other        of Dental and               of Diabetes and               National Institute                                    National Eye
                                                                                                                 of Environmental
 Communication             Craniofacial                Digestive and                 on Drug Abuse                                         Institute
                                                                                                                  Health Sciences
     Disorders               Research                Kidney Diseases




                                                                                                                 National Institute
 National Institute      National Heart,             National Human
                                                                                    National Institute            of Neurological      National Institute
    of General           Lung, and Blood             Genome Research
                                                                                    of Mental Health               Disorders and      of Nursing Research
 Medical Sciences           Institute                    Institute
                                                                                                                      Stroke




                          National Center
National Institute of                                       Fogarty                     National Center                               National Center on
                        for Complementary                                                                        National Library
Biomedical Imaging                                       International                   for Research                                 Minority Health and
                          and Alternative                                                                         of Medicine
and Bioengineering                                          Center                        Resources                                    Health Disparities
                             Medicine




                                                                           Center for
                                                                                                         Center for
                                       Clinical Center                    Information
                                                                                                      Scientific Review
                                                                          Technology
           A Typical Institute/center

National             Office of the IC            Board of
Advisory                Director                 Scientific
Council                                         Counselors



   Extramural                               Intramural


     Scientific
     Programs


                                        Laboratory   Clinical
Grants       Contracts                   Studies     Studies
NIH Extramural Program Mission
                  Manage a portfolio of investments to
                    improve health through science




Identify scientific                                            Foster the best
  opportunities                                                   science


          Ensure proper                               Promote effective
           stewardship                                 communication
 Exemplify and promote the highest level          Promote effective ways to communicate
of scientific integrity, public accountability,   scientific information to scientists, health
     and responsibility in the conduct                   practitioners, and the public
                  of science
   NIH Extramural Program

Grant         patron
              (Assistance, encouragement)

Cooperative   partner
Agreement     (assistance but substantial
              Program involvement)

Contract      purchaser
              (Procurement)
Overall Peer Review Process
    Dual Review System for Grant Applications
First Level of Review
Scientific Review Group (SRG)
    Provides Initial Scientific Merit
     Review of Grant Applications
    Rates Applications and Makes
     Recommendations for Appropriate
     Level of Support and Duration of
     Award

                               Second Level of Review
                                Council
                                      Assesses Quality of SRG
                                        Review of Grant Applications
                                      Makes Recommendation to
                                       Institute Staff on Funding
                                      Evaluates Program Priorities
                                        and Relevance
                                      Advises on Policy
            Review Process for a Research Grant
                                     National Institutes of Health
 Research      School or Other
  Grant        Research Center        Center for Scientific Review
Application
                                        Assigns to IRG/Study Section & IC


Initiates                                      Study Section
Research       Submits Application
Idea                                     Evaluates for Scientific Merit

                                                  Institute
                                       Evaluates for Program Relevance

                                     Advisory Councils and Boards
Conducts        Allocates Funds
Research                                     Recommends Action

                                             Institute Director
                                       Takes final action for NIH Director
                               Overall Timeframe From
                                 Submission to Award

           There are three overlapping cycles per year :
JAN     FEB   MAR      APR   MAY   JUN      JUL AUG    SEP OCT      NOV      DEC   JAN FEB MAR   APR    MAY      JUN    JUL



Cycle 1
      Receipt                            Review        Council
          Referral                                                        Award

                     Cycle 2
                             Receipt                             Review              Council
                              Referral                                                           Award


                                                  Cycle 3
                                                        Receipt                      Review            Council
                                                        Referral                                                       Award
          Typical Timeline for a New
       Individual Research Project Grant
               Application (R01)
 There are three overlapping cycles per year:

–Submit in February (June, October)
  –Review in June (October, February)
    –Council in September (January, May)
       –Earliest award in December (April, July)

Cycle 1----
Cycle 2----
Cycle 3----
       Types of Scientific Review Groups
       Where Are Applications Reviewed?
GROUPS                        APPLICATIONS REVIEWED
                                Research Projects
CSR IRGs                        Academic Research
 Study Sections                  Enhancement Awards
                                Postdoctoral Fellowships
 Special Emphasis Panels
                                Small Business Innovation
                                  Research
                                Shared Instrumentation
INSTITUTES                     Program Projects
 Scientific Review Groups      Centers
                               Institutional Training Grants
                               Conference Grants
                               Career Awards
                               Small Grants
                               RFAs (R01, R21, R03, U01)
 Contract Review Committees    Contracts
      Peer Review of NIH Support Mechanisms
          CSR                                 Institutes
Research project grant (R01)       Program Project grant (P01)
Postdoctoral fellowship (F32)      Center grant (P30, P50, P60)
Senior fellowship (F32)           Institutional fellowship (T32)
Fogarty international center       Academic career award (K07)
   Fellowship (F05, F06)           Mentored clinical scientist
Short-term training (T35)          development award (K08)
Small business grants (R41, R42    Conference grant (R13)*
   R43, R44)                       MARC fellowships (F34, F36, T34)
Academic Research Enhancement      Minority Biomedical Support
   Award (R15)                       Grant (S06)
Biomedical Research Support        Resource grant (P40, P41, R24,
    Shared Instrumentation           R26, R28)
     Grant (S10)                   Contract
                                        *Reviewed by CSR upon request
Center for Scientific Review (CSR)

    Focal point for initial review at NIH
                    •Central receipt point for PHS
                    applications

                    •Referral to Institutes and to IRGs
                    and Study Sections

                    •Review of most research and
                    research training applications ..for
                    scientific merit
  Grant Application
Receipt and Assignment
       Applications Submitted to NIH

• Approximately 46,000
  grant applications are
  submitted to NIH each
  year, of which 25-30%
  are funded
• Competing grant
  applications are received
  for three review cycles
  per year
                    CSR Receipt and Referral:
                  Central Receipt Point for Applications
                  Submitted to the Public Health Service


       National Institutes               Substance Abuse and
               of                        Mental Health Services
            Health                          Administration

                               CSR
                             Receipt &         Agency for Health Care
  Centers for                 Referral           Policy & Research
Disease Control
                                       Office of
                    Food              Assistant
                     &               Secretary for
                    Drug                Health
             Competing Applications
                Reviewed By:
     Fiscal Year                      PHS            NIH           CSR

          1976                       27,213        19,509        16,268

          1985                       31,807        28,407        23,626

          1992                       37,826        31,145        24,708
          1993                       39,968        38,447        26,178
          1994                       43,394        41,739        28,033
          1995                       44,342        41,899        29,203
          1996                       38,579        38,012        27,253
          1997                       38,767        38,065        26,516
          1998                       39,474        37,855        26,074
          1999                       44,395        42,383        31,837
          2000                       45,705        43,814        32,091


NOTE: Starting in FY93, NIDA, NIAAA, and NIMH are included in NIH totals
               Receipt Dates
Depend on the Type of Application

 •   Jan, May, Sept 10: institutional training grant applications
 •   Jan, May, Sept 25: academic research enhancement award
 •   Feb, June, Oct 1: new research applications
 •   Mar, Jul, Nov 1: revised, competing continuations, and
                        Supplemental Applications
          and Small Business Innovation Research Applications
            Flow of a Competing Grant Application
                Through the CSR Referral Section
                                                                          Project Control Unit:
                                       Mailroom                            Receipt & Record Group
   Application                     (Receive, Date Stamp,                   (Process, Enter Skeletal
(Original & 5 Copies)                                                        Data into Computer,
                           Preliminary Sort, Add Bar Code)
                                                                                Print Status)



(Advance Copy)                                        Assignment Unit:
Scientific Review              (Assign to Scientific Review Group and Awarding Organization)
     Group



                                   Project Control Unit:
   (2 Copies)                    Review & Control Group
Scientific Review            (Add Labels, Send Data to IMPAC,
     Group                                                                         (Original)
                                 Print Mailers, Distribute)                   Print Shop


                        (1 copy)       (1 copy)
                        OPERA           Project       (35 copies)    (20 copies)     (1 copy)    (1 copy)
                        EIS,DMC      Control Files       SRG           Original       Project    Research
                                                                      Awarding       Control    Documents
                                                                    Organization                 (IC only)
 Applications are Assigned by

Referral Officers:
 Professional scientists, most of
  whom also serve as scientific
  review administrators of CSR
  study sections
  Applications Are Assigned To:
• Scientific review groups based on:
  – Specific review guidelines for each
    scientific review group

• Institutes based on:
  – Overall mission of the Institute
  – Specific programmatic mandates and
    interests of the Institute
     Assignment to Institutes
Applications are referred to an institute or
center as the potential funding component:

–This assignment is based on a match between
 the research proposed and the overall mission
 of the institute or center

–Where applications are appropriate for more
 than one institute or center, multiple
 assignments are made
         Sample Application Number

         Individual             serial               amended
         Research              number
          Grant

     1    R01     DK      12345          01     A1


   New            national                  grant
Application       cancer                  support
                   Institute                  year
                 Assignment of Applications
                                      Review Group                          Institute

                                                                                       Form Approved Through 9-30-97
 AA                                                                                               OMB No. 0925 0001

Department of Health and Human Services                   Leave Blank--for PHS Use Only
         Public Health Service                            Type 1 Activity R21 Number A154321-01
          GRANT APPLICATION                               Review Group ZDK1      Formerly
Follow instructions carefully. Do not exceed character    Council/Board (Month,Year) Date Received
       length restrictions indicated on sample.
                                                                 09/01                            07-26-01
1. Title of Project (Do not exceed 56 characters, including spaces and punctuation.)
Planning Grant for Diabetes & Sc Ed in Tribal Schools
 2. Response to Specific Request for Applications or Program                      X No      x       Yes (if “Yes”
                                                                                                          state
 Announcement
 Number: DK 01-033 Title: Planning Grant for Diabetes and Science Education …                   number and title)

                                     3. Principal Investigator/Program Director

3a. Name (Last, first, middle)                                   3b. Degree(s)         3c. Social Security No.
Smith, Brian E.                                                        Ph.D.
        Assignment Notification Letter
Dear Dr. Smith:
 Your grant application entitled “PLANNING GRANT FOR
DIABETES AND SCIENCE EDUCATION IN TRIBAL
SCHOOLS ” has been received by the National Institutes of Health
and assigned to a scientific review group (SRG) for scientific merit
evaluation and to an institute/center for funding consideration.
Specific information about your assignment is given below. The
initial peer review should be completed by AUGUST, 2001, and a
funding decision made shortly after the appropriate National
Advisory Group meets in SEPTEMBER, 2001. Questions about
the assignment should be directed to the scientific review
administrator (SRA) or the Division of Receipt and Referral, Center
for Scientific Review (formerly Division of Research Grants) at
(301) 435-0715. Other questions prior to review should be directed
to the scientific review administrator and questions after the review
to the program staff in the institute/center.
Assignment Notification Letter (continued)
Assignment Number: 1 R01 DK12345 - 01
Dual Assignment: NS
Scientific Review Group:
ZDK1 GRB-5 S1 S
Information about this IRG may be found on the
NIDDK Home page (http://www.niddk.nih.gov)
Scientific Review Administrator:
DR. FRANCISCO CALVO, SRA
NATIONAL INSTITUTE OF DIABETES AND
DIGESTIVE AND KIDNEY DISEASES
 6707 DEMOCRACY BLVD, ROOM 752
BETHESDA MD 20892-5452
(301) 594-8885
   Assignment Notification Letter
             (Continued)

Institute/Center:
NATIONAL INSTITUTE OF DIABETES AND
DIGESTIVE AND KIDNEY DISEASES
DIV/EXTRAMURAL ACTIVITIES
6707 DEMOCRACY BLVD, ROOM 752
NATIONAL INSTITUTES OF HEALTH
BETHESDA, MD 20892
(301) 594-8885
Initial Review in IC
            Peer Review
• CSR and IC Study Sections are managed by a
  Scientific Review Administrator (SRA) who is a
  scientist, usually at the Ph.D. level, whose scientific
  background is close to the expertise of the study
  section
• Each CSR/IC standing study section has 12-24
  members who are primarily from academia
• Institute review is also different in that Special
  Emphasis Panels are constituted specifically to review
  applications that have been received in response to an
  RFA or other Institute specific grant mechanisms
• As many as 60-100 applications are reviewed at each
  study section meeting, again this depends on the
  complexity of the review
   Scientific Review Administrator
Designated Federal official with overall
  responsibility for the review process, including:
• Performing administrative and technical review of
  applications to ensure completeness and accuracy
• Selecting reviewers based on broad input
• Managing study section meetings
• Preparing summary statements
• Providing any requested information about study
  section recommendations to institutes and national
  advisory councils/boards
Selection of Peer Reviewers
                  Active and Productive
                       Researchers

         Research
         Capability

        Non-Research



        Non-Doctoral


   Scientific Community
               Criteria for Selection of Peer Reviewers



                              Active & Productive
                                 Researchers
                                                    Meet Internal Administrative
                                                          Considerations
                                                                         •Geography
Doctoral or     Research                                                   •Institutional
Equivalent      Capability                                    Interest         Affiliation
 Degree                                                                       •Non-Federal
                                                                 in
                                                              Serving            Employment
              Non-Research
                                                                               •Gender &
                                                     Expertise in Discipline of Ethnic
                                                        Review Group and          Status
               Non-Doctoral                           Specialization Needed




      Scientific Community                          Active and Productive
                                                         Researchers
Study Section Meeting
         Certification of No Conflict of Interest
This will certify that in the review of applications and proposals
  by (study section) on (date), I did not participate in the
  evaluation of any grant or fellowship applications from (1) any
  organization, institution or university system in which a
  financial interest exists to myself, spouse, parent,child, or
  collaborating investigators; (2) any organization in which I
  serve as officer, director, trustee, employee or collaborating
  investigator; or (3) any organization which I am negotiating or
  have any arrangements concerning prospective employment or
  other such associations.

         SIGNATURES
  __________________
  __________________
                                __________________
                                __________________
  __________________            __________________
  __________________            __________________
          Confidentiality
• Review materials and proceedings of review
  meetings represent privileged information to be
  used only by consultants and NIH staff.
• At the conclusion of each meeting, consultants
  will be asked to destroy or return all review-
  related material.
• Consultants should not discuss review
  proceedings with anyone except the SRA.
• Questions concerning review proceedings
  should be referred to the SRA.

                                                    K185pp.46
      Review of Research Grants
REVIEW CRITERIA:
 –    Significance
 –    Approach
 –    Innovation
 –    Investigator
 –    Environment
 –    Overall Evaluation & Score Reflects Impact on
     Field
   Review Criteria (Continued)
• Significance: Does the study address an important problem?
  How will scientific knowledge be advanced?

• Approach: Are design and methods well-developed and
  appropriate? Are problem areas addressed?

• Innovation: Are there novel concepts or approaches? Are
  the aims original and innovative?

• Investigator: Is the investigator appropriately trained?

• Environment: Does the scientific environment contribute to
  the probability of success? Are there unique features of the
  scientific environment?
         Specific Review Criteria
            RFA DK-01-033
• Administrative, leadership qualifications and
  experience of the Program Director.
• Adequacy and availability of any necessary
  institutional facilities and resources.
• Adequacy of the plan for developing a diabetes
  science education program, including use of existing
  resources, building partnerships with other institutions,
  use of consultants, and plans for critical review of the
  curriculum.
• Documentation of potential participants for the science
  education program.
Research Involving Human Subjects
    Research Involving Human Subjects
Areas exempt from human subject review
• Education research
   – Normal educational practices
• Educational tests, survey or interview procedures, or
  observation of public behavior
   – Subjects not identified
   – Subjects’ privacy rights protected
• Educational tests, survey or interview procedures, or
  observation of public behavior not exempt in previous
  category if:
   – Subjects are public officials or public office candidates
   – Federal statute requires confidentiality without exception
       Research Involving Human Subjects
                  (Continued)
Areas exempt from human subject review
• Collection or study of existing data, documents, records,
  pathological specimens
   – Information publicly available
   – Subjects not identified
• Research and demonstration projects regarding certain public
  benefit or service programs
• Taste and food quality evaluation and consumer acceptance
  studies using
   – Foods without additives
   – U.S. Government approved food ingredient
     Research Involving Children
Children must be considered for inclusion in
all human subject research supported by NIH
Scientific Review Group or Study
          Section Actions

• Scored, Scientific Merit
  Rating (priority scores and
  percentiles)
• Unscored (lower half)
• Deferral
                   Action
• Scored -- Scientific Merit Rating 1.0 to
  approximately 3.0
  Based on the relevant review criteria, the application is
   judged to be in the upper half of applications
   reviewed by the study section or scientific review
   group. The recommendation can be for the requested
   time and amount or for an adjusted time and amount.
   A priority score is provided, and a summary statement
   prepared that incorporates the written critiques plus a
   resume and summary of the discussion.
                 Action
• Unscored
 Application is unanimously judged to be
  in the lower half of applications
  reviewed by the study section or
  scientific review group. No priority
  score is assigned. The summary
  statement provided to the applicant is a
  compilation of reviewers’ comments
  prepared prior to the meeting.
               Action
• Deferral
  The study section cannot make a
   recommendation without additional
   information. This information may be
   obtained by a project site visit or by
   submission of additional material by
   the applicant.
 Post Scientific Review Group
            Actions

• Calculations of priority scores and
  percentile rankings
• Preparation of summary statements
• Removal of applications from national
  advisory council / board consideration
      Summary Statement
Once applications are reviewed, the results are
documented by the SRA in a summary statement
and forwarded to the Institute (and the PI) where a
funding decision is made:
The summary statement contains:
– Overall Resume and Summary of Review Discussion
– Essentially Unedited Critiques
– Priority Score and Percentile Ranking
– Budget Recommendations
– Administrative Notes
National Advisory Council or
       Board Review
      Council Actions

• Concurrence with study section
  action
• Modification of study section
  action
• Deferral for re-review
 What Determines Which
  Awards Are Made?

• Scientific merit
• Program considerations
• Availability of funds
 Mechanisms for Preliminary Studies
• Small grants (R03)
   – Feasibility/new technology/innovative high risk ideas
   – Short term, usually $25,000 - $50,000
   – Extremely variable in detail - example:
         • NCRR: 1 year, $35,000, non-renewable
         • NIDDK: 3-6 months, $12,500 - $25,000
         • NCI: 2 years, $50,000
   – Institute review
• Exploratory/developmental grants (R21)
   –   Feasibility (for those without preliminary data)
   –   $100,000/2 years (for this RFA it is $75,000/1 year)
   –   Increasing use
   –   CSR review (RFA review performed by NIDDK)
Preparation of an Application
PHS Research Grant
 Application Kit (form PHS 398)


             Mail Completed Forms To:
             CENTER FOR SCIENTIFIC REVIEW
             NATIONAL INSTITUTES OF HEALTH
             ROCKLEDGE II ROOM 1040 MSC-7710
             BETHESDA MD 20892-7710
  When Preparing an Application
• Read instructions
• Never assume that reviewers “will know what you
  mean”
• Refer to literature thoroughly
• State rationale of proposed investigation
• Include well-designed tables and figures
• Present an organized, lucid write-up
• Obtain pre-review from faculty at your institution
        Common Problems in Applications
•   Lack of new or original ideas
•   Absence of an acceptable scientific rationale
•   Lack of experience in the essential methodology
•   Questionable reasoning in experimental approach
•   Uncritical approach
•   Diffuse, superficial, or unfocused research plan
•   Lack of sufficient experimental detail
•   Lack of knowledge of published relevant work
•   Unrealistically large amount of work
•   Uncertainty concerning future directions
                                              for Grants
          NIH GUIDE                        and Contracts
                         U.S. Department of Health and Human Services




• Announces NIH Scientific Initiatives

• Provides NIH Policy and Administrative
  Information

• Available on the NIH Web Site :
  http://www.nih.gov
    Program Announcement
• Invites grant applications in a given research area

• May describe new or expanded interest in a
  particular extramural program

• May be a reminder of a continuing interest in a
  particular extramural program

• Generally has no funds set aside

• Applications reviewed in CSR along with
  unsolicited grant applications
    Requests for Applications (RFA)

• Announcement describing an institute initiative in a
  well-defined scientific area

• Invitation to the field to submit research grant
  applications for a one-time competition

• Set-aside of funds for a certain number of awards

• Applications generally reviewed within the issuing
  institute, in this case the NIDDK
NIH Information Sources
       Information on the World Wide Web
                   Selected Sites of Interest
• National Institutes of Health (http://www.nih.gov)
   – Office of Extramural Research (http://www.nih.gov/grants/oer.htm)
   – Grants Policy (http://www.nih.gov/grants/policy/policy.htm)

• Center for Scientific Review (http://www.csr.nih.gov)
   – Referral and Review (http://www.csr.nih.gov/refrev.htm)
   – Overview of Peer Review Process (http://www.
     csr.nih.gov/review/peerrev.htm)
   – CSR Study Section Rosters
     (http://www.csr.nih.gov/committees/rosterindex.asp)
   – NIH Peer Review Notes (http://www.csr.nih.gov/prnotes/prnotes.htm)
  Office of Extramural Research

• Handles requests for grant applications,
  program guidelines, and general information
  regarding grant applications

      Office of Extramural Research
      National Institutes of Health
      6701 Rockledge drive, suite 6095
      Bethesda, Maryland 20892-7910
      Phone: 301-435-0714
      Fax:      301-480-0525
      E-mail: grantsinfo@nih.Gov
There is no grantsmanship that will
 turn a bad idea into a good one,
             but……..

There are many ways to disguise a
           good one.

          Dr. William Raub,
          Former Deputy Director, NIH