Peer Review of NIH Research Grant Applications - PowerPoint

Document Sample
Peer Review of NIH Research Grant Applications - PowerPoint Powered By Docstoc
					Peer Review of NIH
  Research Grant
      Center for Scientific Review

      National Institute of
      Diabetes and Digestive and
      Kidney Diseases

      National Institutes of Health
    U. S. Department of Health and Human Services

                                        Deputy Secretary

                                                                              Health Resources
 Administration for     Administration on                Food and Drug
                                                                                and Services
Children and Families       Aging                        Administration
       (ACF)                (AoA)                            (FDA)

                           Agency for
Health Care Financing                                      Indian Health      National Institutes
                        Health Care Policy
   Administration                                             Services            of Health
                          and Research
       (HCFA)                                                  (IHS)                (NIH)

     Centers for        Agency for Toxic             Substance Abuse and
                                                                              Program Support
   Disease Control      Substances and               Mental Health Services
   and Prevention       Disease Registry                Administration
       (CDC)               (ATSDR)                        (SAMHSA)
National Institutes of Health

                    Much of the
                    biomedical research
                    in the united states
                    is supported by the
                    federal government,
                    primarily the
                    National Institutes
                    of Health (NIH)
                        National Institutes of Health
                                                                   Office of the Director

                                                                                    National Institute                                 National Institute
                         National Institute          National Institute
 National Institute                                                                  of Arthritis and            National Cancer        of Child Health
                         on Alcohol Abuse             of Allergy and
     on Aging                                                                        Musculoskeletal                Institute             and Human
                          and Alcoholism            Infectious Diseases
                                                                                    and Skin Diseases                                    Development

National Institute on    National Institute          National Institute
                                                                                                                 National Institute
Deafness and Other        of Dental and               of Diabetes and               National Institute                                    National Eye
                                                                                                                 of Environmental
 Communication             Craniofacial                Digestive and                 on Drug Abuse                                         Institute
                                                                                                                  Health Sciences
     Disorders               Research                Kidney Diseases

                                                                                                                 National Institute
 National Institute      National Heart,             National Human
                                                                                    National Institute            of Neurological      National Institute
    of General           Lung, and Blood             Genome Research
                                                                                    of Mental Health               Disorders and      of Nursing Research
 Medical Sciences           Institute                    Institute

                          National Center
National Institute of                                       Fogarty                     National Center                               National Center on
                        for Complementary                                                                        National Library
Biomedical Imaging                                       International                   for Research                                 Minority Health and
                          and Alternative                                                                         of Medicine
and Bioengineering                                          Center                        Resources                                    Health Disparities

                                                                           Center for
                                                                                                         Center for
                                       Clinical Center                    Information
                                                                                                      Scientific Review
           A Typical Institute/center

National             Office of the IC            Board of
Advisory                Director                 Scientific
Council                                         Counselors

   Extramural                               Intramural


                                        Laboratory   Clinical
Grants       Contracts                   Studies     Studies
NIH Extramural Program Mission
                  Manage a portfolio of investments to
                    improve health through science

Identify scientific                                            Foster the best
  opportunities                                                   science

          Ensure proper                               Promote effective
           stewardship                                 communication
 Exemplify and promote the highest level          Promote effective ways to communicate
of scientific integrity, public accountability,   scientific information to scientists, health
     and responsibility in the conduct                   practitioners, and the public
                  of science
   NIH Extramural Program

Grant         patron
              (Assistance, encouragement)

Cooperative   partner
Agreement     (assistance but substantial
              Program involvement)

Contract      purchaser
Overall Peer Review Process
    Dual Review System for Grant Applications
First Level of Review
Scientific Review Group (SRG)
    Provides Initial Scientific Merit
     Review of Grant Applications
    Rates Applications and Makes
     Recommendations for Appropriate
     Level of Support and Duration of

                               Second Level of Review
                                      Assesses Quality of SRG
                                        Review of Grant Applications
                                      Makes Recommendation to
                                       Institute Staff on Funding
                                      Evaluates Program Priorities
                                        and Relevance
                                      Advises on Policy
            Review Process for a Research Grant
                                     National Institutes of Health
 Research      School or Other
  Grant        Research Center        Center for Scientific Review
                                        Assigns to IRG/Study Section & IC

Initiates                                      Study Section
Research       Submits Application
Idea                                     Evaluates for Scientific Merit

                                       Evaluates for Program Relevance

                                     Advisory Councils and Boards
Conducts        Allocates Funds
Research                                     Recommends Action

                                             Institute Director
                                       Takes final action for NIH Director
                               Overall Timeframe From
                                 Submission to Award

           There are three overlapping cycles per year :
JAN     FEB   MAR      APR   MAY   JUN      JUL AUG    SEP OCT      NOV      DEC   JAN FEB MAR   APR    MAY      JUN    JUL

Cycle 1
      Receipt                            Review        Council
          Referral                                                        Award

                     Cycle 2
                             Receipt                             Review              Council
                              Referral                                                           Award

                                                  Cycle 3
                                                        Receipt                      Review            Council
                                                        Referral                                                       Award
          Typical Timeline for a New
       Individual Research Project Grant
               Application (R01)
 There are three overlapping cycles per year:

–Submit in February (June, October)
  –Review in June (October, February)
    –Council in September (January, May)
       –Earliest award in December (April, July)

Cycle 1----
Cycle 2----
Cycle 3----
       Types of Scientific Review Groups
       Where Are Applications Reviewed?
                                Research Projects
CSR IRGs                        Academic Research
 Study Sections                  Enhancement Awards
                                Postdoctoral Fellowships
 Special Emphasis Panels
                                Small Business Innovation
                                Shared Instrumentation
INSTITUTES                     Program Projects
 Scientific Review Groups      Centers
                               Institutional Training Grants
                               Conference Grants
                               Career Awards
                               Small Grants
                               RFAs (R01, R21, R03, U01)
 Contract Review Committees    Contracts
      Peer Review of NIH Support Mechanisms
          CSR                                 Institutes
Research project grant (R01)       Program Project grant (P01)
Postdoctoral fellowship (F32)      Center grant (P30, P50, P60)
Senior fellowship (F32)           Institutional fellowship (T32)
Fogarty international center       Academic career award (K07)
   Fellowship (F05, F06)           Mentored clinical scientist
Short-term training (T35)          development award (K08)
Small business grants (R41, R42    Conference grant (R13)*
   R43, R44)                       MARC fellowships (F34, F36, T34)
Academic Research Enhancement      Minority Biomedical Support
   Award (R15)                       Grant (S06)
Biomedical Research Support        Resource grant (P40, P41, R24,
    Shared Instrumentation           R26, R28)
     Grant (S10)                   Contract
                                        *Reviewed by CSR upon request
Center for Scientific Review (CSR)

    Focal point for initial review at NIH
                    •Central receipt point for PHS

                    •Referral to Institutes and to IRGs
                    and Study Sections

                    •Review of most research and
                    research training applications ..for
                    scientific merit
  Grant Application
Receipt and Assignment
       Applications Submitted to NIH

• Approximately 46,000
  grant applications are
  submitted to NIH each
  year, of which 25-30%
  are funded
• Competing grant
  applications are received
  for three review cycles
  per year
                    CSR Receipt and Referral:
                  Central Receipt Point for Applications
                  Submitted to the Public Health Service

       National Institutes               Substance Abuse and
               of                        Mental Health Services
            Health                          Administration

                             Receipt &         Agency for Health Care
  Centers for                 Referral           Policy & Research
Disease Control
                                       Office of
                    Food              Assistant
                     &               Secretary for
                    Drug                Health
             Competing Applications
                Reviewed By:
     Fiscal Year                      PHS            NIH           CSR

          1976                       27,213        19,509        16,268

          1985                       31,807        28,407        23,626

          1992                       37,826        31,145        24,708
          1993                       39,968        38,447        26,178
          1994                       43,394        41,739        28,033
          1995                       44,342        41,899        29,203
          1996                       38,579        38,012        27,253
          1997                       38,767        38,065        26,516
          1998                       39,474        37,855        26,074
          1999                       44,395        42,383        31,837
          2000                       45,705        43,814        32,091

NOTE: Starting in FY93, NIDA, NIAAA, and NIMH are included in NIH totals
               Receipt Dates
Depend on the Type of Application

 •   Jan, May, Sept 10: institutional training grant applications
 •   Jan, May, Sept 25: academic research enhancement award
 •   Feb, June, Oct 1: new research applications
 •   Mar, Jul, Nov 1: revised, competing continuations, and
                        Supplemental Applications
          and Small Business Innovation Research Applications
            Flow of a Competing Grant Application
                Through the CSR Referral Section
                                                                          Project Control Unit:
                                       Mailroom                            Receipt & Record Group
   Application                     (Receive, Date Stamp,                   (Process, Enter Skeletal
(Original & 5 Copies)                                                        Data into Computer,
                           Preliminary Sort, Add Bar Code)
                                                                                Print Status)

(Advance Copy)                                        Assignment Unit:
Scientific Review              (Assign to Scientific Review Group and Awarding Organization)

                                   Project Control Unit:
   (2 Copies)                    Review & Control Group
Scientific Review            (Add Labels, Send Data to IMPAC,
     Group                                                                         (Original)
                                 Print Mailers, Distribute)                   Print Shop

                        (1 copy)       (1 copy)
                        OPERA           Project       (35 copies)    (20 copies)     (1 copy)    (1 copy)
                        EIS,DMC      Control Files       SRG           Original       Project    Research
                                                                      Awarding       Control    Documents
                                                                    Organization                 (IC only)
 Applications are Assigned by

Referral Officers:
 Professional scientists, most of
  whom also serve as scientific
  review administrators of CSR
  study sections
  Applications Are Assigned To:
• Scientific review groups based on:
  – Specific review guidelines for each
    scientific review group

• Institutes based on:
  – Overall mission of the Institute
  – Specific programmatic mandates and
    interests of the Institute
     Assignment to Institutes
Applications are referred to an institute or
center as the potential funding component:

–This assignment is based on a match between
 the research proposed and the overall mission
 of the institute or center

–Where applications are appropriate for more
 than one institute or center, multiple
 assignments are made
         Sample Application Number

         Individual             serial               amended
         Research              number

     1    R01     DK      12345          01     A1

   New            national                  grant
Application       cancer                  support
                   Institute                  year
                 Assignment of Applications
                                      Review Group                          Institute

                                                                                       Form Approved Through 9-30-97
 AA                                                                                               OMB No. 0925 0001

Department of Health and Human Services                   Leave Blank--for PHS Use Only
         Public Health Service                            Type 1 Activity R21 Number A154321-01
          GRANT APPLICATION                               Review Group ZDK1      Formerly
Follow instructions carefully. Do not exceed character    Council/Board (Month,Year) Date Received
       length restrictions indicated on sample.
                                                                 09/01                            07-26-01
1. Title of Project (Do not exceed 56 characters, including spaces and punctuation.)
Planning Grant for Diabetes & Sc Ed in Tribal Schools
 2. Response to Specific Request for Applications or Program                      X No      x       Yes (if “Yes”
 Number: DK 01-033 Title: Planning Grant for Diabetes and Science Education …                   number and title)

                                     3. Principal Investigator/Program Director

3a. Name (Last, first, middle)                                   3b. Degree(s)         3c. Social Security No.
Smith, Brian E.                                                        Ph.D.
        Assignment Notification Letter
Dear Dr. Smith:
 Your grant application entitled “PLANNING GRANT FOR
SCHOOLS ” has been received by the National Institutes of Health
and assigned to a scientific review group (SRG) for scientific merit
evaluation and to an institute/center for funding consideration.
Specific information about your assignment is given below. The
initial peer review should be completed by AUGUST, 2001, and a
funding decision made shortly after the appropriate National
Advisory Group meets in SEPTEMBER, 2001. Questions about
the assignment should be directed to the scientific review
administrator (SRA) or the Division of Receipt and Referral, Center
for Scientific Review (formerly Division of Research Grants) at
(301) 435-0715. Other questions prior to review should be directed
to the scientific review administrator and questions after the review
to the program staff in the institute/center.
Assignment Notification Letter (continued)
Assignment Number: 1 R01 DK12345 - 01
Dual Assignment: NS
Scientific Review Group:
Information about this IRG may be found on the
NIDDK Home page (
Scientific Review Administrator:
BETHESDA MD 20892-5452
(301) 594-8885
   Assignment Notification Letter

(301) 594-8885
Initial Review in IC
            Peer Review
• CSR and IC Study Sections are managed by a
  Scientific Review Administrator (SRA) who is a
  scientist, usually at the Ph.D. level, whose scientific
  background is close to the expertise of the study
• Each CSR/IC standing study section has 12-24
  members who are primarily from academia
• Institute review is also different in that Special
  Emphasis Panels are constituted specifically to review
  applications that have been received in response to an
  RFA or other Institute specific grant mechanisms
• As many as 60-100 applications are reviewed at each
  study section meeting, again this depends on the
  complexity of the review
   Scientific Review Administrator
Designated Federal official with overall
  responsibility for the review process, including:
• Performing administrative and technical review of
  applications to ensure completeness and accuracy
• Selecting reviewers based on broad input
• Managing study section meetings
• Preparing summary statements
• Providing any requested information about study
  section recommendations to institutes and national
  advisory councils/boards
Selection of Peer Reviewers
                  Active and Productive




   Scientific Community
               Criteria for Selection of Peer Reviewers

                              Active & Productive
                                                    Meet Internal Administrative
Doctoral or     Research                                                   •Institutional
Equivalent      Capability                                    Interest         Affiliation
 Degree                                                                       •Non-Federal
                                                              Serving            Employment
                                                                               •Gender &
                                                     Expertise in Discipline of Ethnic
                                                        Review Group and          Status
               Non-Doctoral                           Specialization Needed

      Scientific Community                          Active and Productive
Study Section Meeting
         Certification of No Conflict of Interest
This will certify that in the review of applications and proposals
  by (study section) on (date), I did not participate in the
  evaluation of any grant or fellowship applications from (1) any
  organization, institution or university system in which a
  financial interest exists to myself, spouse, parent,child, or
  collaborating investigators; (2) any organization in which I
  serve as officer, director, trustee, employee or collaborating
  investigator; or (3) any organization which I am negotiating or
  have any arrangements concerning prospective employment or
  other such associations.

  __________________            __________________
  __________________            __________________
• Review materials and proceedings of review
  meetings represent privileged information to be
  used only by consultants and NIH staff.
• At the conclusion of each meeting, consultants
  will be asked to destroy or return all review-
  related material.
• Consultants should not discuss review
  proceedings with anyone except the SRA.
• Questions concerning review proceedings
  should be referred to the SRA.

      Review of Research Grants
 –    Significance
 –    Approach
 –    Innovation
 –    Investigator
 –    Environment
 –    Overall Evaluation & Score Reflects Impact on
   Review Criteria (Continued)
• Significance: Does the study address an important problem?
  How will scientific knowledge be advanced?

• Approach: Are design and methods well-developed and
  appropriate? Are problem areas addressed?

• Innovation: Are there novel concepts or approaches? Are
  the aims original and innovative?

• Investigator: Is the investigator appropriately trained?

• Environment: Does the scientific environment contribute to
  the probability of success? Are there unique features of the
  scientific environment?
         Specific Review Criteria
            RFA DK-01-033
• Administrative, leadership qualifications and
  experience of the Program Director.
• Adequacy and availability of any necessary
  institutional facilities and resources.
• Adequacy of the plan for developing a diabetes
  science education program, including use of existing
  resources, building partnerships with other institutions,
  use of consultants, and plans for critical review of the
• Documentation of potential participants for the science
  education program.
Research Involving Human Subjects
    Research Involving Human Subjects
Areas exempt from human subject review
• Education research
   – Normal educational practices
• Educational tests, survey or interview procedures, or
  observation of public behavior
   – Subjects not identified
   – Subjects’ privacy rights protected
• Educational tests, survey or interview procedures, or
  observation of public behavior not exempt in previous
  category if:
   – Subjects are public officials or public office candidates
   – Federal statute requires confidentiality without exception
       Research Involving Human Subjects
Areas exempt from human subject review
• Collection or study of existing data, documents, records,
  pathological specimens
   – Information publicly available
   – Subjects not identified
• Research and demonstration projects regarding certain public
  benefit or service programs
• Taste and food quality evaluation and consumer acceptance
  studies using
   – Foods without additives
   – U.S. Government approved food ingredient
     Research Involving Children
Children must be considered for inclusion in
all human subject research supported by NIH
Scientific Review Group or Study
          Section Actions

• Scored, Scientific Merit
  Rating (priority scores and
• Unscored (lower half)
• Deferral
• Scored -- Scientific Merit Rating 1.0 to
  approximately 3.0
  Based on the relevant review criteria, the application is
   judged to be in the upper half of applications
   reviewed by the study section or scientific review
   group. The recommendation can be for the requested
   time and amount or for an adjusted time and amount.
   A priority score is provided, and a summary statement
   prepared that incorporates the written critiques plus a
   resume and summary of the discussion.
• Unscored
 Application is unanimously judged to be
  in the lower half of applications
  reviewed by the study section or
  scientific review group. No priority
  score is assigned. The summary
  statement provided to the applicant is a
  compilation of reviewers’ comments
  prepared prior to the meeting.
• Deferral
  The study section cannot make a
   recommendation without additional
   information. This information may be
   obtained by a project site visit or by
   submission of additional material by
   the applicant.
 Post Scientific Review Group

• Calculations of priority scores and
  percentile rankings
• Preparation of summary statements
• Removal of applications from national
  advisory council / board consideration
      Summary Statement
Once applications are reviewed, the results are
documented by the SRA in a summary statement
and forwarded to the Institute (and the PI) where a
funding decision is made:
The summary statement contains:
– Overall Resume and Summary of Review Discussion
– Essentially Unedited Critiques
– Priority Score and Percentile Ranking
– Budget Recommendations
– Administrative Notes
National Advisory Council or
       Board Review
      Council Actions

• Concurrence with study section
• Modification of study section
• Deferral for re-review
 What Determines Which
  Awards Are Made?

• Scientific merit
• Program considerations
• Availability of funds
 Mechanisms for Preliminary Studies
• Small grants (R03)
   – Feasibility/new technology/innovative high risk ideas
   – Short term, usually $25,000 - $50,000
   – Extremely variable in detail - example:
         • NCRR: 1 year, $35,000, non-renewable
         • NIDDK: 3-6 months, $12,500 - $25,000
         • NCI: 2 years, $50,000
   – Institute review
• Exploratory/developmental grants (R21)
   –   Feasibility (for those without preliminary data)
   –   $100,000/2 years (for this RFA it is $75,000/1 year)
   –   Increasing use
   –   CSR review (RFA review performed by NIDDK)
Preparation of an Application
PHS Research Grant
 Application Kit (form PHS 398)

             Mail Completed Forms To:
             ROCKLEDGE II ROOM 1040 MSC-7710
             BETHESDA MD 20892-7710
  When Preparing an Application
• Read instructions
• Never assume that reviewers “will know what you
• Refer to literature thoroughly
• State rationale of proposed investigation
• Include well-designed tables and figures
• Present an organized, lucid write-up
• Obtain pre-review from faculty at your institution
        Common Problems in Applications
•   Lack of new or original ideas
•   Absence of an acceptable scientific rationale
•   Lack of experience in the essential methodology
•   Questionable reasoning in experimental approach
•   Uncritical approach
•   Diffuse, superficial, or unfocused research plan
•   Lack of sufficient experimental detail
•   Lack of knowledge of published relevant work
•   Unrealistically large amount of work
•   Uncertainty concerning future directions
                                              for Grants
          NIH GUIDE                        and Contracts
                         U.S. Department of Health and Human Services

• Announces NIH Scientific Initiatives

• Provides NIH Policy and Administrative

• Available on the NIH Web Site :
    Program Announcement
• Invites grant applications in a given research area

• May describe new or expanded interest in a
  particular extramural program

• May be a reminder of a continuing interest in a
  particular extramural program

• Generally has no funds set aside

• Applications reviewed in CSR along with
  unsolicited grant applications
    Requests for Applications (RFA)

• Announcement describing an institute initiative in a
  well-defined scientific area

• Invitation to the field to submit research grant
  applications for a one-time competition

• Set-aside of funds for a certain number of awards

• Applications generally reviewed within the issuing
  institute, in this case the NIDDK
NIH Information Sources
       Information on the World Wide Web
                   Selected Sites of Interest
• National Institutes of Health (
   – Office of Extramural Research (
   – Grants Policy (

• Center for Scientific Review (
   – Referral and Review (
   – Overview of Peer Review Process (http://www.
   – CSR Study Section Rosters
   – NIH Peer Review Notes (
  Office of Extramural Research

• Handles requests for grant applications,
  program guidelines, and general information
  regarding grant applications

      Office of Extramural Research
      National Institutes of Health
      6701 Rockledge drive, suite 6095
      Bethesda, Maryland 20892-7910
      Phone: 301-435-0714
      Fax:      301-480-0525
      E-mail: grantsinfo@nih.Gov
There is no grantsmanship that will
 turn a bad idea into a good one,

There are many ways to disguise a
           good one.

          Dr. William Raub,
          Former Deputy Director, NIH