Workshop 3 Slides - Social Science Research Institute

Document Sample
Workshop 3 Slides - Social Science Research Institute Powered By Docstoc
     Grant Writing Workshop
                     Linda Collins
Professor of Human Development Director, Methodology Center
                  Michael L. Hecht
                Distinguished Professor
             Communication Arts and Sciences
                  Elizabeth Farmer
     Associate Professor, Health Policy & Administration
    Workshop Outline
   NIH Organization
   NIH Funding Mechanisms
   The Grant Writing Process
     Focus on the R01

   The NIH Review Process
       Overview of Review Meeting
       Scenes from the NIH
       The Scoring Process
       A Penn State example
   Workshop Evaluation
    Thanks to:
   Linda A. McCauley, Professor and Dean, Nell
    Hodgson Woodruff School of Nursing, Emory
   Gregory Strayhorn, Professor, Department
    of Family Medicine Morehouse School of
   Marty Sliwinsky, Professor, Human
    Development & Family Studies, PSU
   Terry Noll & Donna Panasiti, Social Science
    Research Institute
     I. The NIH
   Department of Health and Human Services
       National Institutes of Health
           25 Awarding Institutes/Centers aka ICs (e.g.,
            NICHD, NIMH, NIDA, NIA)
           Center for Scientific Review
           Office of the Director
The NIH Extramural Team

          A. Program

       C. Review    B. Grants
    A. The Program/Institute Staff
        Program Administrator

    Maintains knowledge of scientific area
    Attends study section meetings
    Makes funding recommendations
    Monitors scientific progress
    Identifies scientific area of importance
    Reports to senior staff
    Development of programs and initiatives
         B. Grants Management

   Interprets Federal regulations and policies
   Assures compliance with Federal regulations
    and policies
   Monitors financial aspects of projects
   Interprets regulations and policy
                     C. Review:
         Scientific Review Group (1st Level)

   Center for Scientific Review (CSR) or NIH
    Institute & Center (IC)
   Scientific Review Group (SRG)
       Non-federal scientists with relevant expertise
       Led by a Scientific Review Officer (SRO)
            C. Review:
Advisory Council or Board (2nd Level)

   The potential awarding IC performs the second level of
   Comprised of scientists from the extramural community
    and public representatives.
   NIH program staff examine applications for impact
    (formerly “priority”) scores, percentile rankings, &
    summary statements against the IC’s needs.
   Program staff provide grant funding plan to Advisory
    Council or Board.
   Advisory Council or Board advises the IC director.
   Director makes final decision.
      NIH Grant Application Cycle
   Investigator        Institution            NIH

Initiates Research                         Peer Review
                     Submits Application


                      Allocates Funds
Conducts Research                            Decision
 Grant Application: It’s a
 process, not an event
1. Communicate with Program Officer
      Introducing ideas, getting feedback, pre-review
2. Get your proposal to the right review committee
      Review the rosters and talk to colleagues
      Effectively wording the abstract
      Make a written request
3. Seek feedback from colleagues and consultants on
    drafts of the grant (prepare ahead!)
4. Consider who is likely to review your grant (review
    the rosters) and make sure to know and cite their
    work when relevant
5. Recognize that funding on first submission is rare
II. NIH Grant Mechanisms
Pre-doctoral and Post-doctoral
Training Awards (F-series)
   Focuses on the training environment,
    research training plan, and the candidate’s
   Research strategy is 6 pages
   F31 covers doctoral study tuition and living
    stipend (levels set by NIH). Up to 60 months
    of support.
   F32 covers living stipend and funds for
    research/professional development
      Career Awards (K-series)
   Nine different Career Development (K) Awards
   Body of application is 12 pages for most
   Support individuals after they have completed
    clinical training and have accepted a faculty
   Provide for release time for research and monies
    for research/faculty development
   Small research grant
   Research strategy is 6 pages
   Can be used to collect pilot data, conduct
    feasibility studies
   Often analysis of an existing dataset
   May be used to develop new research
   $50K per year for 2 years
   Exploratory or Developmental Grant
   Research strategy is 6 pages
   Often little or no pilot data (pilot data can
    hurt application in some reviews!)
   Feasibility study
   2 years of funding (Total of $275K)
   Not necessarily the first step for new
   Supports the development of Phase III clinical
          Establishment of the research team

          Development data management tools

          Definition of recruitment strategies

          Finalization of the protocol

   Research strategy is 6 pages
   The planning grant is designed to lead to an
    application for support of a full-scale trial
R15 Area Award
   Used to stimulate research in educational institutions
    that have not been major recipients of NIH support
   Research strategy is 12 pages
   Opportunity to conduct research; strengthen the
    research environment of the institution; and benefit
    the student through exposure to and participation in
   Limited to up to three years of funding for total of
   Preliminary data generally not required
R01     (primary focus today)

   Used to support a discrete, specified,
    circumscribed research project
   NIH’s most commonly used grant program
   Research Strategy is 12 pages
   Advance permission required for $500,000 or
    more (direct costs) in any year
   Generally awarded for 3 to 5 years
    Research Center Grants (P-series)

  Established by ICs to meet special needs
 May support research and/or core facilities

 Usually initiated by the IC

     Request for Applications (RFA)

 P30, P50, P60, U54

 P01 Center grants are investigator-initiated,

  linked R01s with an additional core of funding
Research Supplements to Promote
Diversity in Health-Related Research
   Administrative supplement to R01
    grants and many other types
   Need to have at least 2 years of funding
    left on the grant
   Contact project officer for permission to
   Can be used for students, doctoral
    students, junior investigators
New and Early Stage Investigators:
A Competitive Edge
   New Investigator has not previously
    served as a PI for an R01; may have
    been an investigator or received other
    smaller, developmental or research
    training awards
   Early Stage Investigator (ESI) is within
    10 years of completing his/her terminal
    research degree, or is within 10 years
    of completing medical residency
        III. The Grant Writing Process
   Grant writing is:
       A skill like any other…
       But not the same skill as article writing
       Instead, more of a problem-based writing
        activity (theory and practice problem)
        A few preliminary tips
   Start early, make a timeline and STICK TO IT
       Should allow time for serious pre-submission review
        & subsequent revision
   Develop a relationship with project officers
   It is not possible to overdo clarity
   Let your passion come through in your proposal
   Take advantage of early stage and new
    investigator opportunities
    Getting ready to write
   Know what has been done
   Know what has been funded
      NIH website

   Decide on the problem
       Important enough to get funded but simple enough
        to explain as clean design in 12 pp (for the R01; less
        for other mechanisms)
       Mediators and moderators, mechanisms of change
   Assemble team--CAREFULLY
    Getting ready to write
   Communicate with program officer
        Establish a relationship and trust (funding
       Acquire information on mechanism and
       Obtain input on aims/proposal
Main Sections of the NIH Application
(see Sliwinsky et al. for example)

   Face Page
   Table of Contents
   Performance Sites
   Other information
       Project Summary/Abstract (Description)
       Public Health Relevance Statement
       Facilities & Resources
More Sections
   Key Personnel
       Biosketches --with personal statements
   Budgets (for each study year)
       Budget Justification
   Clinical Trial and HESC
   List of Research Plan Attachments
     Main Sections of the R01:
     Specific Research Plan
   [Introduction – revisions only]
   Specific Aims: The basis for the proposal’s organization
   Research Strategy
       Significance and Innovation
       Approach
          Preliminary studies

          Design

          Sample/recruitment/power analyses

          Procedures & measures

          Analyses
Page Limit Guide: Plan your proposal with
these limits in mind
  Section of Application                                         Page
  Introduction (for resubmission application only)                 1

  Specific Aims                                                    1

  Research Approach: R03, R13/U13, R21, R36, R41,                  6
  R43, Fellowships (F), SC2, SC3
  Research Approachy: R01, single project U01, R10,               12
  R15, R18, U18, R33, R24, R34, U34, R42, R44, DP3, G08,
  G11, G13, UH2, UH3, SC1
  Biographical Sketch                                              4

Page limits may vary for other funding mechanisms.
Check Funding Opportunity Announcement: http://enhancing-peer-
And More Sections
   Protection of Human Subjects
       Women and Minorities
       Planned Enrollment Table
       Children
   References Cited
   Letters of Support
   Resource Sharing Plan
   Checklist
              SO START EARLY!
Keys to R01 Success
   Impact and Significance
       Practice (2-3 sentences)
          Prevalence of problem in population

          Important social concern

       Theory (model)
            Building, testing, using
   Innovation
       Not through critique – new directions, value added
   Importance of Preliminary Research
Keys to Success - Methods
   Methods are very important
       Overall -- clarity and detail
       May include a table that traces aims to hypotheses
        to constructs to measures (table/s)
       Is the design feasible?
       Are there gaps in the methods (e.g., fidelity for
       Stats are very, very important (methodologist
        team member)
       Include a detailed timeline
More Methods - Issues
   Community-based participatory research --
    very strict about how this is done.
   Lab versus field
   Focus groups, interviews, analyses
And More Methods: The Sample
   Preference for representative samples
       Students only if relevant to age/situation
        (e.g., college drinking)
   Generalizability from a single entity
    (university, clinic, state)
   Unit of assignment is unit of analysis
   Direct Costs
     Senior Personnel (PI, co-Is, project director)

       (PSU fringe at 26.6%)
     Other Personnel (staff, RAs) (PSU fringe 14.1%

       for Grad Assts AY; 8.3% for staff and summer)
     Equipment

     Travel

     Participant/Trainee Support Costs
       Other
            Materials and supplies
            Publication costs
            Consultant services
       Subawards/Consortium/Contractual Fees
       Other
   Indirect Costs (> 45% at PSU, but does not
    include all expenses)
   Budget justification
Receiving the Summary
Statements: The Hardest Part!
1. Reviews critical, even harsh
2. Reviewers usually find grant’s weaknesses,
   while recognizing strengths
3. Summary statements spend much more time
   on critique than praise
4. Many investigators experience a mixture of
   rage and depression when they read their
   summary statements and easily lose
5. Take a day or two (or more!) and then read
   again with a cooler head
 Receiving the Summary
 Statements: Bouncing Back!
1. Ask experienced colleagues to read reviews
2. Don’t interpret criticism as hopeless
3. Program Officer may be helpful in clarifying
4. If “discussed” (rather than triaged), you
   have a chance of funding in next round
5. The lower the initial score, the fewer
   problems and more likely to be successful
   after revision
Resilience and Flexibility!
1. Persistence pays off in the grant
2. Second submission must respond to the
   critiques through revision or clearly
   defending reasoning
3. Same reviewers may or may not review
   resubmission, but will see critique
     Most Common Reasons for a
     Poor Score (in priority order)
   Lack of impact or significance
   Lack of new or original ideas – show your passion!
   Hypotheses ill-defined, superficial, lacking, unfocused, or
    unsupported by preliminary data
   Methods unsuitable, not feasible, not rigorous or not likely to yield
    results; methods don’t clearly link to aims
   Design not logical, inappropriate instrumentation, poor timing or
    conditions; doesn’t link well to aims
   Data management and analysis vague, not rigorous; analyses don’t
    clearly link to aims
   Inadequate expertise or knowledge of field for PI; too little time to
    devote to the work
   Poor resources or facilities; limited access to appropriate population
When to Revise
    Basic idea was significant and
     innovative or these can be bolstered
    Design/measurement/analysis
     problems can be clarified (more
     information) or fixed
    Need preliminary data
    Problem is poor writing
IV. The NIH Review Process
A. The Review Meeting:
The SRO’s Role Prior to Meeting
   Point of contact until review group meets (then
    project officer)
   Analyze submissions for completeness and conflicts
   Recruit ad hoc reviewers as needed
   Schedule 1-2 day meeting
   Assign applications to reviewers (at least 3)
       Primary, secondary, discussant
   Create review order based on preliminary impact
    scores from best to worst within categories
Reviewers’ Role Prior to Meeting
   Familiarize self with criteria, mechanisms, and scoring
   Review assigned applications
       Assign scores to each criteria and other areas
       Write bulleted strengths and weaknesses for each criteria
       Reviews are advice to institutes for funding decisions, not
        advice to PI
   Post scores and comments on NIH Commons
   Read other reviews of assigned applications
   Prepare presentation of reviews
   Skim/read non-assigned applications
        Format of the Review Meeting

   SRO opening remarks
   Chair orientation
   New investigator R01 grants
   Other R01 grants
   Other grant types (R03, R15, R21, R34)
       Applications discussed in order of Impact
        Score; bottom 50% are not discussed
SRO Opening Remarks
   Confidentiality
   Review order
   Proposals below median within each
    category may not be discussed
Chair Orientation
    Start with reviewer impact scores
         May differ from posted scores
    Goal of discussion is to clarify not reach
    If scores are similar, shorter discussion
    If scores are dissimilar, longer discussion
    Recommended time
         Primary – 5 minutes
         Secondary – 3 minutes
         Discussant – 2 minutes
Review Discussion
   Identify proposal
   Members in conflict leave
   Reviewers provide preliminary impact scores
   Reviews
       Impact, Significance, Investigators, Innovation, Approach,
       Stress main points, do not repeat previous points
       Non-reviewers typically ask questions to clarify
   Human Subjects issues affecting scoring
   Open discussion to entire committee
Review Discussion (continued)
   Ask for reviewers impact scores again
   Identify the reviewers’ recommended
   Ask if anyone wants to score outside
    the range
   Entire committee records impact score
   Discuss budget and other issues
    B. Scenes from the NIH
  C. The Scoring Process
1. Overall Impact Score: likelihood project will “exert a
   sustained, powerful influence on the research field(s)
   involved (1-9 scale)
2. A separate 1-9 score for each of 5 core criteria
   (Significance, Investigators, Innovation, Approach,
3. Additional review criteria help determine scientific and
   technical merit BUT are not scored separately
4. Additional review considerations are addressed by
   reviewers, but are not scored & are discussed after
   group scores.
     Score Criteria
   Overall Impact: will project exert a sustained, powerful
    influence on the research field(s) as indexed by 5 core
    review criteria
    1. Significance: important problem addressed; how will this
    improve scientific knowledge, technical capability, and/or
    clinical practice
    What is the Difference Between
    Impact and Significance ?
   Impact Addresses:
       Probability of whether the research will exert a
        sustained, powerful influence on the research field
   Significance Addresses:
       Does the project address an important problem or a
        critical barrier to progress in the field?
       If the aims are achieved, how will scientific
        knowledge, technical capability, and/or clinical
        practice be improved?
    Score Criteria (continued)
 2. Investigators: PI & other researchers well suited to the
project; appropriate experience & training; ongoing record
of accomplishments; complementary & integrated
experience; leadership approach, governance, and
organizational structure appropriate for project

3. Innovation: the work challenges and seeks to shift
current research or practice paradigms; utilizing novel
theory, approaches or methods, instrumentation, or
interventions; the work is novel
    Score Criteria (continued)
4. Approach: strategy, methodology, analyses are well-
reasoned and appropriate; potential problems & alternative
strategies thought through; benchmarks set; risk is

5. Environment: the environment will contribute to the
project’s success; institutional support, equipment, & other
resources sufficient; unique features of the environment,
subject population, collaborative arrangements
     Additional Review Criteria (not scored)

Human Subjects:
1.    Protection of human subjects
2.    Data safety monitoring plan (clinical trials only)
3.    Inclusion of women, minorities, children
4.    Vertebrate animals
5.    Biohazards
     Additional Review Considerations
1.    Budget and period of support
2.    Select agent research (infectious agents)
3.    Applications from foreign organizations
4.    Resource sharing plans
5.    Additional comments to applicant
1. Exceptional: Exceptionally strong with essentially no weaknesses
2. Outstanding: Extremely strong with negligible weaknesses
3. Excellent: Very strong with only some minor weaknesses
4. Very Good: Strong but with numerous minor weaknesses
5. Good: Strong but with at least one moderate weakness
6. Satisfactory: Some strengths but also some moderate
7. Fair: Some strengths but with at least one major weakness
8. Marginal: A few strengths and a few major weaknesses
9. Poor: Very few strengths and numerous major weaknesses
   Minor Weakness: An easily addressable
    weakness that does not substantially
    lessen impact
   Moderate Weakness: A weakness that
    lessens impact
   Major Weakness: A weakness that
    severely limits impact
Final Overall Impact Score:
  Mean of all reviewers’ final impact scores X 10
  Range = 10 (high impact) -- 90 (low impact)

NOTE: New scoring likely to produce more applications
 with identical scores (“tie” scores). Thus, other
 factors (e.g., mission relevance, portfolio balance)
 will be considered when all other things are
 essentially equal
    What to Look for in New Proposal Format
   New Research Plan Components
       Specific Aims
          Includes language about the impact of the research

       Research Strategy
          Includes Background & Significance; Preliminary

           Studies/Progress Report; Research Design & Methods
   Facilities and Equipment
       Reflects the Environment criterion
       For ESIs describes the institutional investment in the success of
        the investigator
   Biographical Sketch
       Requires Personal Statement; no more than 15 pubs based on
        recency, importance to field, and/or relevance to the application
         What to Look for in the New Format:
•   Does application challenge/seek to shift current research or
    clinical practice paradigms by utilizing novel theoretical
    concepts, approaches or methodologies, instrumentation, or

•   Concepts, approaches or methodologies, instrumentation, or
    interventions novel to one field of research or novel in a
    broad sense?

•   Refinement, improvement, or new application of theoretical
    concepts, approaches or methodologies, instrumentation, or
    interventions proposed?

•   Not all applications need to be innovative!
        What to Look for in New Format
•   Are the overall strategy, methodology, and analyses
    well-reasoned and appropriate to accomplish the
    Specific Aims of the project?

•   Are potential problems, alternative strategies, and
    benchmarks for success presented?

•   If the project is in the early stages of development,
    will the strategy establish feasibility and will
    particularly risky aspects be managed?
         What to Look for in New Format

• Personal Statement:
   • Why their experience and qualifications make them
     particularly well-suited for their roles in the project
• Publications:
   • Recommended: no more than 15---up to five of the best; up
     to five of the most relevant to the proposed research; up to
     five of the most recent
• If Early Stage Investigators or New Investigators, do
  they have appropriate experience and training?
• If Established, have they demonstrated ongoing record
  of accomplishments that have advanced their field(s)?
D. A Penn State Example
   Starring:
       Elizabeth Farmer as Reviewer 1
       Michael Hecht as Reviewer 2
       Linda Collins as Reviewer 3
                Links of Interest
Enhancing Peer Review Criteria:
Page Limits: http://enhancing-peer-
Human Subjects:
SF424 guidelines for submission:
    General Resources
Weekly NIH Updates
New vs. Revised Applications
NIH Grant Writing Tip Sheets
Getting an RO1
More general resources
NSF Proposal Writing
Other Proposal Writing Guides
Reasons Proposals Fail
   /articles/2310/nsf_grant_reviewer_tells_all/ (see end)
New Investigator Resources
Early Stage Investigators
New Investigators
NIH Websites
  (forms, grant search, etc.)

Shared By:
yaofenji yaofenji