CSR Review Outcomes for Human Subjects Research Applications by NIHhealth

VIEWS: 15 PAGES: 27

									Review Outcomes for R01 Human
   Subjects Research at CSR

                              Michael R. Martin, Ph.D.
       Director, Division of Physiology and Pathology
                           Center for Scientific Review
                          National Institutes of Health
                                      January 23, 2006
                Definition:
              Human Subjects
   A living individual about whom an
    investigator conducting research obtains
    (1) data through intervention or
    interaction with the individual or (2)
    identifiable private information.
   Applications designated Exemption 4 not
    considered human subjects research for
    this analysis
              Definition:
            New Investigator
   The principal investigator has not
    previously served as such on any PHS-
    supported research project other than a
    small grant (R03), an Academic Research
    Enhancement Award (R15), an
    exploratory/developmental grant (R21), or
    mentored career development awards for
    persons at the beginning of their research
    career (K01, K08, K22, and K23, K25).
                            Different Peer Review Outcomes
                      90%

                      80%

                      70%
Cummulative Percent




                      60%                                        Standard
                                                                 SRG
                      50%

                      40%

                      30%

                      20%

                      10%

                      0%
                            0    20    40        60   80   100
                                        Percentile
                        Different Peer Review Outcomes
              90%

              80%

              70%                                          Standard
              60%                                          SRG
Calc % Rank




              50%                                          Group 1
              40%

              30%

              20%

              10%

              0%
                    0       20   40        60   80   100
                                  Percentile
                            Different Peer Review Outcomes
                      90%
                      80%
                      70%                                       Standard
Cummulative Percent




                                                                SRG
                      60%                                       Group 1
                      50%
                                                                Group 2
                      40%
                      30%
                      20%
                      10%
                      0%
                            0   20    40        60   80   100
                                       Percentile
                                                HS+ vs. HS-, all CSR R01s:
                                                   10/04, 1/05 and 5/05
                                                               [IMPAC 2 data set]

                      100%

                      90%

                      80%
                                                                                                    HS+ type 1 estb n=5,217
Cummulative Percent




                      70%                                                                           HS+ type 2 estb n=1,778
                                                                                                    HS+ Type 1 new, n=2,405
                      60%
                                                                                                    HS- Type 1 estb n=9,293
                      50%                                                                           HS- Type 2 estb n=5,273
                                                                                                    HS- Type 1 new, n=4,658
                      40%                                                                           Reference

                      30%

                      20%

                      10%

                       0%
                             0   10   20   30     40      50        60     70       80   90   100
                                                       Percentile
                                         CSR All Type 1 A0 New Investigators
                                                 Oct 2000 - May 2004 Councils
                                                      Source: OER Data
                      35



                      30
                               HS+

                      25
                               HS-
Cummulative Percent




                      20



                      15



                      10



                       5



                       0
                           0         5           10              15             20   25   30
                                                            Percentile
                                 CSR All Type 1 A0 Experienced Investigators
                                             Oct 2000 - May 2004 Councils
                                                  Source: OER Data
                      35



                      30

                               HS+

                      25
                               HS-
Cummulative Percent




                      20



                      15



                      10



                      5



                      0
                           0     5          10              15              20   25   30
                                                         Percentile
                                         CSR All Type 2 A0 Investigators
                                               Oct 2000 - May 2004 Councils
                                                    Source: OER Data
                      35


                      30

                               HS+
                      25
Cummulative Percent




                               HS-

                      20


                      15


                      10


                       5


                       0
                           0         5        10              15              20   25   30
                                                           Percentile
               Observation:

   There are differences between HS+ and
    HS- within each type of application and
    across types of A0 applications [new
    versus experienced PI]
                                         CSR All Type 1 A1 New Investigators
                                                 Oct 2000 - May 2004 Councils
                                                      Source: OER Data
                      35



                      30
                               HS+


                      25       HS-
Cummulative Percent




                      20



                      15



                      10



                      5



                      0
                           0         5           10              15             20   25   30
                                                              Percentile
                                     CSR All Type 1 A1 Experienced Investigators
                                                 Oct 2000 - May 2004 Councils
                                                      Source: OER Data
                      35



                      30
                               HS+

                      25
                               HS-
Cummulative Percent




                      20



                      15



                      10



                      5



                      0
                           0         5          10              15              20   25   30
                                                             Percentile
                                     CSR All Type 2 A1 Investigators
                                           Oct 2000 - May 2004 Councils
                                                Source: OER Data
                      35


                      30

                               HS+

                      25
Cummulative Percent




                               HS-

                      20


                      15


                      10


                       5


                       0
                           0     5        10              15              20   25   30
                                                       Percentile
                Observation:

   Differences across types of A1 applications
    [new versus experienced PI]

   Difference is not as great as with the
    initial A0 application.
                                     CSR All Type 1 A2 New Investigators
                                             Oct 2000 - May 2004 Councils
                                                  Source: OER Data
                      35



                      30

                               HS+

                      25
                               HS-
Cummulative Percent




                      20



                      15



                      10



                       5



                       0
                           0     5          10              15              20   25   30
                                                         Percentile
                                     CSR All Type 1 A2 Experienced Investigators
                                                 Oct 2000 - May 2004 Councils
                                                      Source: OER Data
                      35



                      30
                               HS+

                      25
                               HS-
Cummulative Percent




                      20



                      15



                      10



                       5



                       0
                           0         5          10              15              20   25   30
                                                             Percentile
                                         CSR All Type 2 A2 Investigators
                                               Oct 2000 - May 2004 Councils
                                                    Source: OER Data
                      35
                               HS+

                      30
                               HS-


                      25
Cummulative Percent




                      20


                      15


                      10



                       5


                       0
                           0         5        10              15              20   25   30
                                                           Percentile
                Observation:

   No differences between HS+ and HS-
    within type of A2 application or across
    types of A2 applications [new versus
    experienced PI]
                 Question:

   Why do the HS+ applications improve so
    much on resubmission?

   Do new HS+ PIs submit applications at
    the same rate as new HS- PIs?
  Percent of all HS+ or HS- applications
                    HS+      HS-     delta
Type1NewA0         24.59%   20.20%   4.39%
Type1NewA1         9.20%    7.68%    1.52%
Type1NewA2         2.30%    1.94%    0.36%


Type1ExpA0         28.12%   27.58%   0.54%
Type1ExpA1         12.57%   11.55%   1.02%
Type1ExpA2         3.62%    3.40%    0.22%


Type2A0            11.71%   17.11%   -5.40%
Type2A1            5.95%    7.90%    -1.95%
Type2A2            1.96%    2.65%    -0.69%


Total              100%     100%      0%
                      Percent of All HS+ or HS- Applications Submitted and
                             Reviewed by CSR by Application Type

          30%


          25%


          20%
PERCENT




          15%


          10%
                                                                   HS+ New PI Type 1 A0

          5%                                                       HS- New PI Type 1 A0



          0%
                10


                          01


                                    05


                                              10


                                                        01


                                                                  05


                                                                            10


                                                                                      01


                                                                                                05


                                                                                                          10


                                                                                                                    01


                                                                                                                              05
            00


                      01


                                01


                                          01


                                                    02


                                                              02


                                                                        02


                                                                                  03


                                                                                            03


                                                                                                      03


                                                                                                                04


                                                                                                                          04
           20


                     20


                               20


                                         20


                                                   20


                                                             20


                                                                       20


                                                                                 20


                                                                                           20


                                                                                                     20


                                                                                                               20


                                                                                                                         20
                                                        COUNCIL ROUND DATE
                                Percent of All HS+ or HS- Applications Submitted
                                   and Reviewed by CSR by Application Type

          30%


          25%


          20%
PERCENT




          15%


          10%
                                                                   HS+ New PI Type 1 A0
                                                                   HS- New PI Type 1 A0
          5%
                                                                   HS+ Type 2 A0
                                                                   HS- Type 2 A0
          0%
                  0


                            1


                                     5


                                              0


                                                       1


                                                                5


                                                                          0


                                                                                   1


                                                                                            5


                                                                                                     0


                                                                                                              1


                                                                                                                       5
                   1


                           0


                                    0


                                             1


                                                      0


                                                               0


                                                                         1


                                                                                  0


                                                                                           0


                                                                                                    1


                                                                                                             0


                                                                                                                      0
                00


                        01


                                 01


                                          01


                                                   02


                                                            02


                                                                      02


                                                                               03


                                                                                        03


                                                                                                 03


                                                                                                          04


                                                                                                                   04
            20


                       20


                                20


                                         20


                                                  20


                                                           20


                                                                     20


                                                                              20


                                                                                       20


                                                                                                20


                                                                                                         20


                                                                                                                  20
                                                          COUNCIL ROUND DATE
    All funded Type 1 R01, and R29 awards
    between 1996-1999 reviewed by CSR,
      and subsequent Type 2 applications

                                  # with           % with
              Human        T1     subsequent T2    subsequent T2
PI on T1      Subj in T1   awards application      application

Established PI HS-          4,894          2,826          57.70%

              HS+           2,165            958          44.20%

New PI        HS-           3,472          2,119          61.00%

              HS+           1,330            603          45.30%
              Questions:

   Why, on a percent basis, are HS+ PIs
    more unlikely to apply for a type 2 A0
    or A1 applications?
   To what extent is the different
    percentage of applications in each
    category a factor in the reported
    difference in the global review
    outcomes?
      Acknowledgements

   Charles Dumais, CSR
   Teresa Lindquist, CSR

   Ted Kotchen, MCW and CSR
   Ray Hoffman, MCW

   The staff of OER/OD/NIH
Review Outcomes for R01 Human
   Subjects Research at CSR

                              Michael R. Martin, Ph.D.
       Director, Division of Physiology and Pathology
                           Center for Scientific Review
                          National Institutes of Health
                                      January 23, 2006

								
To top