Get documents about "CSR Review Outcomes for Human Subjects Research Applications
Document Sample
Review Outcomes for R01 Human
Subjects Research at CSR
Michael R. Martin, Ph.D.
Director, Division of Physiology and Pathology
Center for Scientific Review
National Institutes of Health
January 23, 2006
Definition:
Human Subjects
A living individual about whom an
investigator conducting research obtains
(1) data through intervention or
interaction with the individual or (2)
identifiable private information.
Applications designated Exemption 4 not
considered human subjects research for
this analysis
Definition:
New Investigator
The principal investigator has not
previously served as such on any PHS-
supported research project other than a
small grant (R03), an Academic Research
Enhancement Award (R15), an
exploratory/developmental grant (R21), or
mentored career development awards for
persons at the beginning of their research
career (K01, K08, K22, and K23, K25).
Different Peer Review Outcomes
90%
80%
70%
Cummulative Percent
60% Standard
SRG
50%
40%
30%
20%
10%
0%
0 20 40 60 80 100
Percentile
Different Peer Review Outcomes
90%
80%
70% Standard
60% SRG
Calc % Rank
50% Group 1
40%
30%
20%
10%
0%
0 20 40 60 80 100
Percentile
Different Peer Review Outcomes
90%
80%
70% Standard
Cummulative Percent
SRG
60% Group 1
50%
Group 2
40%
30%
20%
10%
0%
0 20 40 60 80 100
Percentile
HS+ vs. HS-, all CSR R01s:
10/04, 1/05 and 5/05
[IMPAC 2 data set]
100%
90%
80%
HS+ type 1 estb n=5,217
Cummulative Percent
70% HS+ type 2 estb n=1,778
HS+ Type 1 new, n=2,405
60%
HS- Type 1 estb n=9,293
50% HS- Type 2 estb n=5,273
HS- Type 1 new, n=4,658
40% Reference
30%
20%
10%
0%
0 10 20 30 40 50 60 70 80 90 100
Percentile
CSR All Type 1 A0 New Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25
HS-
Cummulative Percent
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
CSR All Type 1 A0 Experienced Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25
HS-
Cummulative Percent
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
CSR All Type 2 A0 Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25
Cummulative Percent
HS-
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
Observation:
There are differences between HS+ and
HS- within each type of application and
across types of A0 applications [new
versus experienced PI]
CSR All Type 1 A1 New Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25 HS-
Cummulative Percent
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
CSR All Type 1 A1 Experienced Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25
HS-
Cummulative Percent
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
CSR All Type 2 A1 Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25
Cummulative Percent
HS-
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
Observation:
Differences across types of A1 applications
[new versus experienced PI]
Difference is not as great as with the
initial A0 application.
CSR All Type 1 A2 New Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25
HS-
Cummulative Percent
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
CSR All Type 1 A2 Experienced Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
30
HS+
25
HS-
Cummulative Percent
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
CSR All Type 2 A2 Investigators
Oct 2000 - May 2004 Councils
Source: OER Data
35
HS+
30
HS-
25
Cummulative Percent
20
15
10
5
0
0 5 10 15 20 25 30
Percentile
Observation:
No differences between HS+ and HS-
within type of A2 application or across
types of A2 applications [new versus
experienced PI]
Question:
Why do the HS+ applications improve so
much on resubmission?
Do new HS+ PIs submit applications at
the same rate as new HS- PIs?
Percent of all HS+ or HS- applications
HS+ HS- delta
Type1NewA0 24.59% 20.20% 4.39%
Type1NewA1 9.20% 7.68% 1.52%
Type1NewA2 2.30% 1.94% 0.36%
Type1ExpA0 28.12% 27.58% 0.54%
Type1ExpA1 12.57% 11.55% 1.02%
Type1ExpA2 3.62% 3.40% 0.22%
Type2A0 11.71% 17.11% -5.40%
Type2A1 5.95% 7.90% -1.95%
Type2A2 1.96% 2.65% -0.69%
Total 100% 100% 0%
Percent of All HS+ or HS- Applications Submitted and
Reviewed by CSR by Application Type
30%
25%
20%
PERCENT
15%
10%
HS+ New PI Type 1 A0
5% HS- New PI Type 1 A0
0%
10
01
05
10
01
05
10
01
05
10
01
05
00
01
01
01
02
02
02
03
03
03
04
04
20
20
20
20
20
20
20
20
20
20
20
20
COUNCIL ROUND DATE
Percent of All HS+ or HS- Applications Submitted
and Reviewed by CSR by Application Type
30%
25%
20%
PERCENT
15%
10%
HS+ New PI Type 1 A0
HS- New PI Type 1 A0
5%
HS+ Type 2 A0
HS- Type 2 A0
0%
0
1
5
0
1
5
0
1
5
0
1
5
1
0
0
1
0
0
1
0
0
1
0
0
00
01
01
01
02
02
02
03
03
03
04
04
20
20
20
20
20
20
20
20
20
20
20
20
COUNCIL ROUND DATE
All funded Type 1 R01, and R29 awards
between 1996-1999 reviewed by CSR,
and subsequent Type 2 applications
# with % with
Human T1 subsequent T2 subsequent T2
PI on T1 Subj in T1 awards application application
Established PI HS- 4,894 2,826 57.70%
HS+ 2,165 958 44.20%
New PI HS- 3,472 2,119 61.00%
HS+ 1,330 603 45.30%
Questions:
Why, on a percent basis, are HS+ PIs
more unlikely to apply for a type 2 A0
or A1 applications?
To what extent is the different
percentage of applications in each
category a factor in the reported
difference in the global review
outcomes?
Acknowledgements
Charles Dumais, CSR
Teresa Lindquist, CSR
Ted Kotchen, MCW and CSR
Ray Hoffman, MCW
The staff of OER/OD/NIH
Review Outcomes for R01 Human
Subjects Research at CSR
Michael R. Martin, Ph.D.
Director, Division of Physiology and Pathology
Center for Scientific Review
National Institutes of Health
January 23, 2006
