COV BES Response 2008 - BES 2006 Annual Update to the 2005 COV

Document Sample
COV BES Response 2008 - BES 2006 Annual Update to the 2005 COV Powered By Docstoc
					                                     BES/CBET 2008 Annual Update to the 2005 COV Report

         Comment in COV Report                            2007 BES Response                         2008 BES/CBET Response
QUALITY AND EFFECTIVENESS OF                  CBET has provided online CBET panelist        In addition to the 2007 update, in 2008,
MERIT REVIEW PROCEDURES                       guide for review prior to the panel,          ENG’s Program Director for Diversity and
A.1.3. While reviews are clearly focused      outlining the broader impacts criterion and   Outreach developed a set of viewgraphs
on intellectual merit first and broader       detailing a sample panel summary              aimed at minimizing biases or “schemas”
impact secondarily, awards and feedback       template. Also, program officers who          in proposal evaluation. This was tested in
are generally consistent with stated          serve as panel moderators provide a pre-      the BRIGE proposal review with positive
program criteria. However, the COV felt       panel briefing covering the various merit     feedback. It is available for future use by
that continued improvement in attention to    criteria.                                     POs.
broader impact should be encouraged. In
addition, the COV recommends that BES
provide more guidance for the panel
members about the significance of the
broader impact criterion.

A.1.4. The COV feels that panelists must      BES Program Officers that serve as panel      We continue to emphasize this; no
document more explicitly in the summary       moderators have been requiring that each      significant additional comments beyond
portion of their individual reviews how       panelist revise (if necessary) their review   2007.
each arrives at their overall rating. While   comments to ensure that PIs receive
the majority of reviewers adequately          constructive feedback on declined
comment on the intellectual merit of each     proposals, and that the written comments
proposal, the COV suggests that the panel     are consistent with the overall rating.
moderators ensure that each panelist          Written comments include strengths and
revises (if necessary) their review           weaknesses for each separate category.
comments to ensure the PIs receive
constructive feedback on declined
proposals. The written comments should
be consistent with the overall rating.
A.1.5. While the summaries frequently          BES Program Officers that serve as panel       . We continue to emphasize this; no
reflect some aspects of the panel              moderators have been reviewing panel           significant additional comments beyond
discussion, they do not always fully           summaries more critically suggesting           2007.
address the evaluation criteria. The COV       improvements as appropriate and
recommends that panel moderators review        determining that the panel summary
the summaries and suggest improvements         reflects resolution of divergent
when the summaries are lacking. When           evaluations.
the reviewers’ evaluations are widely
divergent, the panel summary should
reflect the resolution reached during the
panel’s discussion.

A.1.6-comment #1. The COV found that           Jackets for proposals withdrawn due to         We continue to emphasize this; no
some proposals withdrawn due to funding        funding by other agencies must always          significant additional comments beyond
by other agencies did not contain a            contain a context statement summarizing        2007.
summary statement of the review process.       the review process.
The COV recommends that proposals
withdrawn due to funding by other
agencies must still contain a context
statement summarizing the review process.
A.1.6-comment #2. If a program officer’s       If a Program Officer’s funding deviates        We continue to emphasize this; no
funding deviates from the panel’s              from the panel’s recommendation, the           significant additional comments beyond
recommendation, this should be clearly         rationale is being documented in a diary       2007.
documented. The COV suggests adding a          note.
diary note when the rationale is based on
information that is not intended for the PI.
QUESTIONS CONCERNING THE                       CBET’s efforts to recruit industrial           We continue to emphasize this; no
SELECTION OF REVIEWERS                         panelists are being increased, particularly    significant additional comments beyond
A.3.2. A highly qualified group of             SBIR awardees to serve as panelists.           2007.
individuals representing a broad and           Junior faculty are included in all panels as
appropriate range of expertise was             a training device.
recruited to serve as reviewers. Reviewers
included a mix of senior and junior faculty
and representative from industry, though
industrial representation could be
improved. Increase efforts to recruit
professionals from industry are
encouraged; recipients of SBIRs may be a
promising source. The inclusion of junior
faculty as reviewers serves an important
training role and should be continued.
A.3.3. The geographical and institutional      CBET will continue and intensify its           We continue to emphasize this; bio and
distribution among reviewers was strong        efforts to include underrepresented            environmental programs have been
with nearly every state and the District of    minorities and women as reviewers. With        particularly effective in recruiting female
Columbia represented as well as the full       time, graduates funded by NSF grants may       reviewers and panelists.
range of institutional types. While the        be a new source for such reviewers.
representation from members of                 Additionally, CBET has added an on-line
underrepresented groups and women was          tool for possible reviewers to submit their
in line with distributions within the          interest, qualifications and specialty areas
population and within academe, the             through the CBET Reviewer Database.
Program (Division) is encouraged to
continue and intensify its strong efforts to
include underrepresented minorities and
women as reviewers.
QUESTIONS CONCERNING THE                       CBET agrees with the COV, and regrets          We continue to emphasize this; no
RESULTING PORTFOLIO OF                         that CBET’s level of resources is much         significant additional comments beyond
AWARDS UNDER REVIEW                            more constrained than NIH counterparts.        2007.
A.4.2. The average BES award size is
essentially the same dollar amount,            (During the period covered by the COV
$120,998, as the average ENG award of          review, NSF’s goal for award duration was
$119,837, and is appropriate given the         3 years.)
scope of the project proposed. However,
BES-supported research generally has a
higher burden of laboratory /experimental
costs that are not reflected in the typical
award. In fact, BES awards are
considerably smaller and shorter (e.g., 3
yrs vs. 4-5 yrs) than similar awards by
other organizations (e.g., NIH), resulting in
relatively limited scope projects relative to
those funded by other organizations.
A.5.4. BES success rate for 2004 was 13%;       CBET was formed as a merger between        Success rate Continues to hover around
average ENG average was 15%. These              CTS and BES on October 1, 2006. Success    13%. This has been a trend in recent years
success rates are too low—management            rate for 2007 was lower than the ENG       and may reflect the fact that we have
should seek ways to improve this. One           average but efforts continuing to try to   provided additional opportunities (such as
possibility might be to explore mergers and     improve this.                              BRIGE, Nano, CDI, etc.) for PIs to submit
consolidations both within ENG and with                                                    proposals in addition to the twice-a-year
units of other directorates.                                                               unsolicited proposal windows. In some
                                                                                           ways, this takes away valuable time for PIs
                                                                                           to write proposals; on the other hand, this
                                                                                           may also give them the opportunities to
                                                                                           polish their ideas whether they end up
                                                                                           being funded by NSF.
OUTCOME GOAL FOR                                Dr Judy Raper is the CBET Division         Dr. Maria Burka has been acting
ORGANIZATIONAL EXCELLENCE                       Director.                                  DD/CBET since Dr. Raper left in July.
B.4-comment #1. The COV strongly                                                           The new permanent DD will begin in
recommends that BES hire a permanent                                                       October 1, 2008..
Division Director for sustained
management and leadership.

B.4-comment #2. The success rate for            The Division Plan for CBET outlines nine   Nothing beyond 2007
proposals in BES is low and the budget is       strategic goals for discovery, learning,
not projected to increase. Given the            infrastructure, and stewardship.
current budget climate, it is recommended
that BES set focused priority research and
educational areas. The current draft
strategic plan lays out goals and strategies
for the Division in a very broad sense and
lists activities that have already been
implemented to reach the goals. The COV
recommends that a strategic
implementation plan be developed with
future goals that are focused and specific
and a map with out-year budget projections
for the various programs.

B.4-comment #3. A program evaluation           CBET evaluates each program annually         Nothing beyond 2007
process should be put in place that feeds      with respect to meeting strategic objectives
into the strategic implementation plan         during the planning process.
B.4-comment #4. There should be a clear        CBET program officers use WTEC studies Nothing beyond 2007
relationship between the numerous WTEC         to inform decisions on the relative funding
study outcomes and the program priorities      between sub-areas in a program.
and program announcements in the
B.4-comment #5. If possible, BES should        The nanotechnology announcement has         The nanotechnology initiative has been
be more pro-active with the                    been archived. CBET funds                   subsumed into the individual programs.
nanotechnology announcement so the             nanotechnology proposals through its core
funded proposals have a high relevance to      programs.
the Division.
B.5-comment #6. The projected future           CLEANER funding has been transferred to     WATERS is now a partnership of ENG
costs of CLEANER are very large. The           WATERS which is leading to an MREFC         (not just CBET), GEO, and SBE. We had a
Division should seek partnerships with         application. WATERS is a 50/50              major joint workshop with EPA in 2008;
other federal agencies such as NIH             partnership with GEO. Other agencies        and an NSF/EPA MOU is presently being
(NIEHS), EPA, and/or Dept. of Homeland         have been approached to participate with    negotiated. Likewise, an MOU with
Security to help leverage NSF funds.   discussions ongoing.   USDA is in preparation. Early negotiations
                                                              are also underway with the Army Corps of
                                                              Engineers and the USGS.

Shared By:
Lingjuan Ma Lingjuan Ma