Title: Peer Review of a Stated Preference Survey Instrument Supporting Benefits Estimates for
the 316(b) Existing Facilities Rule
Estimated Period of Performance: The period of performance shall be for approximately 2
months from the Effective date of the Purchase Order.
Purchase Order Project Officer: TBD
U.S. EPA Office of Water
In January 1993, responding to recommendations in the report, “Safeguarding the Future:
Credible Science, Credible Decisions,” Administrator William Reilly issued an Agency-wide
policy for peer review. Administrator Carol Browner confirmed and reissued the policy on June
7, 1994, and instituted an Agency-wide implementation program. The principle underlying the
Peer Review Policy is that all major scientific and technical work products should be peer
Peer review is a process for enhancing a scientific or technical work document so that the
decision or position taken by the Agency, based on the technical document, has a sound, credible
basis. The goal of the Agency’s Peer Review Policy is to ensure that scientific and technical
work products receive appropriate levels of critical scrutiny from independent scientific and
technical experts as part of the overall decision making process.
In 2006, the Agency reaffirmed its commitment to quality decision‐making by updating the
Agency Peer Review Policy and issuing the third edition of the Peer Review Handbook
(http://www.epa.gov/peerreview/pdfs/peer_review_handbook_2006.pdf), which amplifies and
clarifies key Agency peer review issues. The Handbook states that, in general, all influential
scientific and technical work products related to Agency decisions should normally be peer
reviewed. Peer review is a documented, critical review of a work product performed by experts
who are independent of those who developed the product. Work products that are developed to
support regulations, such as risk assessments, should be evaluated to determine whether they
meet the criteria for “influential scientific and/or technical work products” and thus should be
subject to peer review.
Regulations themselves are not peer reviewed. The straight‐forward application of accepted,
previously peer‐reviewed economic methods or analyses in Regulatory Impact Analyses (RIAs),
are not typically subject to formal peer review. If, however, the particular facts and
circumstances of any piece of economic analysis in an RIA warrant peer review, the Agency
should accommodate those needs on a case‐by‐case basis.
The U.S. Environmental Protection Agency is developing regulations under section 316(b) of the
Clean Water Act. Section 316(b) requires that the location, design, construction, and capacity of
cooling water intake structures reflect the best technology available for minimizing adverse
environmental impact. The EPA estimates that more than 1,500 industrial facilities use large
volumes of water from lakes, rivers, estuaries, oceans, or other waters of the U.S. for cooling,
including steam electric power generators, pulp and paper makers, chemical manufacturers,
petroleum refiners, and manufacturers of primary metals like iron, steel, and aluminum.
Cooling water intake structures cause adverse environmental impact by pulling large numbers of
fish and shellfish, or their eggs, into cooling systems (entrainment). There, the organisms may
be killed or injured by heat, physical stress, or by chemicals used to clean the cooling system.
Larger organisms may be killed or injured when trapped against screens of an intake structure
EPA is developing Section 316(b) regulations. In December 2001, EPA published final
regulations addressing all new power plants and facilities that withdraw more than two million
gallons of water daily from waters of the U.S. (Phase I).
On February 16, 2004, EPA took final action on the Phase II rule governing cooling water intake
structures at existing facilities that are point sources; that, as their primary activity, both generate
and transmit electric power or generate electric power for sale to another entity for transmission;
that use or propose to use cooling water intake structures with a total design intake flow of 50
MGD or more to withdraw cooling water from waters of the United States; and that use at least
25 percent of the withdrawn water exclusively for cooling purposes. See 69 FR 41576 (July 9,
2004). Industry and environmental stakeholders challenged the Phase II regulations. On judicial
review, the Second Circuit (Riverkeeper, Inc. v. EPA, 475 F.3d 83, (2d Cir., 2007)) remanded
several provisions of the Phase II rule. Some key provisions remanded include: EPA improperly
used a cost-benefit analysis as a criteria for determining BTA, and EPA inappropriately used
ranges in setting performance expectations. In addition, the Second Circuit further ruled
restoration was not permissible as BTA, and that EPA’s cost-benefit site-specific compliance
alternative was also not in accord with the Clean Water Act. In response, EPA suspended the
Phase II regulation in July, 2007 pending further rulemaking. The US Supreme Court granted
certiorari to Entergy Corportation on cost/benefit decision made by the Second Circuit and on
April 1, 2009 the Court, in Entergy Corp. v. Riverkeeper Inc., decided that “EPA permissibly
relied on cost-benefit analysis in setting the national performance standards … as part of the
Phase II regulations.” EPA is now taking a voluntary remand of the rule, thus ending Second
In June of 2006, EPA promulgated the 316(b) Phase III Rule for existing manufacturers, small
flow power plants (facilities that use cooling water intake structures with a total design intake
flow of less than 50 MGD to withdraw cooling water from waters of the United States; and that
use at least 25 percent of the withdrawn water exclusively for cooling purposes), and new
offshore oil and gas facilities. Offshore oil and gas firms, and environmental groups petitioned
for judicial review, which was to occur in the Fifth Circuit. On July 23, 2010, the United States
Court of Appeals for the Fifth Circuit issued a favorable decision upholding EPA's CWA section
316(b) rule for new oil and gas facilities and accepted EPA's request to remand the existing
facility portion of the rule back to the Agency for further rulemaking. EPA has now voluntarily
remanded the existing facilities portion of the Phase III rulemaking, and combined the two
phases into one rulemaking covering all existing facilities.
EPA is develop revised regulations to provide national performance standards for controlling
impacts from all existing facilities with cooling water intake structures under Section 316(b) of
the Clean Water Act (CWA).
PURPOSE AND SCOPE OF WORK
The purpose of this Purchase Order is to conduct external, independent peer reviews that will
evaluate the stated preference survey instrument, sampling plan and proposed willingness-to-pay
estimation methodologies that are to be used in partial support of the Section 316(b) rulemaking.
This stated preference survey is designed to elicit survey respondents’ willingness-to-pay for
reducing I&E at cooling water intakes. Under the guise of this Purchase Order, peer reviewers
will evaluate the stated preference survey instrument, sampling plan and proposed willingness-
to-pay estimation methodologies. The reviewers will comment on the appropriateness of the
survey instrument for the Section 316(b) valuation of fish, eggs, and larvae, and ecosystem
health, the use of the choice experiment framework, the appropriateness of the sample plan,
scope tests, and bias tests, the statistical method for correcting for non-response bias, and the
willingness-to-pay estimation methodology.
STATEMENT OF WORK
The Contractor shall support US EPA in identify and select peer reviewers, conduct peer
reviews, and prepare peer review reports as specified in this statement of work. The Contractor
shall perform all activities under this Purchase Order in accordance with EPA Agency Peer
Review Policy procedures outlined in the following publication, Science Policy Council
Handbook - Peer Review 3rd Edition (EPA 100/B-06/002, January 2006,
http://www.epa.gov/peerreview/pdfs/peer_review_handbook_2006.pdf). For this Task, the
Contractor shall perform the following subtasks to complete peer review of the stated preference
survey instrument, sampling plan and proposed willingness-to-pay estimation methodologies.
Subtask 1: Prepare a Revised Work Plan
The Contractor shall prepare a revised detailed work plan that describes the approach to each of
the following subtasks, and includes a proposed schedule, staffing plan, and budget for the
overall Task Order. EPA has provided in this Purchase Order an estimate of the required level of
effort (LOE) for each task. The Contractor’s revised work plan shall include estimated LOE by
task. . The work plan also shall include procedures to be used for ensuring the absence of any
potential or real conflict of interest.
Subtask 2: Identify Peer Reviewer Pool
The Contractor shall identify a group of candidate peer reviewers and prepare and submit to the
EPA PO the credentials of five to seven nationally-recognized technical experts who are
qualified to independently peer review the stated preference survey instrument, sampling plan
and proposed willingness-to-pay estimation methodologies according to EPA peer review
guidelines. The potential pool of peer reviewers shall include experts in environmental and
natural resource economics, and econometric analysis, with specific experience in the estimation
of non-use, fisheries, and/or recreation benefits. The peer review panel shall not include any
experts that directly or indirectly (by consultation) contributed to EPA’s rulemaking for section
316(b), nor those experts advising other parties in this matter or associated with any of the data
used for various case studies.
The EPA will review and approve the potential pool of peer reviewers based on credentials and
expertise to fulfill the role of peer reviewers of EPA technical documents. The EPA may reject
the use of any particular candidate based on qualifications, conflicts of interest, or past direct
involvement with the work under review. In such cases, the Contractor shall find suitable
replacements to bring the pool back to five to seven potential peer reviewers with the expertise
from the fields previously stated. The EPA shall review and consent to the qualifications of the
pool of peer reviewer candidates, but the ultimate selection of peer reviewers from the approved
pool shall be the responsibility of the Contractor.
Subtask 3: Select Peer Reviewers and Arrange for Peer Review
Following review and approval by the PO of the peer reviewer pool, the Contractor shall select
five peer reviewers from the peer review panel of the five to seven candidates approved by the
EPA PO, determine their availability to perform the review, and initiate the peer review effort.
The Contractor shall distribute the stated preference survey instrument, sampling plan and
proposed willingness-to-pay estimation methodologies, background materials on the 316(b) rule,
and the Charge to Peer Reviewers (provided to the Contractor by EPA, see Government
Responsibilities below) to each selected peer reviewer. The Contractor also shall provide to the
peer reviewers any supplemental information requested by the reviewers and deemed necessary
by the EPA PO to complete a thorough review.
Subtask 4: Conduct Peer Review
The Contractor shall coordinate with the peer reviewers and monitor their progress to complete
the review within the required time and LOE constraints described in this Purchase Order. Peer
reviewers shall conduct their review according to the guidelines detailed in the charge to peer
reviewers. Peer reviewers shall respond to the specific question asked in the charge or briefly
indicate why the question cannot or should not be addressed. It is not necessary that the peer
reviewers jointly reach consensus on their findings and recommendations. It is expected that no
single peer reviewer would expend more than 60 hours performing the peer review; expending
less than 20 hours is not acceptable.
Subtask 5: Prepare Peer Review Report
Upon obtaining comments from the peer reviewers, the Contractor shall assemble and provide to
the EPA all unedited peer review comments and any additional materials submitted by peer
reviewers. A summary of peer review comments is required. If the EPA PO makes a request for
clarification of any peer review comment upon receipt, the Contractor shall contact the peer
reviewer and obtain the needed clarification.
Upon the written request by the EPA PO, the Contractor may be required to present the peer
review findings to any of the following: EPA technical expert panel, EPA managers, non-EPA
or other interested stakeholders identified by EPA PO (e.g., EPA sponsored meeting).
The Contractor shall provide the following deliverables to the EPA PO. All text deliverables
shall be compatible with Word 2007 and provided in electronic format (i.e., e-mail) to the PO.
Two hard copies and one electronic copy on CD or DVD of all deliverables shall be submitted to
EPA. Two hard copies of any handwritten comments, corrections, or edits provided by the
individual peer reviewers on the original materials provided by EPA should be provided to the
EPA PO. Note that EPA anticipates that subtask 1 of this Purchase Order can be initiated
immediately. Subtask 2 will be initiated once EPA provides the peer review charge information
and example peer reviewer list.
Subtask 1 Revised work plan: The Contractor shall submit a work plan as described in Subtask
1 of the SOW within 15 calendar days after receipt of the Purchase Order.
Subtask 2 Identify peer reviewer pool: The Contractor shall submit a list and credentials for five
to seven potential peer reviewers within one week after Contractor receipt of the
survey and supporting material and charge peer review documents.
Subtask 3 Select peer reviewers and arrange for peer review: The Contractor shall select the five
peer reviewers and begin peer review within one week after EPA approval of the peer
Subtask 4 Conduct peer review: The Contractor shall complete peer review within one month of
the peer reviewers receipt of the documents.
Subtask 5A Prepare DRAFT peer review report: The Contractor shall complete and submit to
the EPA PO the peer review report within one week after receiving comments from
Subtask 5B Prepare FINAL peer review report: The Contractor shall complete and submit to
the EPA PO the peer review report within one week after receiving comments from
CONFLICT OF INTEREST
The contractor shall ensure that individual reviewers have disclosed any actual or potential
conflict of interest and have signed and submitted a Conflict of Interest Form. The contractor
must ensure that none of the conflicts disclosed are direct and substantial as to rule out a
particular reviewer. The Contractor must also adhere to the following requirements regarding
conflict of interest.
1) Upon receipt of the Purchase Order, and prior to commencement of any work, the
Contractor shall notify both the EPA Contracting Officer (CO) and Project Officer (PO) of
any actual or potential organizational or personal conflicts of interest. The Contractor shall
not conduct peer review of documents prepared by the Contractor, its subcontractors, or
2) The Contractor shall provide a written certification, within 15 days of receipt of a Purchase
Order, or similar tasking document, that:
a) Either all actual or potential organizational conflicts of interest have been reported to
the EPA CO or that no actual or potential organizational conflicts of interest exist. The
Contractor is directed to assure that none of the conflicts disclosed are so direct and
substantial as to rule out a particular reviewer.
b) All personnel who perform work under this Purchase Order or relating to this Purchase
Order have been informed of their obligation to report personal and organizational
conflicts of interest to the EPA CO.
c) The Contractor recognizes its continuing obligation to identify and report any actual or
potential conflicts of interest arising during performance of this Purchase Order.
3) If an actual or potential organizational conflict of interest is identified during performance
under this Purchase Order, the Contractor shall immediately make a full disclosure in
writing to the EPA CO. The disclosure shall include a description of action which the
Contractor has taken or proposes to take, after consultation with the EPA CO, to avoid,
mitigate, or neutralize the actual or potential conflict of interest.
No travel is anticipated under this Purchase Order. Any travel directly chargeable to this
Purchase Order must be submitted by written or e-mail request to and approved by the EPA PO
prior to undertaking any travel.
The PO will provide to the Contractor the stated preference survey instrument, sampling plan
and proposed willingness-to-pay estimation methodologies, background information on 316(b)
rule, peer review charge questions, information on the results of the previous peer review report
that was conducted on an earlier version of the survey, and any other necessary information or
documents required by the Contractor to perform the Purchase Order.