mom

Document Sample
mom Powered By Docstoc
					User’s Manual for the Chesapeake Bay Program’s
Phase 5 Watershed Model
1   Introduction ................................................................................................................. 4
 1.1      Environmental Issues the Chesapeake Bay Decision Support System is
 Designed to Address ....................................................................................................... 4
 1.2      Watershed Model in the Context of Chesapeake Bay Program Models ............. 4
2 Concepts that should be understood before using the CBP Phase 5 watershed model5
 2.1      HSPF ................................................................................................................... 5
 2.2      Changes from the standard methods of running HSPF....................................... 6
    2.2.1       Multiple UCIs ............................................................................................. 6
    2.2.2       Masslinks and schematic............................................................................. 6
    2.2.3       Standard hspf code ...................................................................................... 6
    2.2.4       Point and click............................................................................................. 7
    2.2.5       Segmentation scheme.................................................................................. 7
       2.2.5.1 Land Segmentation nomenclature........................................................... 7
       2.2.5.2 River Segmentation nomenclature .......................................................... 7
 2.3      Watershed Model Modes .................................................................................... 9
    2.3.1       Scenario Mode ............................................................................................ 9
    2.3.2       Calibration Mode ...................................................................................... 10
    2.3.3       Running the Model ................................................................................... 10
 2.4      What is a Scenario in the context of the watershed model ............................... 12
    2.4.1       Land Scenario ........................................................................................... 12
    2.4.2       River Scenario ........................................................................................... 12
 2.5      Scripts ............................................................................................................... 13
 2.6      Directory Structure............................................................................................ 13
    2.6.1       top level ..................................................................................................... 14
    2.6.2       Using the directory structure ..................................................................... 14
 2.7      Code Structure .................................................................................................. 14
    2.7.1       General coding style ................................................................................. 15
    2.7.2       Lib ............................................................................................................. 15
    2.7.3       Hspf ........................................................................................................... 15
3 Quick Start guide ...................................................................................................... 15
 3.1      Download .......................................................................................................... 15
 3.2      Run a small basin with a single command ........................................................ 16
 3.3      rtfm .................................................................................................................... 16
4 Detailed directory Structure ...................................................................................... 16
 4.1      top level ............................................................................................................. 17
 4.2      code ................................................................................................................... 17
 4.3      config ................................................................................................................ 18
 4.4      documentation ................................................................................................... 19
 4.5      input .................................................................................................................. 19
 4.6      output ................................................................................................................ 21
 4.7      run ..................................................................................................................... 22
 4.8      sumout ............................................................................................................... 23
    4.9      tmp .................................................................................................................... 23
5      Details of the scenario control files .......................................................................... 23
       5.1.1       Notes on control files. ............................................................................... 23
       5.1.2       Land Control File ...................................................................................... 25
       5.1.3       River Control File ..................................................................................... 28
6      Details of running the model using scripts ............................................................... 30
    6.1      Pay attention to the error messages ................................................................... 31
    6.2      General structure of the scripts ......................................................................... 31
       6.2.1       Help with invocation ................................................................................. 31
       6.2.2       Set local variables and create temporary directory ................................... 31
       6.2.3       Body with loop and error messages .......................................................... 32
       6.2.4       Clean up .................................................................................................... 33
    6.3      Standard arguments to scripts ........................................................................... 33
    6.4      Script Directories .............................................................................................. 33
       6.4.1       Standard .................................................................................................... 33
       6.4.2       Useful ........................................................................................................ 36
       6.4.3       Datascripts................................................................................................. 37
          6.4.3.1 precip_and_met directory ..................................................................... 39
          6.4.3.2 add_atdep_to_precip_wdms directory .................................................. 39
          6.4.3.3 diversions directory............................................................................... 40
          6.4.3.4 point_source directory .......................................................................... 40
          6.4.3.5 septic_directory ..................................................................................... 40
          6.4.3.6 observed ................................................................................................ 40
       6.4.4       make_seglists ............................................................................................ 41
       6.4.5       calibration ................................................................................................. 42
       6.4.6       fragments................................................................................................... 42
       6.4.7       link_p5_to_wqm ....................................................................................... 42
       6.4.8       run24 ......................................................................................................... 43
7      Input Data Organization ............................................................................................ 43
    7.1      calib ................................................................................................................... 44
    7.2      param................................................................................................................. 44
    7.3      Unformatted ...................................................................................................... 45
    7.4      Scenario............................................................................................................. 45
       7.4.1       Climate scenario data directory ................................................................ 45
       7.4.2       Land scenario data directory ..................................................................... 46
       7.4.3       River scenario data directory .................................................................... 48
          7.4.3.1 ASCII river scenario data...................................................................... 48
          7.4.3.2 WDM river scenario data ...................................................................... 49
          7.4.3.3 bmptypes directory................................................................................ 49
       7.4.4       Wqm scenario data directory .................................................................... 52
8      Detailed documentation of the config directory ....................................................... 52
    8.1      Working with basins ......................................................................................... 52
    8.2      Control .............................................................................................................. 53
    8.3      Catalog .............................................................................................................. 53
       8.3.1       Geo ............................................................................................................ 53
       8.3.2       iovars ......................................................................................................... 54



Modified 11/7/2010                                                                                                                     2
        8.3.2.1 Files containing variable definitions ..................................................... 55
        8.3.2.2 Linkages between variables .................................................................. 56
        8.3.2.3 Special files ........................................................................................... 58
     8.3.3       modules ..................................................................................................... 59
  8.4      Blank_wdm ....................................................................................................... 60
9 Code documentation ................................................................................................. 60
  9.1      Lib ..................................................................................................................... 61
  9.2      Lug .................................................................................................................... 61
  9.3      Rug .................................................................................................................... 61
  9.4      Etm .................................................................................................................... 61
  9.5      Postproc............................................................................................................. 61
  9.6      Wqm.................................................................................................................. 61
10      A detailed walk through a scenario. ...................................................................... 61
  10.1 Create the new ASCII data sets ........................................................................ 61
  10.2 Create any new wdms that will be necessary.................................................... 61
  10.3 Copy old river, and probably land, scenario control files and modify them .... 61
  10.4 Create the directories for running the scenario ................................................. 62
  10.5 Run the lug ........................................................................................................ 62
  10.6 Run the land ...................................................................................................... 65
  10.7 Run the etm and postprocessor ......................................................................... 66
  10.8 Run the scenario_river ...................................................................................... 67
  10.9 Run the remaining postprocessing steps ........................................................... 69
  10.10      Create the Estuarine model input .................................................................. 69
11      A detailed walk through the calibration ................................................................ 69
  11.1 PWATER .......................................................................................................... 69
  11.2 SEDMNT .......................................................................................................... 69
  11.3 SOLIDS............................................................................................................. 69
  11.4 PSTEMP ........................................................................................................... 69
  11.5 PQUAL ............................................................................................................. 69
  11.6 IQUAL .............................................................................................................. 69
  11.7 AGCHEM ......................................................................................................... 69
  11.8 WQ .................................................................................................................... 69
  11.9 Transport ........................................................................................................... 70
12      Parallel operations ................................................................................................. 70
13      Cool things to be done in the future ...................................................................... 70
  13.1 Parallelize in MPI ............................................................................................. 70
  13.2 Improve calibration ........................................................................................... 70
  13.3 Work in smaller basins...................................................................................... 70
  13.4 New constituents ............................................................................................... 70
  13.5 Uncertainty analysis .......................................................................................... 70
14      References ............................................................................................................. 70




Modified 11/7/2010                                                                                                                   3
1 Introduction

1.1 Environmental Issues the Chesapeake Bay Decision
    Support System is Designed to Address

The Chesapeake Bay—the largest estuary in the United States—is an incredibly complex
ecosystem that includes important habitats and food webs. The Bay and its rivers,
wetlands and forests provide homes, food and protection for complex groups of animals
and plants. Fish of all types and sizes either live in the Bay and its tributaries or use its
waters as they migrate along the East Coast. Most of the Bay‘s waters are degraded. In
2007, 79 percent of the Bay did not meet water quality goals (EPA 2008). The water
quality goals are defined by the healthy levels of dissolved oxygen, chlorophyll a, and
water clarity. All three of these health indicators are dependent on the levels of nutrients
and sediment delivered from the Bay watershed. The Chesapeake Bay watershed
stretches across more than 64,000 square miles, encompassing parts of six states —
Delaware, Maryland, New York, Pennsylvania, Virginia and West Virginia — and the
entire District of Columbia.

The combination of a watershed model, predicting change in nutrient and sediment loads
due to management actions, with an estuarine model, predicting change in water quality
parameters due to changes in loads, has been used by the Chesapeake Bay Program
partnership since the late 1980s to set voluntary cleanup goals. This combination is now
being used to create the Chesapeake TMDL.

The WSM predicts flow, sediment, nitrogen, and phosphorus, including separate species
of nitrogen and phosphorus, and changes in these outputs given changes in the inputs
describing watershed management. It is designed to allow for sensitivity to nutrient
inputs to the land, such as atmospheric deposition, fertilizer, and manure, and also to
structural BMPs that can be input as a percentage reduction.

1.2 Watershed Model in the Context of Chesapeake Bay
    Program Models

Figure 1: Chesapeake Bay Program Decision Support System




Modified 11/7/2010                                                                          4
The Chesapeake Bay Program Decision Support System (figure 1) is comprised of
several models. The overall question that this system is designed to address is: How will
the Chesapeake Bay water quality or living resources respond to changes in management
actions? A given set of management actions, represented in figure 1 by the signatures of
the Chesapeake Bay program partners, is interpreted by several models to produce input
to the watershed model. The CBP land use change model, a combined application of
SLEUTH and GAMe, predicts changes in land use, sewerage, and septic systems given
changes in land use policy (reference). The airshed model, a national application of
CMAQ, predicts changes in deposition of inorganic nitrogen due to changes in emissions
(reference). The Scenario Builder software combines the output of these models with
other data sources, such as the US Census of Agriculture, to generate inputs to the
watershed model. The watershed model predicts the loads of nitrogen, phosphorus, and
sediment that results from the given inputs and feeds them to the estuarine water quality
and sediment transport model. The estuarine water quality and sediment transport model
predicts the changes in water quality due to the changes in input loads. An empirical
watershed model, SpaRRoW (reference) is used to help target restoration activities.

2 Concepts that should be understood before using the
  CBP Phase 5 watershed model
The CBP phase 5 watershed model is a specific application of HSPF. The watershed
model code is comprised of 2 parts: HSPF and the surrounding code that handles input,
output, and calibration. This manual primarily covers the surrounding software, or the
phase 5 modeling system.

2.1 HSPF

HSPF, Hydrologic Simulation Program – Fortran, is a deterministic, dynamic watershed
simulation code. The requirements are extensive and complex for both parameterization


Modified 11/7/2010                                                                       5
and input data. There are many available references on HSPF. Documentation and code
can be downloaded from this site: http://water.usgs.gov/software/HSPF/
Other references are associated with the graphical modeling system Basins, available
from this site: http://www.epa.gov/waterscience/basins/


2.2 Changes from the standard methods of running HSPF
The CBP implementation of HSPF is different in many ways from the typical
implementation of HSPF, either though a standard text-based linux or dos
implementation, or through the graphic interface of WinHSPF or BASINS. From the
author‘s experience, users with general modeling experience are more likely to quickly
understand the application than those with years of specific HSPF modeling. The
purpose of this section is to dispense with the pre-conceptions that may hinder
understanding of the simulation methods.

2.2.1 Multiple UCIs
Many applications of HSPF, including those in BASINS are carried out within a single
UCI (User Controlled Input) file. To allow for maximum flexibility, each land use in
each land segment is simulated in a separate UCI and each river segment is simulated in a
separate UCI. That means for the 308 land segments in the phase 5 domain and 24 land
uses used in the phase 5 application, there are 7392 separate land UCIs as well as 738
river segments for every scenario or calibration run.

2.2.2 Masslinks and schematic
In most applications of HSPF, the translation between land and river simulations is made
in the MASSLINKS section, which matches land variables to river variables, and the
SCHEMATIC section, which matches land segments to river segments and multiplies by
the area. Neither section is used in the Phase 5 application. The functionality of these
two sections has been incorporated and expanded in intermediate software, the external
transfer module (etm). The etm also incorporates time series of land use and
management practices and can apply programming logic to any of these, such as to
simulate the effect of a large storm on BMP effectiveness.

2.2.3 Standard hspf code
HSPF used in the phase5 application is a modified version 11 HSPF. The current
windows version, HSPF 12 has not been ported to linux. Bug fixes that were
incorporated into HSPF 12 have been incorporated into the CBP HSPF 11 as well. In
addition there are differences in the output to the screen. There are two functional
differences in the method of solution/particulate portioning. For Freundlich isotherms,
which are solved numerically, a secondary Newton‘s method solver was added that only
applies then the standard method does not reach a solution. Instantaneous partitioning
coefficients were modified to pertain to concentration, not mass, which resulted in a more
realistic simulation of output concentration during very dry conditions.




Modified 11/7/2010                                                                       6
2.2.4 Point and click
Users who have used WinHSPF or BASINS may be used to a system where they can edit
individual tables in UCIs through a GUI or press an icon to run the model or otherwise
manipulate data. Phase 5 is a linux-based system that uses fortran and shell-script
programming to quickly perform hundreds to millions of operations using information
stored in comma-delimited files.

A typical operation using the phase 5 system would involve replacing data files, such as
for land use in the entire watershed, then using scripts to create the UCIs, run HSPF, and
summarize output, rather than manually editing the land use within a UCI as may be done
for a typical HSPF application.

2.2.5 Segmentation scheme
In many HSPF applications, the river segmentation and the land segmentation is the
same. Each river segment will have a set of land uses which drain to it and it only. In the
phase 5 system, the segmentation schemes are separate. Land segmentation is generally
county-based. There is a simulation of a representative acre of each land use type in each
county. Some counties in mountainous regions where the rainfall patterns varied
significantly have been broken out in to several land segments. River segments are, of
course, watersheds. The segments that result from the intersection of these two
segmentation schemes are known as land-river segments. For more information on the
generation of these segmentation schemes, see USGS 2005.


2.2.5.1 Land Segmentation nomenclature

Land segments names are the 5-digit FIPS code for the county prefixed by an A, B, or C.
Most counties are not subdivided and so use an A for the prefix. In subdivided counties,
the designation of sections A, B, or C are arbitrary. For example, Anne Arundel County,
MD has the land segment name A24003.

2.2.5.2 River Segmentation nomenclature

River segment names are 13-characters with three major parts.
The first three characters are a description, Major basin, Minor basin, and an index to the
average flow volume in the river.

The fourth character is an underscore

Characters 5-8 are a 4 digit unique identifier, assigned generally north-south.

The Ninth character is an underscore

Characters 10-13 are the 5 digit unique ID of the next downstream river.




Modified 11/7/2010                                                                        7
Example: YP4_6720_6750 is the York, Pamunkey with an average flow rate between x
and y. The unique ID is 6720 and it drains to the segment with the unique ID of 6750.

Special cases for the downstream ID
        0001 – simulated river that drains to tidal Chesapeake waters
        0000 – segment with no simulated river. Land use simulations are considered to
drain directly to the tidal Chesapeake
        0004 – segment that does not drain to any downstream water, examples are a
combined sewer system where we have the output data or a reservoir that does not have
any drainage.
        0003 – not a physical segment, but a calculated water quality point to match
against a river gage. River gages just downstream of a confluence have the upstream
simulations combined, then output as a river segment with the unique ID of the
downstream segment and a downstream ID of 0003

Major Basins:
E     Eastern Shore
J     James
P     Potomac
R     Rappahannock
S     Susquehanna
W     Western Shore
X     Patuxent
Y     York

Major and Minor Basins:
AbbrevBasin
BS     Big Sandy
DE     Delaware, Outside of CB watershed
EL     Eastern Shore, Lower
EM     Eastern Shore, Middle
EU     Eastern Shore, Upper
GY     Youghiogheny
JA     James, Appomattox
JB     James, Below Richmond
JL     James, Lower
JU     James, Upper
KU     Yadkin
MN Meherin and Nottoway
NR     New River
OD     Roanoke River, Dan River
OR     Roanoke River, other than Dan River
PL     Potomac River, below DC
PM     Potomac River, Middle
PS     Potomac Shenandoah
PU     Potomac Upper



Modified 11/7/2010                                                                       8
RL     Rappahannock, Below Fredericksburg
RU     Rappahannock, above Fredericksburg
SJ     Susquehanna, Juniata
SL     Susquehanna, Lower
SU     Susquehanna, Upper
SW     Susquehanna, West Branch
TU     Tennessee, Upper
WL     Western Shore, Lower
WM     Western Shore, Middle
WU     Western Shore, Upper
XL     Patuxent, Below Bowie
XU     Patuxent, Below Bowie
YL     York, Lower
YM     York, Mattaponi
YP     York, Pamunkey
ZZ     Non-simulated area


2.3 Watershed Model Modes
The phase 5 watershed model can either be run in scenario mode or calibration mode.
The community version of the watershed model is calibrated so that the user can run
scenarios without recalibrating as long as the same input data and segmentation are used.

2.3.1 Scenario Mode




Figure 2: Operations in Scenario Mode
In scenario mode, the user uses the calibrated model to ask what-if questions. The user
interacts with the scenario builder (documented separately) to produce or modify inputs
for land use, point source, septic loads, agricultural parameters, or other inputs. The user
then runs the model which returns summarized outputs.

It is possible to vary inputs over time during scenario mode, but for most scenarios, the
user is interested in the long-term loads for a given state of management. For example,
the user may select the projected year 2020 land use and a particular combination of


Modified 11/7/2010                                                                          9
management practices that may occur to test the effectiveness of this policy. The model
is then run for the years 1991-2000 and the output is divided by 10 to get the average
annual output given those years. The user can select other years as averaging period if
desired.

2.3.2 Calibration Mode
The phase 5 model as downloaded is calibrated for use in the 2011 Chesapeake Bay
TMDL. The user may wish to recalibrate for a number of reasons, such as: the testing of
a new input data source to see if there is an improvement in the calibration; the testing of
a new calibration methodology; or the application to a smaller model domain with or
without changes in segmentation.




Figure 3: Calibration mode work flow

During calibration mode the model is run iteratively. After each iteration, outputs are
compared to observations or other calibration data to produce calibration metrics. The
parameters are updated automatically by the calibration software until a maximum
iteration is reached or the solution converges.

To control this process, the user interacts with the scenario builder to select or modify
inputs to be used for calibration. The user also interacts with the calibration
methodology. This interaction can be as simple as the modification of parameter limits
or as complex as rewriting the source code.

2.3.3 Running the Model
As noted earlier, the phase 5 model is run with software that automatically generates
UCIs and transfers data between the land and river simulation. This is illustrated in
figure 4:




Modified 11/7/2010                                                                          10
Figure 4: Phase 5 model software major processes
In the above figure, arrows represent the creation or modification of files by software, the
cartoons represent UCIs, and WDMs are the binary format used by HSPF for storage of
time series data.
The software steps are as follows:
    1. The Land UCI Generator (lug) produces a UCI for each land use and land
        segment using stored parameters and input data. The user can specify the
        geographic area and the particular data set to use.
    2. HSPF is run on the land UCIs to generate a separate WDM for each land use and
        land segment. These WDMs contain hourly time series for flow, sediment, and
        species of nutrient from the different soil layers simulated in HSPF. The outputs
        are on a per-acre basis.
    3. The External Transfer Module (etm) runs separately for each river segment being
        simulated in a particular run. It gathers information on acreage of each land use
        and land segment within each river segment and the management practices which
        alter the time series. It translates and aggregates land time series into river input
        time series and stores the individual time series for each constituent in a WDM.
        Each river segment has a separate WDM. The user can specify the geographic
        area, and the land use and management practice data set to use.
    4. The River UCI Generator (rug) produces a UCI for each river segment using
        stored parameters. The user can specify the geographic area and parameters to
        use.
    5. Upstream river loads are added to the WDM for each river, which has been
        loaded with the local land-based loads by the etm. HSPF is run on the river UCIs
        and the results are written back to the same WDM, preserving the input data as
        well
    6. A system of postprocessors generates text output from the land and river WDMs.
        The user can configure the postprocessor to get output on various time and spatial
        scales or to get comparisons of simulated and observed data.




Modified 11/7/2010                                                                        11
2.4 What is a Scenario in the context of the watershed model
Scenarios are the specifications of inputs and outputs for a given model run. A land
scenario is the specifications for running a land simulation. A river scenario is the
specification for running the river. They may often be the same scenario name for a
given run, but it is possible and sometimes efficient, to use different river and land
scenario names or several different land scenarios within the same river scenario.

For example, if you are only changing land use acreage, it would not be necessary to re-
run the per-acre land use simulation, you would just run a new river scenario with a
different land use data set. Likewise, if a scenario only had a difference in the application
to few crop land uses, you could use the forest and urban simulations from a previously-
run scenario.

There are also calibration scenarios, which are specifications of how the user wants the
software to perform the calibration. These are covered under the calibration
documentation.

2.4.1 Land Scenario
―Land Scenario‖ refers to a collection of specifications for land simulation parameters
and inputs.

From the standpoint of the phase 5 software and file organization, the land scenario name
is exactly two things:
1. The name of the control file containing the specifications.
        (examples under: ./config/control/land/)
2. The name of the directories containing the output of programs referencing this file.

A land scenario control file specifies the inputs to the land simulation and the outputs
expected from it. Examples of inputs are precipitation, meteorology, fertilizer, and
manure.

2.4.2 River Scenario

―River Scenario‖ refers to a collection of specifications for river simulation parameters,
inputs, and the land scenarios that go into the river.

From the standpoint of the phase 5 software and file organization, the river scenario name
is exactly two things:
1. The name of the control file containing the specifications
        (examples under: ./config/control/river/)
2. The name of the directories containing the output of programs referencing this file.

A river scenario specifies the inputs to the river simulation and the outputs expected form
it. Examples of inputs are land use, bmp specifications, and the land scenario(s) used.




Modified 11/7/2010                                                                         12
2.5 Scripts
Almost all operations for the watershed model are accomplished with the use of c-shell
scripts in linux. In the phase 5 directory structure, these are under one of the
subdirectories of the ./run/ directory. All of the processes discussed briefly in section
2.3.3 above are accomplished through scripts.

Calibrations are usually scripts recursively calling other scripts with update programs
between. Other scripts are available that correctly format data for use in the phase 5
model and automatically document the program and source data.

Generally these scripts start by creating a temporary work directory under ./tmp/scratch/
and then moving to that directory so that all of the fortran programs running relative
paths can find the necessary files.

Most scripts that run a scenario or calibration take at least two arguments and sometimes
more. The first argument is almost always the scenario you wish to run. The second
argument is almost always the list of segments you wish to run. Just as the scenario name
references a file of the same name under ./config/control/landORriver/, the list of
segments specifier references are file under ./config/seglists/ that contains the list of
segments. There are several methods discussed later to generate and modify these files.

To determine the correct arguments, run the script with no arguments and the instructions
will normally be printed to the screen.

Most of the scripts have the same basic structure:
       1. get input arguments
       2. assemble any needed data or pre-processing programs
       3. loop over segments and perform a single operation or a short series of
           operations.


2.6 Directory Structure
The structure of phase 5 is highly complex, due to the necessary flexibility in run
specifications and input data. The directory structure is mean to be as easily
comprehendible as possible, given the number file and information types that need to be
stored. The directory structure is also meant to make it simple to select entire directories
for backup, while leaving large reproducible data sets without backup. The code and
scripts are written so that most of the file locations can either be referenced by absolute or
relative address.

Understanding the general structure will be helpful in quickly understanding how to
operate the phase 5 watershed model. There is a short description later with a longer
exposition in section 4.

When the directory structure is downloaded and expanded, it should all be under a single
directory. All references from this point forward are relative to that directory. For


Modified 11/7/2010                                                                          13
example, if you download and install under /disk1/userspace/p5model/, the reference to
the directory /disk1/userspace/p5model/code/ will be shortened to ./code


2.6.1 top level
./code/ source code and executables for phase 5 software and HSPF

./config/ configure files used to set up a model run, including input and output variables,
scenario control files, lists of segments to be simulated, standard HSPF modules and
variables, and geographic information and connectivity.

./documentation/ - documentation

./input/ all input data, including scenario data, parameters, and observations.

./output/ all ASCII output files from WSM simulation except for those under ./sumout/

./run/ all scripts used to calibrate and run scenarios for the phase 5 model

./sumout/ summarized WSM outputs, broken out from ./output/ for backup

./tmp/ temporary UCI and WDM files for model runs and temporary locations for
storing model operating messages


2.6.2 Using the directory structure
For a typical model scenario run, the user would go through several steps in different
parts of the directory structure. The user would first generate data sets in the ./input/
directory. This may be done automatically through scenario builder. The data sets would
then be referenced in the control files under ./config/, and then the scripts to run the
model would be executed under ./run/ the output would be picked up under ./output/ and
./sumout.

It is almost never necessary to do anything in the ./tmp/ directory.

The ./code/ directory is not manipulated in the course of running scenarios or the
calibration. It would only be necessary to modify this directory if the user wanted to
change the calibration procedure or update the capability of the model.

2.7 Code Structure
All of the phase5 code is written in fortran. The directories under ./code/ generally
follow the names of the scripts that you run to call the programs. Each directory or
subdirectory will normally have a single program that is compiled from several source
files using a compile script. Many of the variables, parameters, and subroutines that are
common throughout the code system are shared.



Modified 11/7/2010                                                                       14
2.7.1 General coding style
Within each directory containing a program, there is a file called main.f, which is the
main program file. Other files in the directory may contain one or more subroutines,
shared varibles, notes about the program, or compilation instructions. An ‗oldcode‘
directory may be present to store older versions.

Some general style points:
   1. Most program and subroutine files have a header with notes about the purpose of
      the program.
   2. The ‗implicit none‘ statement forces declaration of all variables
   3. Subroutine calls indicate the Input, Modified, and Output arguments
   4. Errors are handled at the end of each program or subroutine. If an expected error
      occurs, a subroutine is called to write a file called ‗problem‘ in the current
      directory. All scripts check for this file and stop on finding it.


2.7.2 Lib
The lib under ./code/src/lib/ contains variables and subroutines used throughout the code.
For example, the parameter that specifies the maximum number of simulated days (to
dimension variables) is set here and referenced throughout the code. Also in this section
are the locations of the various major directories used to locate files.

Subroutines that perform useful tasks are stored here as well. Examples are sorting,
finding a median, reading in a list of segments, reading a comma delimited file, and
writing an error file.

2.7.3 Hspf
The CBP version of the HSPF code is in this directory as well. You will not be able to
substitute your own version. The CBP version has some substantive differences from the
standard version and the standard version has a hard-coded path to the message wdm,
which may not be resolvable on your system.

3 Quick Start guide
You‘ve already read or skimmed through 10 pages and haven‘t executed any commands
yet. Maybe you‘ve jumped directly down here from the table of contents. If you‘re like
most people you‘re ready to get going before reading any more.

You can quickly run a scenario, but to perform any really interesting functions, you‘ll
have to get much more familiar with the code and scripts. Getting more familiar will also
stop you from running the model inefficiently or incorrectly. It is possible and all too
easy to make errors that will erase data or give your incorrect results. So go ahead and
have your fun, but then come back to finish reading the manual.

3.1 Download
The main model download page is:


Modified 11/7/2010                                                                        15
http://ches.communitymodeling.org/models/CBPhase5/index.php
with the links to download the large model files here:
http://ches.communitymodeling.org/models/CBPhase5/modelcode.php
The setup instructions are located on that page and are fairly short and simple, so they
will not be repeated here. It‘s as simple as downloading, unzipping, and recompiling
with a single command. If there are any problems with these steps, please let us know so
that we can increase portability.

3.2 Run a small basin with a single command
The different steps in running a simple scenario can be run through a single command,
./run/standard/runall.csh.
In the linux command line environment, go to ./run/standard/ and type:
run_all.csh test stmary
The scenario description files ./config/control/land/test.con and
./config/control/river/test.con must exist and the segment lists ./config/seglists/stmary.riv
and ./config/seglists/stmary.land must exist. If any of these files do not exist in the
version you are using, select other arguments from the files that are available in the
./config/control/ and ./config/seglists/ directories. Choose smaller files in the
./config/seglists/ directory to limit spatial scale and run time.

If this run is successful, you should see various messages being written to the screen from
the phase 5 software and from HSPF. When the run is finished you will get a prompt
after the last message. If there is a problem during the run, a ‗problem‘ file will be
produced and the contents of this file will be written to the screen, prefaced by
―PROBLEM FILE WRITTEN‖

3.3 rtfm


4 Detailed directory Structure
There are about 20 different major types of files, each with subcategories, scenarios, and
instances. A scenario run of the watershed model uses about 10 scripts to generate
around 60,000 files from an input of several hundred files. Clearly, this calls for an
organized directory structure.

Many models have a simple structure where the inputs are all loaded into a single
directory and then outputs are generated within that same directory. Due to the high
number of inputs and outputs and the flexibility desired, the phase 5 system has a
different organizational structure. All files are grouped by type. For instance, all inputs
are in the same general directory with subdirectories for input target (land or river), input
type (fertilizer, atmospheric deposition), and scenario.

The following is a relatively detailed description of the locations of each type of file.
These locations are hard-coded so that if a change is desired, the change must be made in
the directory structure and in the code as well.



Modified 11/7/2010                                                                         16
4.1 top level
./code/ source code and executables for phase 5 software and HSPF

./config/ configure files used to set up a model run, including input and output variables,
scenario control files, lists of segments to be simulated, standard HSPF modules and
variables, and geographic information and connectivity.

./documentation/ - documentation

./input/ all input data, including scenario data, parameters, and observations.

./output/ all ASCII output files from WSM simulation except for those under ./sumout/

./run/ all scripts used to calibrate and run scenarios for the phase 5 model

./sumout/ summarized WSM outputs, broken out from ./output/ for backup

./tmp/ temporary UCI and WDM files for model runs and temporary locations for
storing model operating messages


4.2 code
./code/bin/ - all compiled executable programs. The executables are placed here by the
complication scripts. The model scripts reference this location in executing the
programs.

./code/src/ - all source codes for the phase 5 framework and HSPF.
        There are a few useful files in this directory:
                        Compile_all.csh – compiles all phase 5 code.
                        grepf, grepcomp, grepinc short scripts for finding character strings
                                within the directories above ./code/src/. For example, if
                                you wanted to know where a particular error message
                                originated, you could use the grepf script to search for it
                                within the code.

       ./lib/ - parts of the code that are generally shared throughout the phase 5 structure.
                ./dsn/ - code that interacts with the HSPF libraries to read WDM files
                ./inc/ - files containing common variables that are included in source code
                         throughout the phase 5 structure.
                ./get/ and ./util/ - common utility and file reading subroutines


       ./data_import/ - Programs used to make input WDMs from raw data and to
              manipulate and interpolate data.

       ./lug/ - land UCI generator, generates a UCI file from input parameters and data


Modified 11/7/2010                                                                        17
       ./rug/ - river UCI generator, generates a UCI file from input parameters and data

       ./etm/ – external transfer module, linking land simulation to river simulation. Also
               some subdirectories for running the etm and postprocessor simultaneously

       ./postproc/       - all post-processors

       ./calibration_utils/ – all utilities used to perform and support automatic
               calibration procedure plus a few utility programs that did not fit into other
               categories. Some of the more important codes:

               ./basingen/ - the code used to produce lists of segments for a specific
                      geography. The user specifies a downstream location and the
                      upstream land and river segments are specified.

               ./observed_data/ - codes to generate the calibration and validation data
                      sets, format observed data for use in the model, count observations,
                      and generally manipulate or summarize observed data files.

               ./change_param/calib_iter/ - Codes to perform the automatic calibrations
                      are above this directory.

       ./hspf/ - HPSF source code
               ./lib3.2/ hspf libraries, these are sometimes called by phase 5 software
                        directly to interact with WDM data files.
               ./hspf11.1/ - the main calling program for HSPF and the HSPF executables
                        are above this directory.


       ./matlab/        - processors used to display model outputs on MATLAB


       ./wqm_load/        - processors used to import WSM outputs to WQM

4.3 config
./config/blank_wdm/ - Unpopulated template WDMs of all types necessary for WSM.
        When a land or river is run, a blank wdm is copied to the working directory and
        populated with the output of that run. There are also blanks for the different types
        of data.

./config/catalog/- files that contain information on the basic setup of the watershed
        model. Which HSPF variables to use, how they connect, and the universe of
        segments that can be run, for example.

       ./geo/ - geographic information of the entire model domain. Contains files with
                the complete list of river segment, land segments, and land-river segments.


Modified 11/7/2010                                                                        18
               Also contains physical linkage to estuarine model. The user can have
               different scenarios above this directory so that a different segmentation
               scheme could be used.

       ./iovars/ - list of files that specify land use variables, river variables, and
               correspondence between them, as well as linkages to variables in other
               models, such as the estuarine model. Scenarios are used above this
               directory as well.

       ./modules/ - list of HSPF tables and variables to be expected with each active
             module. Scenarios are not used here as these specifications are native to
             HSPF and cannot be changed.

./config/control/ - Directory where user specifies the inputs for a given scenario or
        calibration run. See section 5 for a complete discussion of the construction of
        these files.
        ./land/ - list of files that control the specifications for each land scenario
        ./river/ - list of files that control the specifications for each river scenario
        ./calib/ - files and programs that support the automated calibration routines. These
                 are subdivided by calibration type

./config/seglist/ - lists of segments to be called by scripts during a run. These segment
        lists can be generated by hand or automatically by scripts. There are also utilities
        to add, subtract, and rename the lists.


4.4    documentation
./documentation/ - Documentation is supplied with the model download in both native
       format and pdf for some documents.

       ./official/ - final model documentation

       ./informal/ - Helpful documents that are not part of the official published model
              documentation. Some step-by-step instructions for particular tasks may be
              found here, along with graphical representations of the workflow.

       ./hspf/ - HSPF manual

       ./early/ - Drafts and other miscellaneous papers

4.5    input
./input/ - inputs necessary to run the watershed model.

./input/calib/ - calibration data
        ./observed/ - observed flow and concentration data in streams
                ./all_data/ - all available observed flow and WQ data


Modified 11/7/2010                                                                         19
              ./calib/ - observed data that are used for model calibration, split
                       automatically from data sets in all_data
              ./valid/ - observed data that are used for model validation, split
                       automatically from data sets in all_data
              ./estimator/ - flow and load data calculated from USGS‘s Estimator
                       model
              ./random_data_files/ - various data files and summaries.

       ./target/ - sediment and nutrient loading targets for the land calibration, specified
               for each land use within each land segment. The user can specify different
               scenarios of targets.

./input/param/ - model parameter files. These files contain all HSPF parameters
        necessary to run the watershed model for all land uses and segments, and
        simulated rivers.
        ./afo/ - subdirectories with 3-character names correspond to separate land uses.
        ./alf/            Above these directories are subdirectories with correspond to
        ./{etc}/          parameter scenarios. Within the scenario directories are comma-
        ./urs/            delimited files containing the parameters for each HSPF module.

       ./common/ - parameters that are common for all land uses
       ./river/ - river parameters, in parameters scenario subdirectories
       ./scripts/ - scripts used to manipulate parameter sets. Common tasks are copying
               parameters from one scenario to another, copying parameters from one
               land use to another, and resetting default parameters during calibration.
       ./transport/ - nutrient regional factors and sediment transport factors.
       ./excel_files/ - spreadsheets used for various calculations related to parameter

./input/scenario/ - data files for scenario runs. Files above this directory must all be
        formatted correctly to go into the watershed model.
        ./climate/ - weather data
               ./met/ - meteorological WDMs. Potential Evapotranspiration, dewpoint,
                       wind speed, solar radiation, air temperature and cloud cover are
                       stored in binary-format WDM files. Each directory is a separate
                       complete meteorological scenario. Each land segment has its own
                       WDM with these time series
               ./prad/ - precipitation and atmospheric deposition WDMs. The directory
                       is structured the same way as the met directory.

       ./land/ - input data for the land simulation. This information is mostly generated
               by scenario builder. In all subdirectories, comma-delimited files have
               descriptive names that correspond to the specifiers in the land control file.
               ./spec_flags/ - special action flags for land sediment and nutrient
                       simulation. These specify to the lug which other data sets will be
                       referenced from this directory




Modified 11/7/2010                                                                        20
              ./annual_uptake/ - maximum annual plant uptake of nitrogen and
                       phosphorus
              ./crop_cover/ - fraction of land surface under vegetative cover
              ./dets/ - mass of sediment generated by plowing operations.
              ./fertilizer/ - monthly application rate of commercial fertilizer
              ./legume/ - monthly rate of leguminous fixation
              ./manure/ - monthly application rate of manure
              ./monthly_fraction_uptake/ - monthly fraction of plant uptake

       ./river/ - input data for river simulation. Some information is generated by the
               scenario builder, while other data sets are input from other databases.
               ./bmpacres/ - BMP implementation acreages
               ./bmptypes/ - BMP definitions and effectiveness coefficients.
               ./land_use/ - land use acreages
               ./div/ - surface water diversion WDMs
               ./ps/ - point source WDMs
               ./septic/ - septic WDMs

       ./wqm/ - input necessary in some instances for specifying the link between the
             watershed model and the estuarine water quality model. Most watershed
             scenarios do not reference this directory.
             ./adtep_factors/ - spreadsheets helpful in calculating atmospheric
                      deposition scenarios from existing scenarios
             ./marsh/ - inputs for the marsh loads calculation. This is not used often
             ./RiverFactors/ - factors relating an estuarine scenario to a watershed
                      model scenario. For example, start with the phase 5 year 2000
                      scenario and multiply the non-point source by 0.8

./input/unformatted/ - unformatted raw input data. Data that come in from sources
        other than scenario builder and have to be reformatted are stored here.

4.6 output
./output/ - all files in this directory are ASCII files so that they can be viewed easily by
      the user. The eof, eos, and del directories have loads that are organized into a
      very large number of small files. A secondary program is normally used to
      summarize these outputs into a more usable format. This strategy uses the server
      file system as a database. The directories above these directories are organized by
      time aggregation (daily, monthly, annual, average annual) and river scenario.

       ./eof/ - Edge of field loads, the post-processed output of the HSPF land simulation

       ./eos/ - Edge of stream loads, which are the edge of field loads multiplied by land
                use acreage and modified by any bmp effects, regional factors, or sub-
                scale transport factors.




Modified 11/7/2010                                                                        21
       ./del/ - Transport factors, delivery factors, and delivered loads. Transport factors
                are the fractional change in load within a given segment. Delivery factors
                are the fraction change in load from the edge of stream to tidal water, and
                delivered loads are the loads that are delivered to tidal waters.

       ./etm/ - binary land-river transfer files generated from the external transfer
               module. The files under this directory contain time series to translate
               edge-of-field to edge-of-stream loads and summarize by river segment.

       ./hspf/ - standard output and message files from HSPF. The output are sometimes
               used during calibration to gather information on fluxes. The message files
               are checked for successful completion.

       ./input/ - echo out of nutrient application data used in a model run.

       ./pltgen/ - pltgen output files. These are a standard method of getting information
               out of an HSPF run. The user can specify which outputs are to be put into
               pltgens in the control file for a run. They are use in calibration, but are not
               necessary for scenarios.

       ./river/ - post-processed river simulation results and various calculated statistics

       ./wqm_input/ - estuarine water quality model files processed from WSM
             simulation results


4.7    run
./run/ - this directory contains scripts to perform watershed model operations. It is
        organized into general categories of tasks.

       ./make_seglists/ - scripts that allow the user to generate or modify a list of
             segments to be simulated

       ./standard/ - the most common scripts used in running the watershed model for
              scenarios.

       ./datascripts/ - scripts for preparing and transforming observed data and model
              input.

       ./link_p5_to_wqm/ - scripts that convert watershed model outputs to estuarine
               water quality model inputs

       ./useful/ - scripts that perform functions that don‘t fit into the other categories and
               scripts that are variations of standard scripts. Put here to avoid kludgy
               clutter.



Modified 11/7/2010                                                                         22
       ./fragments/ - common parts of scripts that are called from other scripts.

       ./run24/ - scripts that are set up to run parallel operations on the Chesapeake Bay
              Program 24-processor machine.

       ./calibration/ - scripts for running automatic model calibration. Subdirectories
               are generally named for the HSPF modules they are meant to optimize.
               Within the subdirectory there is generally one master script that calls
               several other scripts.

4.8    sumout
./sumout/ summarized output. The output in the ./output/ directory is too disaggregated
      for the most part to be effectively used. The user can use scripts to summarize
      this output on any spatial scale. The output of these summaries are stored here,
      organized into directories by temporal scale (annual, monthly, or average annual)
      and by river scenario. This directory is kept separate from ./output/ to make it
      simple for a system administrator to backup the relatively small ./sumout/
      directory, but not the very large ./output/ directory.

4.9    tmp
./tmp/ - temporary file storage.
        ./scratch/- temporary locations for running operations within phase 5. Generally,
                a script will create a semi-randomly named directory here, perform
                operations from that location and then remove the directory when the
                script has completed.

       ./uci/ - all land and river UCI files. These are automatically generated and read
                and usually do not require any user interaction.

       ./wdm/ - all land output wdms and all river edge-of-stream and output wdms.

5 Details of the scenario control files
5.1.1 Notes on control files.
The name of the control file is the name of the scenario. If you run the scenario myscen,
      there must be a correctly formatted myscen.con in the control file directory. All
      other files and directories are referenced from the control file.

The control files consist of short tables. Tables are formatted differently, but follow a
       few general rules:
1.     The tables start with a keyword, for example ‗TIME‘ and end with the word
       ‗END‘ plus the keyword (END TIME).
2.     Blank lines are allowed outside of tables, but not normally within tables
3.     Comments are allowed outside of tables. They must contain the three-character
       string ‗***‘. This comment style is the same as used in the HSPF UCI files. You



Modified 11/7/2010                                                                         23
       may choose to put several lines of notes, prefaced by ‗***‘ at the beginning of
       each control file.
4.     Comments are sometimes allowed inside of tables. Follow examples for correct
       formatting
5.     Tables may be in any order.
6.     Spacing is important. Characters should be in exact columns
7.     If possible, use a linux-based editor to edit control files. Editing a control file in a
       DOS or Windows based editor will cause the control file to be stored in the DOS
       format, which is slightly different from the linux format. Most portions of the
       phase 5 software account for this difference, but it may not always be the case. In
       addition, it is sometimes difficult to see spacing accurately in a windows-based
       editor.

Samples of control files are in ./config/control/land/ and ./config/control/river/

Important concept: Some of the tables contain references to several data sets of the
same type to generate a time series of that constituent. For example, the land use table in
a river control file may have references to multiple files:

LAND USE
year mo da <-file--->***
1982 07 01 p52cal_eighty2
1987 07 01 p52cal_5years_later
1992 07 01 p52cal_1992
1997 07 01 p52cal_1997
2002 07 01 p52cal_2002
END LAND USE

Using this table, the user is specifying that the file:
./input/scenario/river/land_use/land_use_p52cal_eighty2.csv applies exactly on 7/1/1982
and the file ./input/scenario/river/land_use/land_use_p52cal_5_years_later.csv applies
exactly on 7/1/1987. The user may specify any number of files greater than zero and may
use any dates for their application, whether those dates are within the time period of the
simulation or not. A few restrictions apply on the total number of files and the file name
length, but these are easily expanded in the code if necessary.

For all other days in the simulation other than the specified days, values are interpolated
or extrapolated. In the above example, the land use for 4/1/1985 would be interpolated
between the 7/1/1982 and the 7/1/1987 files and the land use for 12/31/2005 would be
extrapolated from the 7/1/1997 and 7/1/2002 files. If extrapolation is not desired, simply
repeating the specification of p52cal_2002 at any later date would cause the land use to
be held constant at the 7/1/2002 level.

If only one file and date are given, this value is used as a constant throughout the
simulation.




Modified 11/7/2010                                                                          24
5.1.2 Land Control File
The particular group of tables that a scenario control file contains may vary depending on
the simulation. For example, if you are only simulating hydrology or if you are only
simulating forest land use, it is not necessary to specify tables for fertilizer application.
In general for scenario operations, however, the list of tables will remain constant. A
land scenario control file may contain the following tables:

TIME
The start and stop dates of the simulation. The simulation with proceed hourly from the
beginning of the first day to the end of the last.
Requires exactly two lines.

CROP COVER
The fraction of the land surface that is covered by vegetation and not susceptible to
rainfall generation of detached sediment. Files are monthly fractions for a given year.
The user may specify multiple data sets for different points in time. Files referenced are
in ./input/scenario/land/crop_cover/

FERTILIZER
Application rates in pounds per acre per month per nutrient species for given year. The
user may specify multiple data sets for different points in time. Files referenced are in
./input/scenario/land/fertilizer/


MANURE
Application rates in pounds per acre per month per nutrient species for given year. The
user may specify multiple data sets for different points in time. Files referenced are in
./input/scenario/land/manure/


LEGUME
Generation rates for leguminous production of nitrogen in pounds per acre per month for
given year. The user may specify multiple data sets for different points in time. Files
referenced are in ./input/scenario/land/legume/


TOTAL UPTAKE
Total Annual uptake of Nitrogen or Phosphorus. The user may specify multiple data sets
for different points in time. Files referenced are in ./input/scenario/land/annual_uptake/


MONTHLY FRACTION UPTAKE
The fraction of the total uptake that occurs in each month. The user may specify multiple
data sets for different points in time. Files referenced are in
./input/scenario/land/monthly_fraction_uptake/



Modified 11/7/2010                                                                          25
DEFAULT CROPDATA
The name of the file containing default values for all data types. For instance, if a value
is not found for fertilizer application for a given land segment and land use, the lug will
use the value from the fertilizer file with the name that matches the contents of this table.
If this table is missing, it will cause an error only if the regular data are not found. The
user may only specify one line.

SPECIAL ACTIONS FLAGS
A file must be read into the land UCI generator to determine which data sets are needed
by which land uses. For instance, a fertilizer application data set should be expected for
crop land, but not for forest. This table points to a directory containing flags to specify
the actions expected for this scenario. The user may only specify one line. Directory
referenced is in ./input/scenario/land/spec_flags/

PRECIP ATMOS DEPOSITION
Points to the name of the directory containing the WDMs with the precipitation and
atmospheric deposition data. The user may only specify one line. Directory referenced is
above ./input/scenario/climate/prad/

METEOROLOGY
Points to the name of the directory containing the WDMs with the meteorology data. The
user may only specify one line. Directory referenced is above
./input/scenario/climate/met/

PARAMETERS
Points to the name of the directory containing the HSPF parameters to be used. In
scenario mode, these are generally the set of parameters for the calibrated model. The
user may only specify one line. Directory referenced is above ./input/params/???/, where
??? is a separate directory for each 3-character land use. There is at least one separate file
in this directory for each active module.

IOVARS
Points to the name of the directory containing the input, output, and translation
specifications for this scenario. This will not be changed for most scenarios. The user
may only specify one line. Directory referenced is above ./config/catalog/iovars/

GEOMETRY
Points to the name of the directory containing the complete listing of the segments in the
model domain. Note that this is the complete list of possible segments, not the list of
segments to be run in any particular run. This will not be changed for most scenarios.
The user may only specify one line. Directory referenced is above ./config/catalog/geo/.

PER MODULES
The first line is a list of pervious land uses to which this table applies. All subsequent
lines until the ‗END PER MODULES‘ line are a listing of active modules for this list of



Modified 11/7/2010                                                                         26
land uses. The modules are the exact module names used by HSPF. Each active module
must have:
        1. the tables described under ./config/catalog/modules
        2. parameters available under the directory specified in the PARAMETERS
            table in a file with the same name as the module
        3. (optional) output variables assigned in the
            ./config/catalog/iovars/iovar_scen/perlnd file
        4. (needed if output variables assigned in perlnd file) blank dsns added to the
            ./config/blank_wdm/land.wdm
Multiple instances of this table are allowed so that different groups of perlnds may have
different active modules

IMP MODULES
The same as PER MODULES except for impervious land uses.

CALIBSCEN
This table specifies the calibration run that this specific land scenario is based on. This
information is referenced to modify PQUAL and IQUAL parameters to simulate
sensitivity to inputs. The parameters controlling export of nutrients are reduced or
increased depending on the ratio of scenario inputs of fertilizer, manure, leguminous
fixation, and atmospheric deposition to the calibration inputs of these same sources.

PLTGEN
These tables specify the PLTGENs (HSPF text output) to generate for any given run. All
lines within the table are in pairs. Multiple pairs of lines are allowed within a PLTGEN
table. Multiple instances of this table are allowed, which are helpful for organizing
groups of PLTGENs used for different purposes. This table is optional


The first line of the pair is a list of the land uses, separated by a space, to which the
second line of the pair applies. The second line of the pair is the specification of the
variable for this PLTGEN. The specification line is similar to the PLTGEN specification
line in the NETWORK table of HSPF except that it is space delimited rather than fixed
column and has a few extra fields. See the HSPF documentation of the NETWORK and
PLTGEN sections for a better understanding of the following variables

The second line of the pair, or specification line, has the following structure, which is
best shown as an example:
dets SEDMNT DETS 1 0 AVER MEAN 24 AVER daily_average_mass_det_storage_in_tons


       dets – pltname – 8-character or fewer name that the user specifies. This name is
               the extension on the PLTGEN file produced. The full name of the file is
               landuse_landseg.specifier
       SEDMNT – pltgrp – 6-character name of the variable group within HSPF
       DETS – pltmem – 6-character or fewer name of the member within HSPF
       1 – pltnum1 – 1-digit integer for the member name subscript #1
       0 – pltnum2 – 1-digit integer for the member name subscript #2


Modified 11/7/2010                                                                            27
       AVER – plttran1 – 4-character transition function for the NETWORK table in the
               UCI, possible values are SAME, AVER, SUM, MAX, MIN, see section
               4.6.7 in the HSPF documentation
       MEAN – pltmnpt – 5-character function, either MEAN or POINT for the
               NETWORK table in the UCI. Specify the correct one for the source
               variable. Specifying the wrong one will generate an error in HSPF, so
               trial and error will work for this variable.
       24 – pltpivl – integer, number of hours to be aggregated using the transition
               function in the PLOTINFO table in the UCI
       AVER – plttran2 – 4-character transition function for the CURV-DATA table,
               similar options as plttran1
       daily_average_mass_det_storage_in_tons – pltlabel – 26 or fewer-character name
               to be written into the header of the PLTGEN file.

HSPF allows up to 10 time series per PLTGEN, but the phase 5 structure allows only a
single time series in each PLTGEN.


5.1.3 River Control File
All tables in the river control file are mandatory except for the PLTGEN table.

TIME
The start and stop dates of the simulation. The simulation with proceed hourly from the
beginning of the first day to the end of the last.
Requires exactly two lines.

LAND USE
Acreage of each land use in each land-river segment. The user may specify multiple data
sets for different points in time. Files referenced are in ./input/scenario/river/land_use/

TYPEBMP
One line expected. Refers to a directory above ./input/scenario/river/bmptypes/. In this
directory are several files that described the attributes of all bmps.

BMPACRES
Acres of each BMP in each land use in each land-river segment. The user may specify
multiple data sets for different points in time. Files referenced are in
./input/scenario/river/bmpacres/

LAND SCENARIO
The land scenario (name of the land control file) for this river scenario. Each land use is
listed on a separate line. Often, the name of the land scenario and river scenario will be
the same. In some cases, the user may want to use an existing scenario so that the land
simulation would not have to be re-run. For instance if there is no change in atmospheric
deposition or meteorology between scenario A and scenario B, they could use the same
forest simulation and it would only have to be run once.


Modified 11/7/2010                                                                       28
DIVERSIONS
Names the directory above ./inputs/scenario/river/div/ that contains the diversion wdms.
Each river segment has a separate file

SEPTIC
Names the directory above ./inputs/scenario/river/septic/ that contains the septic wdms.
Each land-river segment has a separate file.

POINT SOURCE
Names the directory above ./inputs/scenario/river/ps/ that contains the point source
wdms. Each land-river segment has a separate file.

TRANSPORT
One line expected. Names a file above ./input/param/transport/ that regional factors by
land use and constituent (TN, TP, and TSS).

PRECIP ATMOS DEPOSITION
Points to the name of the directory containing the WDMs with the precipitation and
atmospheric deposition data. The user may only specify one line. Directory referenced is
above ./input/scenario/climate/prad/

METEOROLOGY
Points to the name of the directory containing the WDMs with the meteorology data. The
user may only specify one line. Directory referenced is above
./input/scenario/climate/met/

PARAMETERS
Points to the name of the directory containing the HSPF parameters to be used. In
scenario mode, these are generally the set of parameters for the calibrated model. The
user may only specify one line. Directory referenced is above ./input/params/river/.
There is at least one separate file in this directory for each active module. There are also
specifications for FTABLES and for variable FTABLES used to simulate reservoirs.

IOVARS
Points to the name of the directory containing the input, output, and translation
specifications for this scenario. This will not be changed for most scenarios. The user
may only specify one line. Directory referenced is above ./config/catalog/iovars/

GEOMETRY
Points to the name of the directory containing the complete listing of the segments in the
model domain. Note that this is the complete list of possible segments, not the list of
segments to be run in any particular run. This will not be changed for most scenarios.
The user may only specify one line. Directory referenced is above ./config/catalog/geo/.

MODULES



Modified 11/7/2010                                                                         29
A listing of active HSPF modules for this run. The modules are the exact module names
used by HSPF. Each active module must have:
        5. the tables described under ./config/catalog/modules
        6. parameters available under the directory specified in the PARAMETERS
            table in a file with the same name as the module

CALIBSCEN
This table specifies the calibration run that this specific land scenario is based on. This
information is referenced to modify sediment parameters to simulate sensitivity to inputs.
The parameters controlling scour and benthic release of nutrients and sediment are
reduced or increased depending on the ratio of riverine scenario inputs to riverine
calibration inputs.

PLTGEN
These tables specify the PLTGENs (HSPF text output) to generate for any given run.
The function is identical to the PLTGEN table for the land control file.

6 Details of running the model using scripts
Almost all functions of the watershed model are carried out using c-shell scripts from the
command line in linux. The scripts generally take some type of argument from the
command line to specify the scenario, basin, years, or other parameter to run. A typical
operation involves generating some data through some outside means, saving the data in
a specific format with a specific file name, and then performing some model-related task
on this outside data using the scripts.

For scenarios, data on land use, nutrient inputs, etc are generated through scenario
builder. The files are saved to a specific location, control files are generated or modifies
to point to those data, and scripts are invoked to run the model and summarize the output.

For calibration, some new data set may be supplied as in the scenario or code may be
modified as a test of an improved method. The calibration is then run through a set of
linked c-shell scripts which call the supplied data and code.

Scripts are also used to generate wdms from ASCII data for diversions, septic loads, point
sources, atmospheric deposition, and meteorological inputs.

To run a script, go to the directory where the script is located. The file permissions
should be set so that the file is executable. Type the name of the script followed by any
arguments needed by the script. If you have not entered the correct number of
arguments, you will see an error message that tells you which arguments to use.
Generally, a useful way to see what arguments to use is to invoke the script with no
arguments.

Most scripts can be run interactively or be called by another script. If the script is being
run by another script, the directory tree to get to the phase 5 base directory from the



Modified 11/7/2010                                                                         30
temporary directory where the calling script is executing commands is entered as an extra
argument. This argument is generally referred to as ‗tree‘.

6.1 Pay attention to the error messages
As with most scientific code, it is all too easy in phase 5 to supply data in incorrect
formats or to run the model out of sequence. Although all possible errors are not caught,
a good number of them are caught and error messages are generated and sent to the
screen. The most common errors are when an expected file does not exist or is in an
incorrect format. Generally the scripts will let the user know when this is a problem.

Do not assume that the error messages are ‗just unix garble‘. They indicate a real
problem and proceeding without fixing the problem will result in a failure of the next
program at best and incorrect model results at worst.

6.2 General structure of the scripts
The scripts follow a general programming style that will be relatively simple to read as
you get more comfortable with using the system. There are normally several sections in
each script that are in the same order and have generally the same function. These are
described below using ./run/standard/run_land.csh as an example.

6.2.1 Help with invocation
The first part of the script has some code to print the help message to the screen if an
incorrect number of arguments is entered. Below is an example:




Figure 5 Initial section of a script
If the number of entered arguments is neither two nor three, the usage instructions will be
printed to the screen and the script will stop execution

6.2.2 Set local variables and create temporary directory
The arguments are stored as local variables and may be used to set more local variables
through referencing other files. Also, if this script is being run interactively, a temporary
directory is created and the working directory is changed to this temporary directory.




Modified 11/7/2010                                                                         31
Figure 6 - Section of the script to deal with variable definitions
The scenario and basin names are stored, then the ‗tree‘ variable is set if this is called by
another script, otherwise it is set from a preexisting file and a temporary directory is
created.
Other files are then called to set local variables. The set_landuse files populates the
variables ‗perlnds‘ and ‗implnds‘ with the list of pervious and impervious land uses.
Set_quiet_hspf sets the path to the HSPF executable. ${basin}.land is the file that
contains the list of segments in the basin.

6.2.3 Body with loop and error messages




Figure 7 - main body of a script
The above code loops over all segments in the $basin.land file, then all the land uses in
the set_landuse file, using the variables $segments, $perlnds, and $implnds, running the
land UCI generator code for all land uses and segments. All phase 5 code creates a file
called ‗problem‘ in the current directory if there is an error encountered. All of the
scripts check for this file and echo it to the screen if encountered.


Modified 11/7/2010                                                                          32
6.2.4 Clean up




Figure 8 - end portion of a script
The final section of a script generally has the above lines that remove the temporary
directory if the script is called directly by the user. It may also move files to final
locations or generate log files, particularly for scripts that generate input data.

6.3 Standard arguments to scripts
Scenario – described in section 2.4. Can be either river or land scenario depending on the
       context. Generally, except for running the land uci generator and the land
       simulation itself, most other scripts take the river scenario as the argument. As
       mentioned above, the scenario directs the script and programs to the directories of
       the same name and the control file of the same name.
Basin – the name of the file above ./config/seglists/ that contains the list of segments to
       be run in the script. Scripts expect either a $basin.land file containing land
       segments or a $basin.riv file containing river segments to exist previously in this
       directory. These can be generated by hand or, more often by the scripts under
       ./run/make_seglists/ directory. More information can be found in section 8.1


6.4 Script Directories
The scripts are organized into several directories for ease of use. Some scripts and
       directories are relatively static and are used frequently by users directly and called
       by other scripts. Others are generated on a more ad-hoc basis as modifications of
       other scripts and are modified frequently. It is possible to move scripts between
       directories and reorganize them, but care must be taken on two counts. 1) Some
       scripts are called by other scripts. The name and location of the called script must
       not be changed without changing the calling script. 2) Relative paths must be
       updated. If the move is parallel, the script will probably function correctly, but if
       it moved up or down a directory all relative paths must be changed.

Below are descriptions of the scripts used most often in phase 5. Other scripts may exist
      within the directory that are not listed here. These are generally special-case
      scripts that can be understood through inspection or just by the name of the script.

Throughout most of these directories, there will be a version of the check_slurm script.
      This is used to quickly check the standard output file for the slurm job scheduler
      when running in parallel model. If you are not running in parallel mode or are not
      using the slurm job scheduler, then you can delete these files.

6.4.1 Standard
make_land_directories.csh


Modified 11/7/2010                                                                        33
Takes the land scenario name as the argument.
Creates all of the directories needed to store the output and intermediate files for a land
scenario for all land uses.

make_river_directories.csh
Takes the river scenario name as the argument.
Creates all of the directories needed to store the output and intermediate files for a river
scenario.

remove_land_directories.csh
Takes the land scenario name as the argument.
Removes most of the directories and associated files for a land scenario. The output of a
scenario is very space intensive, so this script is used to free up space on the computer
system. The input files and parameters are not removed so that the scenario can be
regenerated if necessary. Running this script will

remove_river_directories.csh
Takes the river scenario name as the argument.
Removes most of the directories and associated files for a river scenario. The output of a
scenario is very space intensive, so this script is used to free up space on the computer
system. The summarized output files are not removed. The input files and parameters
are not removed so that the scenario can be regenerated if necessary.

run_lug.csh
Takes the arguments of land scenario and basin.
Runs the Land UCI Generator program, creating a UCI for each land use and land
       segment in the basin.

run_land.csh
Takes the arguments of land scenario and basin.
Runs HSPF on the UCI for each land use and land segment in the basin.
       copies a blank wdm to the name $landuse$landseg.wdm
       runs the UCI with HSPF
       moves the wdm, out, and echo file to the appropriate directories.

run_etm.csh
Takes the arguments of river scenario and basin.
runs the External Transfer Module that stands in for the MASSLINKS and
        SCHEMATIC.
        Runs the program ‗make_binary_transfer_coeffs.exe‘ that combines the time-
        varying BMP, time-varying land use, and static EOF-EOS transfer coefficients
        into hourly time series and stores them in binary files.
        runs the program ‗transfer_wdms.exe‘ that reads the binary files and multiplies
        the land wdm outputs by these factors and creates wdm files of EOS loads in
        RCHRES variables using the transfer file ./pp/catalog/iovars/land_to_river




Modified 11/7/2010                                                                             34
run_rug.csh
Takes the arguments of river scenario and basin.
Runs the river UCI Generator program. This script is for calibration or other special
       cases only. For scenarios, use run_scenario_river.csh

run_river.csh
Takes the arguments of river scenario and basin.
Runs the river UCIs with HSPF. runs each river segment in the basin. This script is for
       calibration or other special cases only. For scenarios, use run_scenario_river.csh
       Copies a blank wdm to the name riv$riverseg.wdm
       Copies the EOS wdm made with the ETM to a working directory.
       Adds the output of any upstream river reaches to the EOS loads.
       Combines point source, septic, diversions, and atmospheric deposition in to a
       single wdm
       Runs the UCI with HSPF
       Moves the wdm, out, and echo file to the appropriate place

run_scenario_river.csh
Takes the arguments of river scenario and basin.
Modifies UCI parameters for scenarios, generates the UCI, and runs it.
River parameters controlling scour and nutrient release from sediment are reduced or
       increased relative to upstream changes in loads for scenarios. This script is the
       correct one to use for running scenarios rather than a calibration run.
       Copies a blank wdm to the name riv$riverseg.wdm
       Copies the EOS wdm made with the ETM to a working directory.
       Adds the output of any upstream river reaches to the EOS loads.
       Combines point source, septic, diversions, and atmospheric deposition in to a
       single wdm
       Compares the input to the calibration input for each river
       Modifies UCI parameters appropriately
       Generates the UCI
       Runs the UCI with HSPF
       Moves the wdm, out, and echo file to the appropriate place

run_postproc.csh
Takes the arguments of river scenario and basin.
Generates ASCII output from the WDM files. Flags near the top of the script are set to
       specify the output. The user should modify this file as needed.
Available output from land simulation:
       monthly, annual, or average annual: EOF, EOS, or delivered to bay
       daily:           EOF and EOS
Available output from river simulation:
       monthly, annual, or average annual loads and flows
       daily concentrations and flows
       summary statistics of flow, load, and concentration calibrations.




Modified 11/7/2010                                                                         35
run_scenario_postproc.csh
Takes the arguments of river scenario and basin.
Same as run_postproc.csh, except that the flags are preset to the standard average annual
       scenario output. The user should not modify this file.

run_all.csh
Takes the arguments of scenario and basin.
Runs lug, rug, land, etm, river, and postproc for the calibration. Requires that the river
       and land scenarios have the same name and that the river and land basins have the
       same name and are consistent.

run_scenario.csh
Takes the arguments of scenario and basin.
Runs lug, rug, land, etm, river, and postproc for a scenario. Requires that the river and
       land scenarios have the same name and that the river and land basins have the
       same name and are consistent.

summarize_input_aveann.csh
Takes the arguments of river scenario, basin, and first and last year for averaging.
Creates a summary file of the inputs.
Requires that the lug was run at some point in the past with the dates of the control file
       equal to the first and last year for averaging. The land scenario for the lug is the
       land scenario specified for each land use in the river control file.

summarize_output_aveann.csh
Takes the arguments of river scenario, basin, and first and last year for averaging.
Creates a summary file of the outputs.
Requires that the postproc was run at some point in the past with the same averaging
       dates.


6.4.2 Useful
Scripts here are generally modifications of scripts in the ‗standard‘ directory. They are
        kept here so as not to crowd that directory. When you see a ‗*‘ below, it‘s a
        wildcard character, meaning that there are several different scripts that are similar.

Many of the scripts here are one-off scripts and may not do exactly what is expected or
      may have arguments that are different from the help that is printed to screen when
      the script is invoked with no arguments. It is safest to read and understand the
      script before using it.

run_postproc_*.csh
There are many versions of run_postproc in this directory. In many instances it is easier
       to make a copy of run_postproc.csh and set the flags rather than going back in and
       continually modifying the flags.




Modified 11/7/2010                                                                          36
run_postproc_tf_df_del_aveann.csh
This script generates the transport factors, delivery factors, and delivered loads for a
       scenario once the river average annual files and the edge-of-stream average
       annual files are generated. This is useful when the edge-of-stream files are
       generated with the etm.

run_parallel_*.csh
These are for use particularly on the CBP machine. The parallelization setup is fairly
       rudimentary which is appropriate for this class of computing problem. Depending
       on your setup, you may or may not find these useful.

copy_*.csh, move_*.csh
Generally these are to copy or move files or parameters from one scenario to another,
      which saves time in re-running scenarios. These are useful when running parallel
      calibrations of different basins for the most part.

random.ksh
The external transfer module uses a random number generator to specify a distribution
       for BMPs if indicated in the BMP setup files. This is called whenever the ETM is
       invoked.

run_1land.csh, run_1lug.csh
Generally for scenarios, all land uses will be run, but for some applications, it is useful to
      only run a single land use.

run_etm_and_land_and_dat_simultaneously.csh
The ETM and the land and data postprocessors repeat the same time-consuming
      algorithms. This script runs them simultaneously. This should generally be used
      for scenarios. The postprocessor would then be run_postproc_tf_df_del.csh in
      this directory.

run_taus.csh
generates and summarizes the shear stresses for rivers. Needed whenever the hydrology
       is re-calibrated

run*_oneseg.csh
scripts that have the ‗oneseg‘ designation usually take the segment as the argument,
         rather than the basin.

6.4.3 Datascripts
Most time series inputs are supplied to HSPF through WDM files. In the phase 5
implementation, every DSN (data set number used to query the WDM) is related to a
specific data type and no other DSN carries that type of data. For example, precipitation
is always DSN 2000, and nitrate from point sources is always 3003. The name of the
wdm contains the segment to which the datasets apply. Depending on the data type, there




Modified 11/7/2010                                                                          37
can be wdm for each land segment or land-river segment. More information on the
organization of wdms is in section 7.

The scripts above this directory take formatted ASCII data, either from scenario builder
or from other sources, and create WDM data sets.

The scripts above this directory are different from other scripts within phase 5 in that they
do not take meaningful arguments. These scripts are run infrequently and can easily
overwrite previously created datasets if the user is not careful. Instead of taking the
specifiers as arguments, the script is edited directly to supply the necessary variables.
The script is then invoked using ―scriptname.csh GO‖. The script will create the dataset
according to the user-defined variables and then write a file in the same directory as the
created WDMs that describes the dataset, code, time, and user.

These scripts generally have the following structure.
       1. check to make sure that the word ‗go‘ is entered as the only argument.
       2. Section for the user to define variables, which usually include at a minimum
           the name of the code, the name of the input data set, and the name of the
           output data set.
       3. Loop over segments and create a WDM for each. This loop may be internal to
           the fortran program or performed in the script. In most cases, a summary file
           is written.
       4. Echo the variables in the user-definition section back out to a file in the new
           WDM directory.

The scripts may have the user set a subset of the following variables. The meanings are
generally constant among the scripts.
       1. code: the path to the executable. The user would not normally alter this.
       2. pradscen, metscen, atdepscen, etc: The directory name above
           ./input/scenario/climate/datatype/ to be used to store the output.
       3. basin: same meaning as other scripts, the list of segments to operate on.
       4. datasource, version, longfilename, etc, specification of the directory above
           ./input/unformatted/datatype/ where the data are stored and possibly the file
           names associated with the data.
       5. year1, year2: the first and last year to process
       6. scenyear: if the output data set is taking a time series and creating a static data
           set (for example creating the 1995 septic data set from the 1984-2005
           calibration septic data set), this is the year that the scenario represents.
       7. newscen, oldscen: for scripts that copy and modify an existing WDM data
           set, these specify the new scenario name and the scenario to be copied.
       8. rscen: river scenario. This is used to access the geometry files. The data set
           generated can still be used for any river scenario where it is appropriate

Some of the scripts will be used often and without much modification. Point sources, for
example, will be generated based on ASCII output from the scenario builder. The user
will simply modify the specifiers that denote the location of the input data and the



Modified 11/7/2010                                                                        38
directory where the output will be stored. Other scripts, such as the scripts generating
precipitation input, will not be used often, if at all. It is likely that for those types,
additional code modification may be necessary for different data structures. All of the
code that is referenced by the data scripts is above ./code/src/data_input/.

6.4.3.1 precip_and_met directory
create_precip.wdms.csh
This script creates a precipitation data set from individual hourly precipitation files output
from a USGS model.

change_precip_nov_storm.csh
As described in the main documentation, the remnants of hurricane Juan stalled over the
upper Potomac in an event that was not captured sufficiently by rain gauges. The rainfall
has been increased to allow for proper simulation of this important storm. This script
performs those changes on a rainfall WDM data set. Do not invoke this script more than
once as the affects will be multiplicative.

create_met_wdms.csh
Creates meteorological WDMs for wind, solar radiation, dew point, and cloud cover from
the weather station data stored in an input set of WDMS.

add_xyz_met.csh
Adds temperature and evapotranspiration to existing WDMS from the same model that
produced the rainfall.

modify_met_for_jan_1996_storm.csh
As described in the main documentation, a large snow storm in January of 1996 was
almost completely melted by a rainstorm that followed a few days later. The rainfall was
at a higher temperature than the air, which led to faster melting. This script increases the
temperature during the period of rainfall for the affected counties in the Potomac and
Susquehanna drainage. Do not invoke this script more than once as the affects will be
multiplicative.

6.4.3.2 add_atdep_to_precip_wdms directory
add_atdep_to_precip_wdms_calib.csh
adds the calibration atmospheric deposition to existing precipitation WDMS. The
calibration atmospheric deposition is a combination of the output of a regression model
from Penn State for wet deposition and CMAQ for dry deposition

add_atdep_to_precip_wdms_year_scenario.csh
Detrends the calibration data set to the specified year and stores it as a separate scenario.

add_atdep_to_precip_wdms_CMAQ_scenario.csh
Uses output of the CMAQ model to get the reduction ratios from the year 2000 and then
applies those reductions to the detrended data set.



Modified 11/7/2010                                                                           39
add_atdep_to_precip_wdms_percent_of_existing.csh
Multiplies an existing scenario by factors specified in the script and stores it as a new
scenario.

6.4.3.3 diversions directory
create_calib_diversion_wdms.csh
creates the diversion time series based on ASCII files generated by the USGS

create_year_scenario_diversion_wdms_from_calib.csh
copies a given year within the calibration time period to all hydrologic years

create_zero_diversion_wdms.csh
creates wdms with no diversions for running special case scenarios.

6.4.3.4 point_source directory
create_calib_ps_wdms.csh
creates the calibration point sources from a file supplied by scenario builder

create_scenario_ps_wdms_from_calib.csh
copies a given year within the calibration time period to all hydrologic years

create_scenario_ps_wdms_from_csv.csh
creates point source scenario WDMS from a file supplied by scenario builder

6.4.3.5 septic_directory
create_calib_septic_wdms.csh
creates the calibration septic WDMs from a file supplied by scenario builder

create_year_scenario_septic_wdms_from_calib.csh
copies a given year within the calibration time period to all hydrologic years

create_scenario_septic_wdms_from_scenario_csv.csh
creates septic scenario WDMS from a file supplied by scenario builder

6.4.3.6 observed
Scripts in this directory are generally used to count observations and split observations
for validation data set purposes. They are utilities that are not used in the general course
of running scenarios and so they should be used with caution. Before running any of
these, it would be prudent to check the code that it is running to be sure that you
understand it.

count*.csh
These scripts are generally used to count all of the observations of a particular type
between 2 years. The output is a listing of segments and the number of observations




Modified 11/7/2010                                                                          40
between the given years. The count_goodobs.csh script gives only the number of non-
limit-of-detect observations.

split_*.csh
Splits data sets according to different methodologies. The final method was to only use
30 sites as validation sites and split them such that the 1991-2000 data set was intact in
the calibration. See main documentation for more details on the method used. Match the
code to the script to see what each one does.

6.4.4 make_seglists
basingen.csh
This script takes the arguments of a river scenario and the unique id of a river segment
and generates three files in the ./run/seglists/ directory listing the land segments, river
segments, and calibration stations. These files are used as the basin identifier for
subsequent scripts. The unique ID is the four digit number in the middle of the river
segment name. The river scenario is just used to reference the correct geometry files. A
basin can be used with any river segment that uses the same geometry files. See section
8.3.1 for description of these files.

riverinfo.csh
Takes the arguments of a river scenario and the unique id of a river segment. The river
segment is just used to reference the geometry files. Riverinfo has several types of
output, which are put out to the screen successively. It may be necessary to redirect
output to a file so that it doesn‘t scroll too far up. The outputs are as follows: 1) a list of
river segments upstream of the input segment. 2) a list of land segments that drain to the
upstream watershed. For each land segment there is a listing of total acreage within the
basin in 1000 acres, percent of the land segment that is inside the basin, and percent of
the basin that is made up of the land segment. 3) a list of land-river segments that share
land segments with the upstream watershed. The first column has a U for upstream, D
for downstream, and blank for neither. The second and third column are the river and
land segments, the last three columns are the same as the last three columns in #2 above.

divide_rseg_by_stream_order.csh
Arguments: river scenario and river segment list name. The river segment is just used to
reference the geometry files. This script divides the input stream into different order
streams. The stream order is defined by the modeled stream network and would not
necessarily resemble the stream order estimated from the field. This script may be
helpful in defining dependencies for a parallel run.

splitsegs.csh
Arguments: segment list name and number of splits. This script simply splits the file into
arbitrary lists. Useful for parallelization.




Modified 11/7/2010                                                                           41
6.4.5 calibration
The scripts in this section run the automated calibration of the phase 5 model. A short
description of these directories and scripts would not be sufficient or helpful. A full
discussion and step-by-step directions for running a calibration is in section 11.

6.4.6 fragments
Many scripts in other directories call or source files from the ‗fragments‘ directory to set
internal variables.
set_landuse: sets the variables perlnd and implnd which contain listings of all the
pervious and impervious land uses respectively
set_hspf: sets the variables hspf to the full path of the HSPF executable
set_quiet_hspf: same as above except this version of HSPF has less output
set_tree: sets the variable tree to the relative path between the temporary running
directories and the root directory for the phase 5 system
abs_tree: sets the variable tree to the absolute path to the root directory for the phase 5
system. This was the original setup and some scripts still require the absolute path.

6.4.7 link_p5_to_wqm
Scripts in this directory are used for various linking purposes, watershed model to
estuarine model, atmospheric deposition to estuarine model, and watershed model to
other data outputs. Generally, to link the watershed model to another model, the user just
needs the physical linkage between watershed and estuarine segments, and the logical
linkage between watershed and estuarine model variables, described in sections 8.3.1 and
8.3.2 respectively.

The estuarine water quality model input has three parts: nonpoint source, point source,
and atmospheric deposition. A single script generates the point source and nonpoint
source files, while a separate script generates the atmospheric deposition input files.

run_nps_and_ps_to_wqm13k.csh
run_nps_and_ps_to_wqm57k.csh
run_nps_to_wqm13k.csh
run_nps_to_wqm57k.csh
run_ps_to_wqm13k.csh
run_ps_to_wqm57k.csh
All of the above scripts operate in the same manner. The user can load the 57,000 cell
estuarine water quality model or the 13,000 cell estuarine water quality model. The user
can choose to generate point source only, nonpoint source only, or both. All scripts take
the arguments river scenario, point source method, start year, and end year. River
scenario is just that. Start year and end year are the start and end year for the estuarine
model input. They do not have to be consistent with the years in the river control file, but
the scenario must have been run in the watershed model with the river scenario years
inclusive of the start year and end year. Point source method is the method that is used
to load the point sources into the estuarine model. It can take the values ‗hcell‘, ‗wcell‘,
or ‗lrseg‘. ‗hcell‘ means that the point sources are organized into WDMS using
hydrodynamic cell numbers. ‗wcell‘ means that they are organized using estuarine water


Modified 11/7/2010                                                                        42
quality model cell numbers. ‗lrseg‘ means that the point sources are not divided up by
estuarine cell number and that the fraction of each land-river segment flowing to each
estuarine model cell determines the fraction of the point sources within that land-river
segment.

factor_scenario_run_nps_and_ps_to_wqm57k.csh
quick_kluge_nps_and_ps_to_wqm57k.csh
The factor_scenario script is a quick method of generating an estuarine model input as a
fraction of a current watershed model run without having to run the watershed model
over again. The user sets the factors in an input file. The quick_kluge script is just that.

run_atdep12_to_wqm57k_calib.csh
run_atdep12_to_wqm57k_CMAQ_scenario.csh
run_atdep12_to_wqm57k_percent_of_CMAQ_scenario.csh
run_atdep12_to_wqm57k_year_scenario.csh
The above scripts generate atmospheric deposition inputs to the estuarine water quality
model. ‗Atdep12‘ refers to the 12 kilometer version of the CMAQ model. These scripts
operate similarly to those in section 6.4.3.2. The ‗calib‘ script generates the calibration
data set, the ‗year scenario‘ script creates a detrended data set from the calibration data
set. ‗CMAQ_scenario‘ creates atmospheric deposition from a specific CMAQ scenario,
and ‗percent‘ is a way to get a quick atmospheric deposition scenario that is a percentage
of any other scenario.

Summarize_nps_and_ps_from_wqm57k_inputs.csh
Used to check the inputs for a scenario. Generates summarized ASCII files.

old_scripts/
Previous versions of the linkage to the wqm have been used to load geographically
specific models of the Potomac, York, and James, and also to generate monthly loads for
academic studies. The code is somewhat out of date, but these show how the code can be
modified to generate output for different models or uses.

6.4.8 run24
The scripts in this area are used to run the model in parallel on the CBP 24-processor
machine with the SLURM job scheduler.

7 Input Data Organization
HSPF in general and phase 5 in particular require extensive input data. It is important
that the input is well-structured so that it is understandable and easily updated for
scenarios.

Inputs are either in ASCII or WDM format. The output of scenario builder or other
models is in ASCII format. The time series required by HSPF to be in WDM format start
out as ASCII data and are converted before use.

ASCII inputs are in several different formats. Most fall into one of three categories:


Modified 11/7/2010                                                                         43
   1. table format. Each row corresponds to a segment with columns in the table
      corresponding to land uses or months
   2. single data column format. Each row has only one datum, but may have several
      columns to identify that datum.
   3. separate file for each segment, rows are used for time. Hourly and daily time-
      series data are usually in this format.

All model inputs are under the ./input/ directory. The ./input/ directory is divided into
four general categories, which are given their own subheadings below.

7.1 calib
All calibration data, meaning data to compare the output of the watershed model against,
is kept here. The information this directory pertains only to the calibration and is not
used in scenario mode. There are two major types of calibration data, observed in-stream
data and land export targets.

./input/calib/observed/
The observed river flow and concentration data are stored in the
./input/calib/observed/alldata/ directory. There is a subdirectory for each data type
(FLOW, TOTN, etc). Within each subdirectory are separate files for each river segment
that has this type of data. The files are comma-delimited with the following columns:
year, month, day, hour, minute, observed value in cfs or mg/l, flag, and station name.
The flag can either be ‗-‗ which means the observation is valid, or ‗<‘ which means the
observation is at the limit of detection and is handled differently in the calibration
routines. Other flags cause an error.

The ‗valid‘ and ‗calib‘ directories are parallel to the ‗alldata‘ directory and contain the
calibration and validation data sets. Various methods were used to generate the
validation and calibration data sets from the ‗alldata‘ directory. Any changes or additions
should be made to the ‗alldata‘ data set, and then the calibration and validation data sets
should be generated from this.

The ‗estimator‘ directory has output of the USGS ESTIMATOR model which is used in
the estimation of regional factors during the calibration process.

./input/calib/target/
The targets are a priori estimates of annual export of TN, TP, and sediment for each land
use and land segment. They are based on literature and other sources and vary spatially
with input loads. See the main Phase 5 documentation for more details

7.2 param
The model parameters, such as hydrologic and chemical rate constants are stored above
this directory. All 3-character directories above the ./input/param/ directory are separate
directories for each land use. Within each land use directory, there is a directory for
different parameter scenarios. The river parameters are stored in different parameters
above the ./input/param/river directory. For each scenario above the land and river


Modified 11/7/2010                                                                          44
directories, there are one or more comma-delimited files for each active module, which is
named after than module. For example, there is a file called PWATER.csv that is read
when the PWATER module is active in that scenario. Each of these files stores table
names in the first row and variable names in the second row. The remaining rows each
contain the parameters for a separate land or river segment. The table and variable names
expected by the phase 5 software are stored in the directory ./config/catalog/modules/.
Also in the ./input/param directory are regional and sub-grid transfer factors above the
‗transport‘ directory and scripts to perform operations on the parameters above the
‗scripts‘ directory.

Generally for scenarios, the parameters will be kept constant under changing input loads.
In calibration mode, the parameters will be changed, but normally through an automated
process. The user is not generally required to make modifications in this directory except
for the execution of scripts.

7.3 Unformatted
The ‗unformatted‘ directory contains data that is received from other data sources that is
not formatted correctly to go into the phase 5 model. Most of these data are formatted for
use in phase 5 through the use of scripts that create wdms. Precipitation, atmospheric
deposition, meteorology, diversions, point sources, and septic loads are in this category.
Other data sets may require some spreadsheet manipulation before being moved to the
proper place in the ./input/scenario/ directory. More information on some of these data
types and how they are formed into wdm data sets can be found in section 6.4.3 on data
scripts.

7.4 Scenario
The ./input/scenario/ directory contains all scenario input data. ASCII data types are
created above this directory either by spreadsheet manipulation or by direct scenario
builder placement. WDM data types are created by data scripts. ASCII data types are
placed into subdirectories according to file type, with scenarios differentiated by file
name. ASCII data types will have all segments in a single file. WDM data types with
time series data will have a separate file for each segment with scenarios differentiated by
subdirectory name. The control file for the scenario (see section 5 for details) contains
specifiers for each data type.

There are four subdirectories: ‗land‘ and ‗river‘ contain data that pertain to land and river
scenarios respectively. ‗climate‘ contains data that pertain to both land and river
scenarios. ‗wqm‘ contains data that is used to modify watershed model output for input
to the estuarine water quality model. These are discussed in more detail below.

7.4.1 Climate scenario data directory
Above the ./input/scenario/climate/ directory are two subdirectories, ‗met‘ containing
meteorological WDMs, and ‗prad‘ containing precipitation and atmospheric deposition
WDMs. Each directory above the ‗met‘ or ‗prad‘ directory contains a separate climate
scenario for all land segments. River segments are assigned a land segment WDM so that
this large dataset did not have to be created for each river segment. The spatial resolution


Modified 11/7/2010                                                                         45
of the data and the relative insensitivity of the river simulation to these inputs justify this
approach. The linkages between river segment and land segment-based WDMs are
contained in the files ./config/catalog/geo/geoscen/river_met_wdm.csv and
./config/catalog/geo/geoscen/river_prad_wdm.csv.

The meteorological scenario above the ‗met‘ directory will be the same for every land or
river scenario other than climate change scenarios. Use the same meteorological scenario
as in the calibration that any given land or river scenario is based on. Within each
scenario directory is either a ‗NOTES‘ file or an ‗AutoMetNotes‘ file detailing the
creation of each particular data set. Each wdm contains daily potential
evapotranspiration, dew point, and cloud cover, and hourly wind speed, solar radiation,
and air temperature.

The precipitation and atmospheric deposition above the ‗prad‘ directory will change for
either climate change scenarios or atmospheric deposition scenarios. Most scenarios
should not use the calibration data set as this has an observed trend in atmospheric
deposition. The user should select or generate atmospheric deposition appropriate to the
particular scenario. Within each scenario directory is either a ‗NOTES‘ file, an
‗AutoAtdepNotes‘ file, or both detailing the creation of each particular data set. Each
wdm contains hourly precipitation and daily atmospheric deposition of wet nitrate, dry
nitrate, wet ammonia, dry ammonia, wet phosphate, wet organic nitrogen, and wet
organic phosphorus.

7.4.2 Land scenario data directory
With the exception of the spec_flags directory, all of the directories above
./input/scenario/land/ are similar in their organization. Each contains multiple comma-
delimited files that are different snapshots of the watershed for a give point in time or
scenario. A land scenario may reference several of them representing different points in
time and interpolate between them, or may reference a single file representing a data set
for that data type that is static through time.

If the land use and land segment combination does not exist for that scenario or time, this
should be indicated by a ―-9‖ in the value field. This has a different meaning than an
entry of ―0‖, such as when there are no manure applications for a particular land use.

These files are created by the Scenario Builder software. See the scenario builder
documentation for a much more detailed treatment of these files.

The various file types are formatted as follows
annual_uptake/max_uptake_UptakeScenario.csv
landseg,landuse,nutrient,max_uptake
A24047,hyw,Nitr,150
A24041,hyw,Nitr,150
…
Nutrient = {Nitr,Phos}
Max_uptake is in lbs per acre per year.



Modified 11/7/2010                                                                           46
monthly_fraction_uptake/uptakecurve_MonthlyFractionScenario.csv
landseg,landuse,nutrient,jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec
A10001,hom,Phos,0,0,0.03,0.19,0.37,0.27,0.06,0.07,0.01,0.00,0.00,0
A10001,hwm,Nitr,0.01,0,0.05,0.06,0.1,0.26,0.3,0.12,0.04,0.03,0.03,0.01
A10001,hyw,Nitr,0,0,0.00,0.06,0.25,0.00,0.04,0.06,0.29,0.29,0.00,0
…
Nutrient = {Nitr,Phos}
Monthly Fractions of uptake must add up to 1 for each land segment, land use, and
nutrient combination

crop_cover/crop_cover_CoverScenario.csv
landseg,landuse,jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec
A24001,pas,0.29,0.29,0.28,0.28,0.42,0.67,0.57,0.60,0.68,0.34,0.32,0.32
B51019,nal,-9,-9,-9,-9,-9,-9,-9,-9,-9,-9,-9,-9
…
Monthly percent cover by land use and land segment, -9 means that the combination does
not exist.

detached_sediment/detached_sediment_DetsScenario.csv
landseg,landuse,jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec
A24001,pas,0,0,0.1,0,0,0.3,0,0.05,0,0,0,0
B51019,nal,0,0,0,0.15,0,0.2,0,0,0,0,0,0
…
Monthly tons of new detached sediment due to land plowing operations.

fertilizer/fertilizer_FertilizerScenario.csv
manure/manure_ManureScenario.csv
legume/legume_LegumeScenario.csv
landseg,landuse,nutrient,jan,feb,mar,apr,may,jun,jul,aug,sep,oct,nov,dec
A10001,alf,orgp,0,0,0,0,0,0,0,0,0,0,0,0
A10001,alf,po4p,0,0,0,62.0,0,0,0,0,0,0,0,0
A10001,hyw,no3n,0,0,1.07,0,2.76,0,0,0,0,8.2,0,0
A10001,hyw,po4p,0,0,5.34,0,31.20,0,0,0,0,32.7,0,0
A10001,lwm,po4p,0,0,0.78,14.59,31.74,0,0,0,0,18.11,1.20,0
…
Monthly pounds of fertilizer or manure applied, or nutrient fixed by crops
Nutrient = {no3n,nh3n,orgp,po4p,orgp}

Spec_flags/SpecFlagScen/
landseg,for,hvf,pas,npa,urs,afo,trp,alf,nal,hyw,nhy,hyo,hom,nho,hwm,nhi,lwm,nlo,pul,puh,ext,bar,iml,imh
A10001,,,MVR,MVR,FV,,MV,FVTUR,FVTUR,FMVLTUR,FMVLTUR,VTU,FVPLTUR,FVPLTUR,FMVPLTUR,FMVPLTUR,FMVPLTUR,FMVPLTUR,FV,FV,,,,
…

Flags for each land use and land segment detailing the types of data available and the
special actions to implement. The flags may be in any order.
V = variable cover in sediment (crop_cover)
P = plowing (detached_sediment)
F = fertilizer
M = manure
L = fixation for legumes
T = variable total annual uptake target (max_uptake)
U = variable monthly fraction of uptake target (monthly_fraction_uptake)



Modified 11/7/2010                                                                                                  47
R = removal of refractory organic N to simulate harvest
(./input/param/landuse/ParamScenario/RORGN.csv)

Fertilizer, manure, legume fixation, and detached sediment can only be simulated through
special actions. Crop cover and uptake can be modified in the UCI using the files in
./pp/param/landuse/ParamScenario/.

7.4.3 River scenario data directory
The directories above ./input/scenario/river/ contain three different types of organization.

The ‗land_use‘ and ‗bmpacres‘ directories are similar to the land data directories with
each containing multiple comma-delimited files that are different snapshots of the
watershed for a give point in time or scenario. A land scenario may reference several of
them representing different points in time and interpolate between them, or may reference
a single file representing a data set for that data type that is static through time. These
files can be produced directly by the scenario builder.

The ‗div‘, ‗ps‘, and ‗septic‘ directories are similar to the climate data directories, with
each containing several subdirectories designating different scenarios. Each subdirectory
contains WDMs for each land-river segment or river segment. These file can be
produced using data scripts from files produced by the scenario builder.

The ‗bmptypes‘ directory contains different scenarios of specifications for the BMP
simulation. These files can be produced directly by the scenario builder.

7.4.3.1 ASCII river scenario data
The various file types and directories are formatted as follows
land_use/land_use_LandUseScenario.csv
riverseg,landseg,afo,alf,bar,ext,for,hom,hvf,hwm,hyo,hyw,imh,iml,lwm,nal,nhi,nho,nhy,nlo,npa,pas,puh,pul,trp,urs,wat
DE0_3380_0000,A10001,8,170,61,18,2750,1511,28,3076,109,245,445,489,6869,0,0,0,0,0,0,225,800,2051,0,2,94
DE0_3410_0000,A10001,19,405,126,45,29464,3551,298,7228,259,583,756,464,16141,0,0,0,0,0,0,534,1713,2630,0,4,2257
DE0_3790_0000,A10001,11,289,188,14,6828,2071,69,4215,185,416,2138,1571,9413,0,0,0,0,0,0,381,1845,7051,0,2,487
…
Land uses acreages for each land-river segment.
Land use columns may be in any order.

bmpacres/bmpacres_BMPAcresScenario.csv
rseg,lseg,Shortname,Landuse_Type,Acres,constrained
EL0_4560_4562,A10001,ForestBuffersN ,hwm, 51.7,y
EL0_4560_4562,A10001,ForestBuffersN ,nhi, 292.,y
EL0_4560_4562,A10001,ForestBuffersN ,hom, 24.1,y
EL0_4560_4562,A10001,ForestBuffersN ,nho, 136.,y
…
Acres of each BMP type for each land use in each land-river segment
Shortname must correspond to a shortname in the file
       bmptypes/BMPtypeScenario/BMP_descriptions.csv
Contrained is a flag to determine if this particular BMP in this location is constrained by
       the max_implement value in the file
       bmptypes/BMPtypeScenario/BMP_descriptions.csv


Modified 11/7/2010                                                                                                     48
7.4.3.2 WDM river scenario data
Point Source WDMs in the ‘ps’ directory
Each directory above the ‗ps‘ directory contains a different point source scenario. Within
each scenario is a separate WDM file for each land-river segment. Each WDM contains
variables for flow, heat, ammonia, nitrate, refractory organic N, phosphate, refractory
organic P, biochemical oxygen demand, clay, dissolved oxygen, and refractory organic
carbon. Also in that directory is an AutoNotes file detailing how this data set was created
and an lrseg_ps_loads.csv file with the monthly loads by land-river segment. For more
details on creating the data set see section 6.4.3.4

Diversion WDMs in the ‘div’ directory
Each directory above the ‗div‘ directory contains a different water diversion scenario.
Within each scenario is a separate WDM file for each river segment. Each WDM
contains two separate flow variables: one for industrial diversions, and one for
agricultural diverions. Also in that directory is an AutoNotes file detailing how this data
set was created. For more details on creating the data set see section 6.4.3.3.

Septic WDMs in the ‘septic’ directory
Each directory above the ‗septic‘ directory contains a different septic loading scenario.
Within each scenario is a separate WDM file for each land-river segment. Each WDM
contains a variable for nitrate only. Also in that directory is an AutoNotes file detailing
how this data set was created. For more details on creating the data set see section
6.4.3.5.

7.4.3.3 bmptypes directory
The bmptypes directory contains several subdirectories, each containing a full description
of the BMPs allowed for any give BMPtype scenario as specified in the river control file.
Several files make up a full description. The user can be creative with the population of
these files to simulate a wide range of BMPs. For example, if two states or two counties
have slightly different implementations of the same BMP, then simply splitting the BMP
type into two separate BMPs will allow the user to incorporate that difference. The user
can also use the HGMR file, as described below, to simulate spatially varying effects of
the same BMP.

BMP_descriptions.csv
BMP,Shortname,Landuse_Type,HGMR,Max_Implement,TN,TP,SED
Conservation Plans,ConPlan,Hi_Till,all,1,0.08,0.15,0.25
Conservation Plans,ConPlan,Low_Till,all,1,0.03,0.05,0.08
Conservation Plans,ConPlan,allhay,all,1,0.03,0.05,0.08
Conservation Plans,ConPlan,Pasture,all,1,0.05,0.1,0.14
Animal Waste Management Livestock,AnimalWasteMngt,afo,all,1,1,1,0
BMP                    Descriptive name of the BMP – no commas allowed, 60 characters
                              maximum length
Shortname              Short unique nickname to be used in all other files. No commas,
                              no spaces, 15 characters max.




Modified 11/7/2010                                                                        49
Landuse_type    land use to which this BMP specification applies, can be a 3-
                       character identifier or a collective identifier from
                       landuse_categories.csv
HGMR            Hydrogeomorphic region. Some BMPs act differently in different
                       regions. This is a classification system according to
                       morphology, but any classification system can be
                       substituted with changes in the HGMRs.csv and
                       lrseg_HGMRs.csv files. A single HGMR or ‗all‘ can be
                       specified here.
Max_Implement   Maximum fraction of any given land use in a land-river segment
                       that can receive this BMP. This specification can be
                       circumvented for specific land-river segments in the
                       bmpacres file.
TN,TP,BOD,SED … The nominal fractional reduction from the implementation of this
                       BMP for each of the listed constituents. The list of possible
                       constituents to reduce is in
                       ./config/catalog/iovars/iovarscen/bmp_constituents and the
                       effects of these constituents on river variables is in
                       ./config/catalog/iovars/iovarscen/rchres_in

Note that a unique BMP identifier is a 3-part key consisting of shortname, landuse_type,
and HGMR for the purposes of setting nominal reduction efficiencies. Other files are
keyed to the BMP shortname so that it would be necessary to create a new BMP to
differentiate for some files.

HGRMs.csv
APCN
APSN
…
A listing of all possible hydrogeomorphic regions in the model domain

lrseg_HGRMs.csv
river,land,final HGMR
DE0_3380_0000,A10001,CPUN
DE0_3410_0000,A10001,CPUN
…
The hydrogeomorphic region for each land-river segment

landuse_categories.csv
catname,number of land uses, lu1, lu2, etc             ***
forest,2,for,hvf
urban,4,imh,iml,puh,pul
Impervious_Urban,2,imh,iml
…
A listing of each 3-character land use that makes up each land use category to be used in
BMP_descriptions.csv, hydrologic_parameters.csv, and
random_distribution_parameters.csv
catname                category name, no spaces or commas, 20 charaters max
number of land uses number of 3-character land uses in this catname


Modified 11/7/2010                                                                     50
lu1, lu2, etc          comma separated listing of land uses in this category

additive_bmps.csv
BMP,                     Shortname,         ForestBuff, GrassBuff, ContinuousNT
Forest Buffers,          ForestBuff,              1,        1,          0
Grass Buffers,           GrassBuff,               1,        1,          0
Continuous No-Till,      ContinuousNT,            0,        0,          1
Some sets of BMPs can be applied to the same acre of land and some are mutually
exclusive. additive_bmps.csv is a matrix of which can not be applied to the same acre of
land. A one means that the BMPs are mutually exclusive and a zero means that they are
not. BMP and shortname must be exactly the same as in BMP_descriptions.csv and both
files must agree on the number of BMPs. additive_bmps.csv must be symmetric about
the main diagonal and have all ones on the main diagonal.

hydrologic_parameters.csv
BMPs perform differently in different hydrologic states. Many fail or partially fail when
a high recurrence interval storm occurs. This table allows the user to specify a model
relating return frequency of the unit hydrograph to a reduction in BMP efficency
BMP,Shortname,Landuse,HGMR,Const,hydtype,hydparm1,hydparm2,hydparm3 ***
all, all,       all,   all, all,    1,      5,      17,       0.2
Animal Waste Management,AnimalWasteMngt,afo,all,all,1,25,32,0.2

BMP , shortname, landuse, and HGMR all have the same meaning as in previously-
described tables. ‗all‘ is acceptable for each of the columns. Any previous entry can be
superseded by a later one. The second row in the table above supersedes the
parameterization of the hydrologic model in the first row for that particular BMP and
land use. ‗Const‘ is for constituent, which can be any of the constituents that are in the
columns of BMP_descriptions.csv. ‗all‘ is also acceptable here. The ‗hydtype‘ and
‗hydparm‘ variables specify the particular model.

hydtype         0 = no model, always use maximum efficiency
                       hydparm1-3 = no effect
                1 = michaelis-menton model
                       hydparm1 = return frequency below which there is no reduction
                       hydparm2 = half-saturation constant
                       hydparm3 = minimum hydrologic factor under any storm

random_distribution_parameters.csv
BMP,Shortname,Landuse_Type,HGMR,Const,DistType,DistParm1,Distparm2,Distparm3 ***
all,all,all,all,all,0,0,0,0
The result of the random model is used as a multiplier for the BMP efficiency from the
BMP_descriptions.csv and any hydrologic effects. The table format is similar to
hydrologic_parameters.csv above with the following models available:
DistType        0 = no model, always use default efficiency
                       Distparm1-3 = no effect
                1 = uniform distribution applied daily
                       Distparm1 = minimum multiplier
                       Distparm2 = maximum multiplier
                2 = uniform distribution as a fraction of the base BMP efficiency


Modified 11/7/2010                                                                       51
                      Distparm1 = fraction
                      Distparm2 = 0 – no out of bounds checking
                                     1 – constrain final to between zero and 1
               4 = same as model 1 except that each BMP has a single random value
                      throughout the simulation rather than a daily random value
               5 = same as model 2 except that each BMP has a single random value
                      throughout the simulation rather than a daily random value


7.4.4 Wqm scenario data directory
See section 10.11 for greater detail about creating the input for the 57,000 cell estuarine
model. Most inputs can be created without data from this directory as they come directly
from the output of the watershed model.

./input/scenario/wqm/marsh/ Marsh loads of sediment and organic carbon are
significant for a small group of segments in the estuarine model. These inputs are
available in this directory.

./input/scenario/wqm/RiverFactors/ In certain circumstances it may be useful to run
estuarine model scenarios that are created by multiplying existing watershed model
scenarios by a particular factor. For example, to determine the water quality response in
the tidal water to a 30% reduction in the Patuxent and York, it would be difficult to create
a watershed model run that reached exactly those numbers. The file in this directory in
connection with scripts in section 10.11 can quickly produce those inputs.

Within each subdirectory above Riverfactors are the following files:
       riverfactors.csv = listing of river segments and factors to multiply each constituent
       investigate_factors.csv = helps the user determine factors for each river system
       calc_factors.xls = Once river-based factors are determined, creates the
               riverfactors.csv file.

8 Detailed documentation of the config directory
Section 4.3 gave a listing and short description of the directory tree above ./config/. This
section will give more details on many of the configuration files.

8.1 Working with basins
Section 6 dealt with performing operations on the watershed model using scripts. Almost
all of these scripts required a geographic extent on which to perform the operation.
Scripts requiring a geographic extent or segment list take an argument with the name of a
file that contains the list of segments to work on. All such files are stored in the directory
./config/seglists/. The file name extension is related to the contents of the file:
         .land land segments
         .riv    river segments
         .calib a subset of river segments that have calibration data




Modified 11/7/2010                                                                         52
For example, if you give a script the basin argument MyRiver and the script is expecting
a list of river segments, then the file ./config/seglists/MyRiver.riv must exist.
Furthermore it must contain the list of segments set to the variable name ‗segments‘
The segment list file WM3_4060_0001.riv is as follows:
set segments = ( WM0_3881_3880 WM1_3882_3880 WM3_3880_4060 WM3_4060_0001 )
Notice that it is in upstream to downstream order. If this is not the case, certain
operations will produce errors or incorrect results. There are multiple possible correct
orders in most cases. Segment lists generated through basingen.csh are always in a
correct order. Section 6.4.4 is a discussion of scripts used to create segment lists.

A few scripts and programs in the ./config/seglists/ directory exist that will help the user
manage segment lists.
add                     - adds two seglists together, listing each segment only once.
subtract                - subtracts one segment list from another
rename.csh              - renames the .riv .land and .calib files
reverse_seglist.exe     - puts the segment list in reverse order

8.2 Control
The land and river control files were detailed in section 5. The calibration control files
are too complex for this section and will be dealt with under the individual module
sections in the calibration documentation

8.3 Catalog
The catalog is the specification of the model setup. Generally, the information on the
model domain, the simulated variables, and the linkages between them can be found here.

8.3.1 Geo
The ./config/catalog/geo/ directory contains files that list the segments in the model
domain, some of their properties, and linkages between them. Directly above the geo
directory are one or more subdirectories that contain alternate model domains that can be
specified in the control files. Separate geo scenarios should only be used when using an
alternative segmentation, at higher or lower order HUC, for example. Subsets of existing
segmentation should be handled through the ./config/segslist files and the basin argument
to scripts.

Files within a geo scenario:
landnames.csv – a listing of the land segments and an alternative identifier to make some
       outputs more readable. In the case of the phase 5 default segmentation, the
       alternative identifier is the county name.
rivernames.csv - a listing of the river segments and an alternative identifier.
land_water_area.csv – list of each land-river segment and the area in acres of the land-
       river segment. At the end of this file are land river segments that have no
       physical area. These are purely for use in the point source calculation, where a
       land segment may be responsible for a load that is discharged to a neighboring
       watershed.



Modified 11/7/2010                                                                           53
watershed_area.csv – list of all river segment with the area of the entire upstream
        watershed from the pour point of each segment in square miles. Used only for
        calculating river baseflow and stormflow separation for the hydrology calibration.
river_met_wdm.csv – list of meteorological wdms to use for each river segment
river_prad_wdm.csv – list of precipitation and atmospheric deposition wdms to use for
        each river segment
calibsites.csv – list of river segments with calibration data. This is not used for many
        functions and should be phased out as it is more robust to check the observed data
doubles.csv – list of ‗confluence‘ segments with a downstream ID of ‗0003‘ (see section
        2.2.5.2. When a river gage is located immediately downstream of a confluence, it
        is necessary to combine the upstream river‘s flow and water quality constituents
        into a confluence segment, which does not exist physically, but has output in the
        same format as a regular river segment. This file lists confluence segments and
        the segments that are directly upstream of them.
cal5_river_names.csv – The matlab-based GUI used for visualizing the calibration uses
        this file to produce titles. The names are more readable and also include
        confluence segments
p5_to_ch3d.prn, p5_to_57k_wqm.prn – linkage files between the watershed model and
        the estuarine hydrodynamic (ch3d) and water quality (57k) models. Each river
        segment in the Chesapeake watershed with a downstream ID of 0001 or 0000
        empties into the tidal water and must be represented in the file. Any other
        downstream ID indicates that the segment is not the terminal segment and should
        not be included in these files. The two files differ only in the estuarine model
        numbering system. The files have the following structure:
        cell – estuarine model identifier
        number of pairs – number of river segments with a downstream ID of 0001 plus
                 the number of land-river segments where the river segment has the
                 downstream ID of 0000 that load completely or partially into this cell
        factor 1 – for first pair, the fraction of the river segment or land-river segment
                 that goes to this cell
        land segment 1 – for the first pair, the land segment if this is a land-river
                 segment, or ―0‖ if it is a river segment
        river segment 1 – for first pair, the unique ID of the river segment
        (repeat factor, land segment, and river segment for all pairs)

Other files exist in this directory that are used for one-off programs or are useful files to
pull into spreadsheets. Only the above files are necessary. All files must be consistent.
For example, if a land segment exists in land_water_area.csv, it must also exist in
landnames.csv.

8.3.2 iovars
The files above ./config/catalog/iovars/ contain input and output variables for the land,
river, and estuary and the linkages between them. As with the geo specifications, there
can be multiple subdirectories above iovars so that multiple types of models may be run.
For example, the phase 5 model can be run mostly with AGCHEM nutrient simulation or




Modified 11/7/2010                                                                          54
completely with PQUAL nutrient simulation by changing the iovar scenario in the river
control file.

The following sections detail the files within an iovars scenario:

8.3.2.1 Files containing variable definitions
perlnd – space delimited file specifying the variables that each land use type should write
       out to a WDM.
       Column         Description
       1-4            WDM data set number (dsn)
       7-23           HSPF variable that goes into that dsn
       26-29          variable name in the WDM
       31-67          text description of that variable
       68 and greater are a listing of the land uses to which this variable applies. The
               land uses are identified by their 3-character names separated by one space.
       Note that some WDM constituents can have more than one HSPF source variable.
       For instance surface nitrate is PQUAL SOQO 2 for land uses running PQUAL
       exclusively and NITR TSNO3 1 for those simulated with AGCHEM

implnd – identical in construction to the perlnd file, but with implnd HSPF and WDM
      variables

rchres_in – space delimited file specifying variables that are read into a river simulation
       from a WDM.
       Column         Description
       1-6            HSPF river module to which the variable applies
       8-10           WDM data set number (dsn)
       14-30          HSPF river variable to read into
       35-38          WDM variable name
       42-44          BMP type that applies to the variable (see section 7.4.3.3) and
                              description of bmp_constituents below
       52+            text description of the variable

rchres_out_3x, rchres_out_4x – space delimited files specifying the variables to be
       output to the WDM. The ‗3x‘ file is for rivers with 3 exits. The exits are drinking
       water diversions, agricultural diversions, and flow to the next downstream
       receiving water. The fourth exit for ‗4x‘ files is pre-calculated reservoir output.
       68 and greater are a listing of the land uses to which this variable applies. The
               land uses are identified by their 3-character names separated by one space.
       Column         Description
       1-6            HSPF river module to which the variable applies
       8-10           WDM data set number (dsn) of the river variable to export
       14-30          HSPF variable name to of the river variable to export
       35-38          WDM name of the river variable to export
       42 and greater is a text description of the variable



Modified 11/7/2010                                                                       55
data_dsns – space delimited files specifying the variables to be output to the WDM. The
      ‗3x‘ file is for rivers with 3 exits. The exits are drinking water diversions,
      agricultural diversions, and flow to the next downstream receiving water. The
      fourth exit for ‗4x‘ files is pre-calculated reservoir output.
      68 and greater are a listing of the land uses to which this variable applies. The
              land uses are identified by their 3-character names separated by one space.
      Column            Description
      1-6               HSPF river module to which the variable applies
      8-10              WDM data set number (dsn) of the river variable to export
      14-30             HSPF variable name to of the river variable to export
      35-38             WDM name of the river variable to export
      42 and greater is a text description of the variable

piqual – listing of the parameters simulated by PQUAL and IQUAL. The parameters
       listed should be consistent in name and order through this file, perlnd, implnd, and
       quals_to_special_action_species in this directory and all of the parameter files
       described in section 7.2. All lines that are not blank or a comment, indicated by
       three consecutive *‘s, should have a number and a parameter. This file is comma
       delimited with the first column being the index and the second being the variable
       name.

bmp_constituents – list of constituents allowed in the bmp type specifications file
     BMP_descrptions.csv in the scenario input. See section 7.4.3.3 for more
     information on the bmp type specifications. Any line that is not blank space or a
     comment (containing three consecutive *s) is interpreted as a BMP constituent

special_action_species – listing of the species that may occur in special action files such
       as those in section 7.4.2. All lines that are not blank or a comment, indicated by
       three consecutive *‘s, should have a parameter.

8.3.2.2 Linkages between variables
land_to_river, septic_to_river, atdep_to_river, pointsource_to_river – space
       delimited files specifying linkages between land and river variables. ―land‖ is for
       both perlnd and implnd, ―septic‖ and ―pointsource‖ are self-explanatory, ―atdep‖
       is atmospheric deposition of nutrients. The specifications end when a literal ‗end‘
       is found in the first three columns of the file. Any text after that is not read, so it
       can be used for comments.
       Column          Description
       1-6             HSPF river module to which the variable applies
       10-12           WDM data set number (dsn) of the receiving river variable
       16-19           WDM variable name of the receiving river variable
       23-25           WDM data set number (dsn) of the sending land variable
       29-32           WDM variable name of the sending land variable
       35-44           multiplication factor. This is generally used for unit or type
                                conversions that apply generally to all land uses. There are
                                other mechanisms to apply BMPs and other reduction


Modified 11/7/2010                                                                         56
                               factors. In the ‗land_to_river‘ file only, if this value is
                               equal to -9, the factors vary by land segment and the
                               specific factors are found above the current directory in the
                               variable_l2r_factors directory
       47 and greater – listing of land uses to which this applies (land_to_river file only)

rchres_out_to_conc – space delimited file specifying the concentrations to be output by
       the postprocessor and the river variable constituents that make them up. Each line
       contains the specifications for one output concentration. The specifications end
       when a literal ‗end‘ is found in the first three columns of the file. Any text after
       that is not read, so it can be used for comments.
       Column          Description
       1-4             Name of the data type to create. This will be the file extension for
                                each file created for this data type
       6-9             Units of the output. This is for documentation in this file only
       11-14           Normalization variable. This is generally the river variable name
                                in rchres_out_3x that is for water
       17-20           number of constituents that make up the data type
       22-25           The river variable name in rchres_out_3x for the first constituent
       27-34           The factor to convert from a load per flow to a concentration for
                                the first constituent
       remaining columns are contain the remaining constituents and factors up to the
                number specified in this line

rchres_out_to_load – space delimited file specifying the loads to be output by the
       postprocessor and the river variable constituents that make them up. Each line
       contains the specifications for one output load. The specifications end when a
       literal ‗end‘ is found in the first three columns of the file. Any text after that is
       not read, so it can be used for comments.
       Column           Description
       1-4              Name of the data type to create. This will be the file extension for
                                each file created for this data type
       6-9              Units of the output. This is for documentation in this file only
       11-14            number of constituents that make up the data type
       16-19            The river variable name in rchres_out_3x for the first constituent
       21-28            The factor to convert from a load per flow to a concentration for
                                the first constituent
       remaining columns are contain the remaining constituents and factors up to the
                number specified in this line

river_to_wqm57kPS, river_to_wqm57kNPS, etc – space delimited files that create a
       translation between the river variables and any downstream model or data output.
       The wqm57kPS and wqm57kNPS are the 57,000 cell estuarine water quality
       model point source and nonpoint source inputs that the phase 5.2 watershed
       model is used with in the Chesapeake Bay Program. The specifications end when




Modified 11/7/2010                                                                        57
        a literal ‗end‘ is found in the first three columns of the file. Any text after that is
        not read, so it can be used for comments.
        Column           Description
        1-6              Name of the data type to create. Multiple instances are allowed
                                and added together
        10-15            Units of the output. This is for documentation in this file only
        19-22            River variable name consistent with those in rchres_out_3x and
                                rchres_out_4x
        26-32            Watershed model output units for documentation in this file only
        34-43            Conversion factor
        46-49            River variable to divide by for outputs in concentration blank if
                                output is in load and nothing to divide by

atdep_to_wqm57k – space delimited file that creates a translation between the
      atmospheric deposition WDM variables and the 57,000 cell estuarine water
      quality model inputs. The specifications end when a literal ‗end‘ is found in the
      first three columns of the file. Any text after that is not read, so it can be used for
      comments.
      Column          Description
      1-6             Name of the data type to create. Multiple instances are allowed
                              and added together
      10-15           Units of the output. This is for documentation in this file only
      19-22           Atmospheric deposition DSN.
      25-28           Atmospheric deposition WDM variable name. This name and the
                              DSN above must be in agreement with the file data_dsns in
                              this directory.
      34-43           Conversion factor

quals_to_special_action_species – comma delimited file that create a translation
       between the PQUAL and IQUAL variables listed in ―piqual‖ in this directory and
       the special action variables listed in the ―special_action_species‖, also in this
       directory. Any line that is not blank space or a comment (containing three
       consecutive *s) is interpreted as a specification line
       Order           Description
       1               qual number to match from ―piqual‖
       2               Number of special action species tied to that qual
       3+              special action species tied to the qual in the first column


8.3.2.3 Special files
perlnd_extsources, implnd_extsources, rchres_extsources –files listing the external
sources for each type of HSPF simulation. These files are directly written into the UCIs
without modification except that all lines after and inclusive of the literal ‗end‘ in the first
three columns. The WDM references must match the references in data_dsns.




Modified 11/7/2010                                                                            58
atm_to_rchres – space delimited file that loads atmospheric deposition of sediment into
      the river simulation. This file is different from the others in that it creates the
      time series and does not translate it from one variable to another. The deposition
      of sediment is very small and not expected to change over scenarios. The
      specifications end when a literal ‗end‘ is found in the first three columns of the
      file. Any text after that is not read, so it can be used for comments.
      Column          Description
      1-6             HSPF river module to which the variable applies
      8-10            WDM data set number (dsn) of the receiving river variable
      14-23           Sediment deposition in tons/hour
      27-43           HSPF receiving variable
      48-51           WDM variable name of the receiving river variable
      55 and greater is a text description of the variable

scenario_parameter_modifications – Certain river parameters are adjusted for scenarios
       based on the change in total inputs to the river between the calibration and a
       scenario. This space delimited table lists the parameters that change in response
       to changes in loads. The loads in the final column of this table must be listed in
       the first column of the ―rchres_out_to_load‖ file in this directory. The
       specifications end when a literal ‗end‘ is found in the first three columns of the
       file. Any text after that is not read, so it can be used for comments.
       Column          Description
       1-6             HSPF river module to which the adjustment applies
       9-22            HSPF table as listed in ./config/catalog/modules/rivtables
       24-37           HSPF variable as listed in ./config/catalog/modules/rivtables
       39-42           load associated with this parameter change

delivery_ratio_calc – Delivery ratios are an accounting of the fraction of material that is
       transported through a particular stream system. They are the ratio of output to
       input. Some delivery ratios are stand-ins for others. Since transformations
       between nitrogen species and between phosphorus species are simulated, the TN
       ratio and TP ratios are applied to all of the individual species. This space
       delimited table lists loads on which delivery factors will be calculated and the
       loads to which the calculated delivery factors will be applied. The specifications
       end when a literal ‗end‘ is found in the first three columns of the file. Any text
       after that is not read, so it can be used for comments.
       Column           Description
       1-4              Load variable as listed in ―rchres_out_to_load‖ in this directory
       6                number of other loads that use this delivery factor
       8+               Listing of loads that use this delivery factor

/variable_l4r_factors/ - directory that contains files specifying spatially explicit
relationships between the land and river. If a ―-9‖ appears in the multiplication column
of the land to river file, a file must appear in this directory with the land and river
variables appearing in the file name. For example land SEDM goes to river SILT with a




Modified 11/7/2010                                                                       59
―-9‖ factor, so the file SEDM_to_SILT.csv appears in this directory. Each file is comma
delimited, listing the land segments and the factor.

8.3.3 modules
This section contains all of the information on model parameters to be used for each
       module. Two space delimited files in this directory contain the names of
       variables stored in tables above ./input/param.

The file 'tables' has an exhaustive list of table names, header lines, and variables for
        perlnd and implnd simulation. Each section that starts with module name
        beginning in the first column will have a listing of the tables that make up that
        module. The listing of tables continues until the ‗END module‘ line is found.
        The tables are specified using groups of lines. The first line in the group is the
        table name in columns 3-14. Columns 19-23 contain the format specifier for the
        table. Columns 19-20 are the number of variables, columns 22-23 are the number
        of spaces for each variable, and column 21 is the variable type. An ‗i‘ in column
        21 indicates a integer table, an ‗f‘ indicates a real or floating point table, and a ‗u‘
        indicates a unique format that will be detailed in the ‗Uformat‘ file in this
        directory. The header lines, including the variable name line are all comments,
        indicated by having three consecutive *s. The variable line has a 'V' as the third
        character. The tables names and variables must be identical to those found in the
        parameter files above ./input/param.

The file 'rivtables' is the same file for rivers.

The ‗Uformat‘ file is consulted if a ‗u‘ is found in column 21 in either ‗tables‘ or
      ‗rivtables‘ as described above. Generally, the table specifications in the file have
      a row of starting columns for variables and a row of ending columns for variables.
      Modification of this file requires code modifications under ./code/src/lug for land
      or ./code/src/rug for rivers.

8.4 blank_wdm
      The general method of operation for the land simulation, the river simulation, and
the external transfer module is to copy a blank WDM to a local directory, populate it
through a fortran program or a run of HSPF, and then move it to the appropriate location.
The blank_wdm directory contains WDMs with blank variables added through WDMutil
or some other WDM utility. The dsn and variable names must match the files perlnd,
implnd, rchres_in, rchres_out, etc in the ./config/catalog/iovars/iovarscenario/ directory.

9 Code documentation
A few error messages are generated from the scripts directly, but most are generated from
       the compiled code. At times the error messages will be unhelpful due to
       unforeseen circumstances. The easiest way to trace back these issues is to search
       the code using the script ./code/src/grepf. Modify this script to search for a
       particular part of the error message and it should lead you to the point in the code



Modified 11/7/2010                                                                           60
9.1 Lib


9.2 Lug


9.3 Rug


9.4 Etm


9.5 Postproc


9.6 Wqm

10      A detailed walk through a scenario.
Say you have some new model inputs that you want to try out. You may have the files
       generated from scenario builder, or you may have come up with them on your
       own. Running the model only takes a few steps, but going through those steps in
       detail will help you understand how the whole system works.

There are several scripts that will be called. Generally the steps that make one script
       unique will be described below. The scripts also contain boilerplate sections used
       to get input data, move to a directory, and clean up. These sections are discussed
       in section 6.2 and won‘t be repeated for all of the scripts below.

10.1 Decide on the spatial extent of your scenario
call it Basin

10.2 Create the new ASCII data sets
These only have to be for the spatial extent above

10.3 Create any new wdms that will be necessary


10.4 Copy old river, and probably land, scenario control files
     and modify them
Create a name for your scenario and make it as descriptive as possible. For the purposes
of the rest of this description, we will call them LandScen and RiverScen




Modified 11/7/2010                                                                     61
10.5 Create the directories for running the scenario
./run/standard/make_land_directories.csh LandScen
No fortran programs are called with this script. It takes the scenario as an argument, then
reads in the land use 3-character names. For each land use, it creates the directories to
hold the files for the run:
directory                                             contents
./tmp/uci/land/landuse/LandScen/                      UCIs
./tmp/wdm/land/landuse/LandScen/                      land output WDMs
./output/pltgen/land/LandScen/                        Pltgens ordered in the control file
./output/pltgen/summary/LandScen/                     summaries for calibration
./output/input/LandScen/                              echo of input files by land segment
./output/hspf/land/out/landuse/LandScen/              HSPF standard output files
./output/hspf/land/ech/landuse/LandScen/              HSPF standard echo files

./run/standard/make_river_directories.csh RiverScen
No fortran programs are called with this script. It takes the scenario as an argument and
creates the directories to hold the files for the run:
directory                                              contents
./tmp/uci/river/RiverScen                              river UCI files
./tmp/wdm/river/RiverScen/eos                          external transfer model WDMs
./tmp/wdm/river/RiverScen/stream                       River output WDMs
./output/eof/TimeAggregation/RiverScen                 Edge of Field output
./output/eos/TimeAggregation/RiverScen                 Edge of Stream output
./output/river/TimeAggregation/RiverScen               river output
./output/river/peaks/RiverScen                         information on flow peak calibration
./output/river/stats/RiverScen                         river calibration statistics
./output/river/window/RiverScen                        more river calibration statistics
./output/river/rating/RiverScen                        more river calibration statistics
./output/river/cfd/RiverScen                           even more river calibration statistcs
./output/del/tfs/TimeAggregation/RiverScen             transport factors
./output/del/dfs/TimeAggregation/RiverScen             delivery factors
./output/del/lseg/TimeAggregation/RiverScen            delivered loads by land segment
./output/pltgen/river/RiverScen                        river Pltgens ordered by control file
./sumout/TimeAggregation/RiverScen                     summary output
./output/etm/RiverScen                                 binary etm files
./output/hspf/river/out/RiverScen                      HSPF standard output files
./output/hspf/river/ech/RiverScen                      HSPF standard echo files

10.6 Run the lug
The lug is the land UCI generator. Understanding what the lug does is important in
understanding the directory and file structures and in becoming familiar with the
operations of the watershed model. In reading through this, the user should navigate to
the various file locations to see where the information is located. An additional level of
understanding would come from following the code as well.

To kick off your scenario, go to ./run/standard/ and run: run_lug.csh LandScen Basin


Modified 11/7/2010                                                                        62
The script simply calls the program ./code/bin/lug.exe for each land use in each land
segment in the Basin file. The code for the lug program is in ./code/src/lug

Steps in the lug program:
   1. read in land segment, land use, and land scenario
   2. look in the land control file to determine:
            a. if the land use is perlnd or implnd by seeing if it occurs under PER
                MODULES or IMP MODULES
            b. if there are pltgens by looking for a valid line in the PLTGEN table for
                this land use
            c. the iovar scenario
            d. the meteorological and precipitation / atmospheric deposition data sets to
                use under their respective tables
            e. the active modules
            f. the start and stop times
   3. Read the tables
            a. consult the files in the ./config/catalog/modules/ directory, see section
                8.3.3, to get the list of tables for each module, the list of parameters in
                each table, and the formatting specifications for each parameter.
            b. consult the files in ./input/param/landuse/ to populate variables holding the
                values of each parameter identified above
   4. Open the UCI file in ./tmp/uci/land/LandUse/LandScenario/
   5. Write the GLOBAL block with the start and stop time from the control file and
        information on the segment and scenario
   6. Write the FILES block
            a. Reference the location of the meteorological and precipitation /
                atmospheric deposition WDMs as indicated in the control file
            b. name the output wdm, the message file, and the standard HSPF output
                using the segment and land use name
            c. Reference the location of each PLTGEN file specified in the control file
   7. Write the OPN SEQUENCE or ‗operation sequence‘ block with a call to either
        the PERLND or IMPLND module followed by any PLTGEN calls specified in
        the control file
   8. Write the Special Actions block
            a. get the species available for special actions from
                ./config/catalog/iovars/IovarScenario/special_action_species
            b. get the special action flags from
                ./input/scenario/land/spec_flags/SpecActFlagScenario/spec_flags.csv
                where SpecActFlagScenario is found from the land control file. These
                flags determine which special actions are used for this land use as follows:
                      i. V = variable crop cover, table MON-COVER
                     ii. P = plowing, increase in detached storage of sediment
                    iii. F = fertilizer, increase in storage of nutrient species
                    iv. M = manure, increase in storage of nutrient species
                     v. L = fixation for legumes, increase in storage of nutrient species
                    vi. T = variable total annual nutrient uptake targets



Modified 11/7/2010                                                                       63
                 vii. U = variable monthly fraction of nutrient uptake target
                viii. R = removal of refractory organic N from storage to simulate crop
                       harvest
          c. Write the header lines for special actions that define the following types of
             aliases and distribution. See the HSPF documentation for a more
             complete description.
                    i. write the distribution lines used to spread monthly values over the
                       course of the month. The distributions are hard-coded. Using
                       distribution 2, monthly values are spread into 10 equal applications
                       three days apart.
                   ii. write the user-defined time series quantity aliases, or UVQUANs.
                       The only UVQUAN typically used in CBP modeling is prec, for
                       precipitation. The UVQUAN is referenced in an ‗IF‘ statement to
                       delay applications to fields during rain events.
                 iii. write the user-defined variable aliases, or UVNAMES. These are
                       specific to each type of special action. For example, a fertilizer
                       application may be written as FNO3, which is translated to 50% in
                       the surface and 50% in the upper layer
          d. Write the action lines for each type of action specified in the
             spec_flags.csv file. The general procedure for each different data type is
             detailed below:
                    i. get the appropriate file names from the land control file
                   ii. read the values from each file in ./inputs/scenario/land/data_type/
                       that was specified above
                 iii. if multiple files are specified representing multiple points in time,
                       check the data to make sure there are no missing values that would
                       occur when the land use acreage is equal to zero. If some values
                       exist, but others are missing, use the exiting values to fill in the
                       missing ones through interpolation
                  iv. If all values are missing look for the default crop cover file as
                       specified in the control file. If data are missing from the control
                       file report an error
                   v. Write out an action for each year and month by interpolating
                       between specified data points. Interpolation is between the same
                       month of multiple years, not between different months in the same
                       year.
                  vi. Write out files by land segment with the annual and average annual
                       application if this action is for fertilizer, manure, legume, or
                       uptake. The files are stored in ./output/input/LandScenario/.
          e. Removal of organic nitrogen to simulate harvest is specified in the file
             ./input/param/LandUse/ParameterScenario/rorgn.csv and put into the
             input file for each year.
   9. Read in the atmospheric deposition from the appropriate WDM according to the
      land segment and the precipitation / atmospheric deposition scenario specified in
      the control file and write out the annual and average annual values as in 8.d.vi just
      above.



Modified 11/7/2010                                                                      64
   10. Use special actions to modify the PQUAL and IQUAL parameters through time
       and as a function of the difference between this scenario and the calibration
       scenario. To simulate sensitivity, the inputs for a particular scenario are
       compared to the inputs for the calibration scenario and factors are determined. In
       addition, if there is a trend in inputs over time, the factors will vary with the trend
       in inputs on an annual basis. This trend will be part of the calibration as well.
       Both relationships are such that the parameter is halved when the input balance is
       equal to zero and equal to the nominal calibration value when the balance is equal
       to the average or calibration value. The PQUAL and IQUAL parameters are not
       modified in the ./input/param/ directory, only internally to this program and in the
       SPECIAL ACTIONS section of the UCI.
           a. Read the land scenario control file to get the calibration scenario
           b. Read the annual fertilizer, manure, legume fixation, atmospheric
               deposition, and uptake files, if available, for the scenario in
               ./output/input/LandScenario/ and for the calibration in
               ./output/input/CalibrationScenario/.
           c. Get the relationship between PQUAL and IQUAL variables and special
               action variables in
               ./config/catalog/iovars/IOvarScenario/quals_to_special_action_species
           d. For each of the special action species in a PQUAL or IQUAL variable
               calculate the overall PQUAL or IQUAL modification and the temporal
               modification, if any and apply it to the internal PQUAL or IQUAL
               variables.
           e. Write the PQUAL or IQUAL action lines to the SPECIAL ACTIONS
               block of the UCI.
   11. Write the parameter tables for each MODULE specified in the control file using
       the previously stored table names, parameter names, and parameter values
   12. Write the EXT SOURCES block using the file perlnd_extsources or
       implnd_extsources in the directory ./config/catalog/iovars/IovarScenario/ linking
       meteorological and precipitation / atmospheric deposition WDM dsns to HSPF
       inputs
   13. Write the EXT TARGETS block using the file perlnd or implnd in
       ./config/catalog/iovars/IovarScenario/ linking HSPF outputs to WDM dsns
   14. If PLTGENs are specified in the control file, write the appropriate blocks
           a. NETWORK block linking HSPF variables to PLTGENs
           b. write the PLTGEN block with the tables PLOTINFO, GEN-LABELS,
               SCALING, and CURV-DATA using information gathered from the
               control file.

10.7 Run the land
The land UCIs generated in the last section how have to be run with HSPF. Each land
use within each land segment has a separate UCI, requires a separate run of HSPF, and
generates its own output. For each land use and segment, the outputs of this process are:
a WDM with the time series output of the simulation, an hspf-standard output file, an
hspf-standard echo file, and any PLTGENs specified in the land control file.



Modified 11/7/2010                                                                         65
To kick off the land HSPF run, go to ./run/standard/ and run:
run_land.csh LandScen Basin.
The script loops over each land use in each land segment in the Basin file and performs
the following steps
    1. copy a blank land WDM from ./config/blank_wdm/land.wdm to the current
        working directory and name it LanduseLandsegment.uci
    2. run the UCI with HSPF
    3. test for successful completion of HSPF by comparing the last line of the echo file
        LanduseLandsegment.ech with the file ./run/test/EOJ. Stop and report the
        problem if there was an issue.
    4. Move the HSPF standard out to ./output/hspf/land/out/Landuse/LandScenario/
    5. Move the HSPF standard echo to ./output/hspf/land/ech/Landuse/LandScenario/
    6. Move the output WDM to ./tmp/wdm/land/Landuse/LandScenario/


10.8 Run the etm and postprocessor
The external transfer module and the postprocessor both spend most of their
computational time reading time series from the land WDMs, taking averages, and
multiplying by translation factors. It saves a great deal of time if these two functions are
combined. They can be run separately by running ./run/standard/run_etm.csh and
run/standard/run_postproc.csh where the entire postprocessor is run after the river. See
section 10.10 for more details on choosing which postprocessor to run and how to edit the
postprocessing script to get the appropriate output.

The etm and postprocessor script creates a file of transfer coefficients between land and
river variables, creates the river WDMs, creates land use edge-of-stream (EOS) and edge-
of-field (EOF) loads, and creates EOS and EOF loads for point source, septic, and
atmospheric deposition loads.

The script ./run/useful/run_etm_and_land_and_dat_simultaneously.csh may be edited to
get the desired output. After the initial variable setup from the invocation statement, the
script contains several lines where output flags are set to either 1 or 0. Zero indicates that
no output of this type should be generated, while 1 means generate the output. The
variables are daily, monthly, annual, and average annual output for EOS, EOF, and DAT
outputs, where DAT indicates data output for point source, septic, and atmospheric
deposition. The user can also specify the level of screen output generated. The variable
‗loud‘ is set to 0 for low screen output and 1 for verbose output.

The script ./run/useful/run_etm_and_land_and_dat_simultaneously.csh loops over river
segments and performs the following tasks for each segment:
   1. Make the binary transfer file containing coefficients between land and river
       variables. The code for this part is in
       ./code/src/etm/make_binary_transfer_coeffs/
            a. Gather information needed for the calculations




Modified 11/7/2010                                                                         66
                  i. Determine the number of land segments that intersect this river
                     segment using the file
                     ./config/catalog/geo/GeoScenario/land_water_area.csv
                 ii. Read the river scenario control file to determine:
                         1. the time interval of the run
                         2. the number of land use files used in the run, the names of
                              those files, and the date for which each one is valid
                         3. the number of BMP files used in the run, the names of
                              those files, and the date for which each one is valid
                         4. The BMP specification scenario
                         5. The land scenario for each land use
                         6. the name of the transfer file used for regional or subgrid
                              factors
                         7. The I/O scenario
                iii. Read in the BMP constituents from the file
                     ./config/catalog/iovars/IOscen/bmp_constituents
                iv. Read in the BMP specifications as detailed in section 7.4.3.3.
                         1. read in the Hydrogeomorphic region for each land-river
                              segment
                 v. Read in the land use data from ./input/scenario/river/land_use/ as
                     directed by the river control file
                vi. Read in the BMP implemented acres data from
                     ./input/scenario/river/bmpacres/ as directed by the river control file
               vii. Read in the regional factor data from ./input/param/transport/ as
                     directed by the river control file
         b. Create a time series of land use acres for each land use and lrseg
                  i. If only one data set is specified in the control file create a duplicate
                     copy as the second point for interpolation
                 ii. interpolate and extrapolate the land use acreage between specified
                     points in time
                iii. correct for acreages less than zero by increasing all other land uses
         c. Calculate Aggregate BMP efficiency for each land use and day
                  i. Read the land WDM flow time series to get the return frequency
                     for all days
                 ii. Apply a hydrologic effect, if any
                iii. Apply a random effect, if any
                iv. Interpolate BMP acres
                 v. combine multiple BMPs on the same land use into aggregate
                     efficiencies
         d. Write a binary file for each lrseg containing the start and stop dates, land
             use, BMP factors, transport factors, and ancillary information. The files
             are located in ./output/etm/RiverScen/
         e. Echo out the BMP acres
   2. Generate the river input WDMs and summarized text output for the land
      simulation. The code for this section is in ./code/src/etm/etm_and_postproc/
         a. Get information necessary to generate the files including:



Modified 11/7/2010                                                                        67
                   i. time of run and land scenarios from river control file
                  ii. land variable to river variable translation information from
                      ./config/catalog/iovars/IOscen/land_to_river
                 iii. river variable to load translation information from
                      ./config/catalog/iovars/IOscen/rchres_out_to_load
         b. For each river variable, determine the land variables that go into it. Read
             the time series from the land wdms, multiply them by the appropriate land
             acreage, bmp factors, regional factors, and variable translation factors to
             create a time series of river input
         c. store the river variables in the river input WDM in
             ./tmp/wdm/river/p52An/eos/
         d. For each land use and land-river segment, create a text file of loads.
             Depending on the variables set in the script, these can be edge-of-field or
             edge-of-stream loads and summarized on daily, monthly, or annual time
             steps, or an overall temporal average. The text files are stored above
             /output/ in descriptive directories. The average annual edge-of-stream
             files, for example, are stored in ./output/eos/aveann/RiverScen/.
   3. Generate the summarized text output for point sources, atmospheric deposition,
      and septic loads. The code for this section is in ./code/src/postproc/data/.
         a. Get information from ./config/catalog/iovars/IOscen/ to relate data types
             to river variables and river variables to load types.
         b. Get the point source, septic, and atmospheric deposition data set
             specifications from the river control file
         c. Read the data WDMs, translating data time series into river variable time
             series, and then into load time series.
         d. Write out text files as in the land use loads above. There are no edge-of-
             field loads, only edge-of-stream.

10.9 Run the scenario_river
Running the river requires several steps, which are contained in the
./run/standard/run_scenario_river.csh script. Each river must go through all simulation
steps before proceeding to the next downstream river as the downstream river is
dependent on the simulation of the upstream river. The river script performs the
following steps for each river in the basin:
    1. generates output for streams at confluences (see section 2.2.5.2 ). If a confluence
        station exists, according to the file ./config/catalog/geo/GeoScen/doubles.csv, A
        river WDM is generated with the output loads that are the sum of the upstream
        river loads of the current river segment. This WDM has the same riverseg name
        as the current segment, except that the downstream ID (last four digits) are 0003.
        The WDM is moved to the output river wdm directory
        ./tmp/wdm/river/RiverScen/stream/
    2. adds upstream loads to the edge-of-stream WDM
            a. Copy the edge-of-stream WDM to the current directory
            b. Add the output of all upstream rivers to it
    3. Combine the point source, septic, diversion, and atmospheric deposition of
        sediment wdms into a single WDM and store in the current directory.


Modified 11/7/2010                                                                      68
     4. Compare the current scenario inputs to the calibration scenario inputs
                                 a. get the calibration scenario from the river control
                                     file


1. generates output for streams at confluences
2. adds upstream loads to the edge-of-stream WDM, assembles
3.
1.

10.10 Run the remaining postprocessing steps
note about dealing with the run_postproc.csh flags and the other versions of the
postprocessor script


10.11 Create the Estuarine model input


11      A detailed walk through the calibration

11.1 PWATER


11.2 SEDMNT


11.3 SOLIDS


11.4 PSTEMP


11.5 PQUAL


11.6 IQUAL


11.7 AGCHEM


11.8 WQ


Modified 11/7/2010                                                                        69
11.9 Transport


12 Parallel operations
parallelized at the script level rather than the code level
several strategies
         break by land use
         break by region
         run single and let slurm schedule
scripts to do domain decomposition


13      Cool things to be done in the future

13.1 Parallelize in MPI


13.2 Improve calibration


13.3 Work in smaller basins


13.4 New constituents
13.5 Uncertainty analysis



14 References
EPA 2008. Chesapeake Bay 2007 Health and Restoration Assessment. EPA-903-R-08-
      002. EPA, Washington, DC. March 2008.

USGS 2005. Development of Land Segmentation, Stream-Reach Network, and
     Watersheds in Support of Hydrologic Simulation Program- Fortran (HSPF)
     Modeling, Chesapeake Bay Watershed, and Adjacent Parts of Maryland,
     Delaware, and Virginia. Scientific Investigations Report 2005-5073. USGS,
     Reston, VA. 2006.




Modified 11/7/2010                                                               70

				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:68
posted:11/8/2010
language:English
pages:70