1

Document Sample
1 Powered By Docstoc
					Real Time Forecast Verification (RTFV)
Development Manual

Table of Contents
1      Introduction ................................................................................................................. 1
    1.1     Goals and objectives ........................................................................................... 1
    1.2     Statement of scope .............................................................................................. 1
    1.3     Software context.................................................................................................. 1
    1.4     Major constraints ................................................................................................. 1
2      System overview ......................................................................................................... 2
    2.1     Description .......................................................................................................... 2
    2.2     How to run RTFV as a stand-alone (compiled) application ............................... 3
    2.3     How to run RTFV from an interactive IDL session ............................................ 4
    2.4     How to run RTFV as a scheduled job ................................................................. 5
3      Data design .................................................................................................................. 5
    3.1     Configuration files .............................................................................................. 6
    3.2     Internal data structures ...................................................................................... 12
    3.3     Global data structure - info ............................................................................... 18
    3.4     GUI data structure ............................................................................................. 18
    3.5     Database description ......................................................................................... 18
4      Architectural and component-level design ................................................................ 21
    4.1     Architecture diagrams ....................................................................................... 21
    4.2     Utility routines .................................................................................................. 24
    4.3     Spatial transformation ....................................................................................... 25
    4.4     Memory management........................................................................................ 26
    4.5     GUI .................................................................................................................... 26
    4.6     Software Interface Description.......................................................................... 27
5      User interface design ................................................................................................. 28
    5.1     Description of the user interface ....................................................................... 28
    5.2     Screen images .................................................................................................... 28
    5.3     Objects and actions ........................................................................................... 30
6      Testing ....................................................................................................................... 30
    6.1     Classes of tests .................................................................................................. 30
    6.2     Expected software response .............................................................................. 31
    6.3     Output logging .................................................................................................. 32
    6.4     Performance bounds .......................................................................................... 33
    6.5     Critical components .......................................................................................... 33
7      Appendices ................................................................................................................ 33
    7.1     Packaging and installation................................................................................. 33
    7.2     Adding and removing components ................................................................... 40
    7.3     Adding and updating code ................................................................................ 41
    7.4     Supplementary information ............................................................................... 41




Beth Ebert, BMRC                                                                                 Last updated: 12 July 2008
Address questions to e.ebert@bom.gov.au
  1      Introduction

 1.1     Goals and objectives
        This document describes all data, architectural, interface and component-level design for
        the Real Time Forecast Verification (RTFV) system. It is intended to provide assistance
        to those maintaining and modifying the system. A general introduction to the handling of
        data including input data, output data and system configuration is given. This is followed
        by an introduction to the system design and implementation. The main processes are
        explained along with lower level procedures.
        Detailed instructions for using RTFV and examples of RTFV graphical and text products
        can be found in the RTFV User Manual.

 1.2     Statement of scope
        The RTFV system provides visual and statistical verification of meteorological forecasts
        both in near-real time and post-real time.

 1.3     Software context
        The initial development of RTFV is in response to the need for a real time verification
        system to evaluate nowcasts and short range forecasts during the WWRP Beijing 2008
        Olympics Forecast Demonstration Project. The Bureau of Meteorology will also use
        RTFV to verify nowcasts and short range forecasts. The system design is extensible to
        allow verification of a wide range of meteorological forecasts against a variety of
        observational data. RTFV will be made available to the broader meteorological
        community.

 1.4     Major constraints

1.4.1    Language
        The initial development will be done in the (proprietary) IDL language to make use of
        existing forecast verification software and speed the process of development. Once a
        prototype system has been developed and tested in IDL then it will be recoded in Python.
        Python is an open source language with mathematical and graphical libraries and is
        becoming increasingly popular in physical sciences communities.

1.4.2    Input data
        The sources and formats of the input data (observations and forecasts) are dictated
        externally. RTFV must be able to read the data as provided by a data server or from
        external files.

1.4.3    Output data
        The requirements for output statistical and graphical products are dictated externally. For
        example, web browsers must access image files and tabular results. The statistical results
        must be accessible by RTFV for downstream processing such as summarizing and
        plotting.


                                                   1
1.4.4    Platform
        RTFV is a stand-alone application that should be platform-independent. It can run on
        Windows or Linux systems that are networked to a data server or other source of input
        data. To be effective in an operational forecast setting the processing of the data must
        occur quickly enough so that users can take the results into account in their decision
        making process, i.e., it must "keep up" with the inflow of new data. This may dictate the
        CPU requirements of the platform used to run RTFV.

1.4.5    Adaptability
        The design of RTFV must be sufficiently modular so that it can easily incorporate new
        data sources and verification methods that will become available in the future.


  2      System overview

 2.1     Description
        The RTFV system is a new self-contained system. It inputs single-level forecast and
        observational data, and generates graphical and numerical (statistical) products
        representing various aspects of the forecast quality. It is designed to be run in real time in
        conjunction with nowcast and short-term forecast systems in an operational weather
        forecasting office, with the forecast and observational data flowing directly into the
        RTFV system. It can also be used in research (manual) mode to do post-event
        verification. It can be operated via a graphical user interface (GUI) or via the command
        line.
        The overall structure of the system is shown below. RTFV is initialized with
        configuration information describing the forecast data, observational data, verification
        domain, etc. Details of the configuration files are given in Section 3.1. Forecast and
        observation data are provided to RTFV by a data server, either on receipt of new
        observational data (automated verification mode) or by request from RTFV (manual
        mode). RTFV generates the set of verification graphical and numerical products specified
        by the user and outputs them to a display and also to an archive. External applications
        such as web browsers may read from the verification archive, and RTFV itself reads from
        the archive in order to perform post-processing tasks such as time-aggregated statistics.

                                          Configuration
                                           information


            Forecast data                                                     Display
                                       Real Time Forecast
                                       Verification System
                                             (RTFV)
          Observation data                                              Verification archive

        RTFV can generate verification products with three different levels of complexity:
        (a) The simplest are the "visual" products such as maps and time series. These help the
        user to see how the forecasts and observations differ from each other, and are produced as


                                                    2
      graphical output only. Visual products can be generated in both automated and manual
      mode.
      (b) "Statistical" products represent quantitative comparisons between forecasts and
      observations, using such measures as root mean square error, correlation coefficient,
      probability of detection, etc. These products are stored as both numerical output (for
      archival and later reference) and graphical output. Statistical products can be generated in
      both automated and manual mode.
      (c) The most complex verification products are the "diagnostic" products generated using
      advanced verification methods such as object-oriented verification, intensity-scale
      verification, practically perfect hindcasts, and so on. As with the statistical products the
      output is stored as both numerical and graphical output. Because these methods are more
      computationally intensive than the simple statistical products, diagnostic products can be
      generated only in manual mode.
      In automated mode RTFV is normally initiated as a scheduled job, producing new
      verification products as new observations become available and there are forecasts which
      can be verified using the new observations. The automated verification runs without the
      GUI being activated.
      Running RTFV in manual mode is done via the GUI. This is an easy way for the user to
      select a specific configuration of forecast, observation, start and end time, desired
      verification products, etc.

2.2    How to run RTFV as a stand-alone (compiled) application

      2.2.1 Starting by double-clicking
      In the RTFV directory double-click on the rtfv.sav file. This brings up a small interface to
      start and stop the application:




      2.2.2 Starting from the command line prompt

      In Windows:

         1. Select Run from the Start menu, and enter cmd to open a DOS command line
            prompt.
         2. Change the directory to the RSI\IDL directory \bin\bin.x86 directory (e.g.,
            C:\RSI\IDL61\bin\bin.x86)



                                                 3
           3. Enter the following at the command line prompt:
               idlrt -vm=<path><filename>
               where <path> is the path to the .sav file, and <filename> is the name of the . sav
               file. If you omit the path and filename (leaving only the -vm flag), IDL will open
               a file selection dialog allowing you to select the .sav file.
        In Linux:
           1. From the RTFV/src/scripts directory call the script run_rtfv.csh.
        (work on these instructions some more)

 2.3     How to run RTFV from an interactive IDL session
        If the machine running RTFV has an IDL development license then RTFV can be run
        from an interactive IDL session. Change the directory to RTFV/src using the cd
        command so that all relevant procedures will be found. (An alternative would be to
        specify the RTFV/src directory as a library by selecting File/Preferences/Path in the
        IDLDE.) The main program for running RTFV is rtfv.pro.

        IDL> cd, 'RTFV/src'             (include the full path in the cd command)

        IDL> rtfv

2.3.1    Running in automated mode from an IDL session
        At the IDL prompt type:

        IDL> rtfv

        This brings up the interface shown above to start and stop the application. Click the
        button labeled "Start automated verification".
        Once it is started the most recent data is loaded into memory and RTFV verifies all
        forecasts for which there is observational data available. Verification products are
        created according to the specifications in the configuration files (see Section 3.1). All
        products that are enabled (priority>0; see Section 3.1) are generated. It then checks for
        new data on a regular basis and continues to generate verification products. End RTFV by
        clicking the "End RTFV" button in the small GUI.
        Due to IDL's accumulation of memory over time, it is better to run RTFV as a scheduled
        job if extended running is desired. See Section 2.4.

2.3.2    Running the GUI (manual mode) from an IDL session
        At the IDL prompt type:

        IDL> rtfv

        This is the same as typing

        IDL> rtfv, 'default'



                                                   4
        where 'default' specifies the default GUI configuration file default_config.xml. You
        can specify a different GUI configuration file. Click the button labeled "Start manual
        verification (GUI)" to start the GUI. For an example of the GUI see Section 5.2.
        The user can load a particular configuration file, or alter the default values. Verification
        proceeds in automated mode with all products that are enabled (prority>0; see Section 3.1)
        being generated. The graphical products specified by the GUI configuration file are
        displayed in the GUI. When the GUI is exited the memory is cleared.

2.3.3    Verifying cases outside of real time
        The RTFV User Manual describes some procedures for verifying forecasts outside of
        real time. Briefly these routines are:
              do_rtfv.pro - verify all forecasts valid within a given period of time
              run_rtfv_case.pro - verify cases outside of real time
              run_rtfv_statsplots.pro - produce statistics plots and tables outside of real time
        Please refer to the RTFV User Manual for details on using these procedures.

 2.4 How to run RTFV as a scheduled job
        The do_rtfv.pro procedure can be called without any arguments to verify the most recent
        forecasts against the most recent observations. Scheduling do_rtfv.pro to run regularly in
        real time requires calling it from a batch file (Windows) or a script (Linux/Unix).
        Invoke the Windows Scheduled Task manager to initiate a batch job called
        run_do_rtfv.bat, located in the RTFV directory. For Linux/Unix set the Cron Table to
        make regular calls to the run_do_rtfv.sh script, located in the RTFV directory. Section 7.1
        on Installation contains more information on the contents of these batch and script files.

  3      Data design
        The RTFV directory contains a number of subdirectories whose contents are described
        below:
        Table 1. RTFV subdirectories.
        Subdirectory                Contents
        src                         source code (IDL procedures and batch files)
        archive                     local archive (not currently used)
           image                    image archive
           TS                       time series data archive
           stats                    statistics archive
           text                     text archive
        config                      configuration files
           palettes                 color tables used in making plots
        data                        supporting data (map information, etc.)
           fcst                     local forecast data files (for stand-alone testing)
           obs                      local observation data files (for stand-alone testing)
                                    map data (coastlines, district boundaries, etc.), including a
           map
                                    gshhs directory with detailed map information
           log                      log files (4 per day, overwritten)


                                                     5
          input                      list of forecast data files input to verification
      GUI                            graphical files to support GUI (button bitmaps); help files
      schema                         schemas to validate XML data files
      src                            source code (IDL procedures and batch files)
      test                           input files (for stand-alone testing)
      other                          (not currently used)
      <yyyymmdd>/image               local graphical output
      <yyyymmdd>/TS                  local time series output
      <yyyymmdd>/stats               local statistical output
      <yyyymmdd>/text                local tabular output

      The forecast and observation data are normally read from files on an external data server.
      Alternatively the forecast and observation data can be read from files the data/fcst and
      data/obs subdirectories, and the data serving is a component of the RTFV system (see
      Sections 3.5.1 and 4.1). The product directories for images, statistics, and text files may
      also be in a separate location, for example, a product server. See Sections 3.5.2 and 4.1.

3.1    Configuration files
      On initiation RTFV requires a number of input configuration files to specify information
      about the forecasts, observations, regions, configurations, etc. The start-up configuration
      file, rtfv_startup.dat, is a text file located in the RTFV directory. It specifies the locations
      of the other subdirectories listed above, as well as other basic information required by
      RTFV, in this order:
      Table 2. Elements of rtfv_startup.dat file.
      Variable             Description
      rtfv_info_dir        directory to hold configuration files defining RTFV system
      config_dir           directory to hold users' configuration files
      GUI_dir              directory with GUI building files (button images)
      obs_dir              source directory holding observations files
      fcst_dir             source directory holding forecast files
                           directory to hold forecast and observation files copied from
      data_dir
                           their source
      image_dir            local directory to hold recent verification graphical products
      statistics_dir       local directory to hold recent verification statistics data
      text_dir             local directory to hold recent verification tables
      archive_dir          archive directory to hold older verification products
                           schema for validating forecast XML files
      schema
                           (path/wxml.<version>.xsd)
      tolerance            maximum time separating forecast and observation valid times
      data_check_frequency how often to check for new data (sec) - used by trigger_rtfv.pro
      maxfcsts             maximum # forecasts or observations to hold in memory
      maximages            maximum # images of each type to hold in memory
      maxstats             maximum # statistics to hold in memory
      data_life            maximum lifespan of data in memory (minutes of data)
      aggregation_time     time over which statistics are aggregated (minutes)
                           time interval for making statistics plots in automated mode
      statsplot_time
                              -1 don't make any statistics plots


                                                     6
                              0 make statistics plots every time
                              X>0 make statistics plots every X minutes
log_level                  level at which to log output (0=don’t, 1=some, 2=all)

The configuration files in the config subdirectory are written in simple XML format so
that elements can be modified and added easily. RTFV reads the names of the tags
themselves in order to build data structures, which are referred to throughout the code.
When new data types, domain definitions, etc., are added to the verification system these
configuration files must be manually edited to include the new information. As an
example, an extract from rtfv_fcst.xml is printed below.
     <?xml version="1.0" encoding="ISO-8859-1"?>
     <root>

     <forecast>
      <name>TIFS</name>
      <namecode>ti</namecode>
      <forecast_type>nowcast</forecast_type>
      <quantity>thunderstorm</quantity>
      <spatial_type>cell</spatial_type>
      <spatial_res>0</spatial_res>
      <spatial_units>km</spatial_units>
      <time_res>10</time_res>
      <num_times>6</num_times>
      <projection>10,20,30,40,50,60</projection>
      <fcst_frequency>6</fcst_frequency>
      <accum_period>0</accum_period>
      <priority>1</priority>
     </forecast>
     (etc.)
     </root>

The "namecodes" are used in naming output files to indicate their contents (see section
3.5.2). They values of the namecodes are specified in the configuration files for forecasts,
observations, meteorological quantities, and domains.
Not all possible combinations of forecasts, observations, plots, statistics, etc. may be
required, and in fact, it may not be feasible or desirable to process everything in real time.
Data (for example, forecasts from a particular source) and products can be prioritized or
disabled by giving the "priority" element a value of 0, 1, or 2:
        priority =0       ignore this data or product
        priority =1       use this data or product in both manual and automated modes
        priority =2       use this data or product in manual mode only1
This avoids the need to make significant modifications to the configuration XML file.
The configuration files used by RTFV are listed below, along with the information they
contain. Tables 3 through 11 describe system configuration information that is unlikely to
change very often. Edit these files in order to add new forecast types, meteorological
quantities, domains, etc. to the system.

1
 Sites with priority=1 and 2 are used in point-based verification in both manual and automated modes;
however only priority=1 sites will be used to create time series plots in automated mode.


                                                  7
It is critical that the meteorological quantities, domains, forecast types, etc., be self-
consistent among the configuration files, as these are cross-referenced to each other
throughout RTFV. For example, if a forecast system listed in the forecast configuration
file rtfv_fcst.xml predicts 'precipitation', then that quantity should also be called
'precipitation' in the quantity configuration file rtfv_quantity.xml, rather than 'rainfall' or
'rain'.
Table 12 describes the configuration file for the GUI display. GUI configuration files
may be created and updated by users of RTFV, and may have user-specified names.
Table 3. Forecast information (<forecast> in rtfv_fcst.xml)
Variable (tag name)          Description
name                         name of forecast
namecode                     namecode
infile_prefix                prefix of input files (ex: 'swirls_qpf_')
forecast_type                forecast type (nowcast, NWP)
quantity                     meteorological quantity
spatial_type                 spatial type (point, cell, grid, area, line)
spatial_res                  spatial resolution (0 if not grid)
spatial_units                units of spatial resolution
domain                       domain to be used in verification (from rtfv_domain.xml)
time_res                     time resolution (min)
num_times                    number of forecast lead times
projection                   comma-separated values of forecast lead times (min)
fcst_frequency               forecast frequency (min)
accum_period                 accumulation period (min)
priority                     0=ignore, 1=manual mode, 2=manual and auto modes

Table 4. Observation information (<observation> in rtfv_obs.xml)
Variable (tag name)          Description
name                         name of observation
namecode                     namecode
infile_prefix                prefix of input files (ex: 'tifs_storm_')
quantity                     meteorological quantity
spatial_type                 spatial type (point, cell, grid, area, line)
spatial_res                  spatial resolution (0 if not grid)
spatial_units                units of spatial resolution
time_res                     time resolution (min)
obs_frequency                forecast frequency (min)
accum_period                 accumulation/averaging period (min)
priority                     0=ignore, 1=manual mode, 2=manual and auto modes

Cell-tracking nowcasts such as those made from TITAN are verified against cell
detections from the same system. To enable this to occur the information for the cell-
tracking system must appear both in the forecast and observation configuration files.




                                                8
Table 5. Meteorological quantity information (<quantity> in rtfv_quantity.xml)
Variable (tag name)           Description
name                          name of quantity
namecode                      namecode
units                         units
contour                       comma-separated values of contour intervals
threshold                     comma-separated values of thresholds for categorical statistics
palette                       name of color palette
stat                          type(s) of verification statistic (categorical, object, etc.)
priority                      0=ignore, 1=manual mode, 2=manual and auto modes

Table 6. Domain information (<domain> in rtfv_domain.xml)
Variable (tag name)           Description
name                          name of domain or region
namecode                      namecode
SWlat                         latitude of southwest corner
SWlon                         longitude of southwest corner
NElat                         latitude of northeast corner
NElon                         longitude of northeast corner
mapfile                       name of district map file (in data/map directory)
priority                      0=ignore, 1=manual mode, 2=manual and auto modes

Table 7. Site information (<site> in rtfv_site.xml)
Variable (tag name)           Description
name                          name of site
ID                            ID number or string
lat                           latitude
lon                           longitude
priority                      0=ignore, 1=manual mode, 2=manual and auto modes

Table 8. Color palette information (<palette> in rtfv_palette.xml)
Variable (tag name)           Description
name                          name of palette
rgb_file                      name of file with rgb values (in config subdirectory)
priority                      0=ignore, 1=manual mode, 2=manual and auto modes

The color palettes are held in text files in the config directory and have names ending in
_rgb.dat. Each palette file contains five lines of information:
    1.     256 1-byte values for red (r)
    2.     256 1-byte values for green (g)
    3.     256 1-byte values for blue (b)
    4.     number of contour intervals
    5.     values of contour intervals




                                                  9
To create a new palette file write the appropriate rgb values and contour intervals as
above. The format of the lines is unimportant as long as the values are separated by
spaces, commas or tabs.
0-based color indeces numbered from 30 to 59 should be the same for all palettes, and are
used for the statistics plots. Indeces 60-79 are used when plotting difference maps. Color
index 0 should have rgb values of [255,255,255] to give a white background, and color
index 255 should have rgb values of [0,0,0] to plot black lines. A few other color indeces
are reserved:
           97    missing data color
           98    borderlines color
           99    sea color
           100   land color
Table 9. Visual information (<vis> in rtfv_image.xml)
Variable (tag name)            Description
type                           type of visual element (map_type, corresp_plot, palette, etc.)
                               name of visual element
                                  for map_type: forecast, observations, difference, overlay
name
                                  for corresp_plot: scatterplot, Q-Q plot, cond. Q-Q plot
                                  for palette: name of palette (see Table 8)
xsize                          x dimension of image in pixels (a power of 2 is best)
ysize                          y dimension of image in pixels (a power of 2 is best)
priority                       0=ignore, 1=manual mode, 2=manual and auto modes

Table 10. Statistics information (<statistic> in rtfv_stat.xml)
Variable (tag name)            Description
name                           name of statistic
namecode                       namecode
type                           type of statistic (continuous, categorical, probabilistic)
priority                       0=ignore, 1=manual mode, 2=manual and auto modes

The names of the statistics specified above are also used in the statistics routines
aggregate_scores.pro, bootstrap_ci.pro and plot_stats.pro. It is important that they match
exactly.

Table 11. Diagnostic methods information (<diagnostic> in rtfv_diag.xml)
Variable (tag name)            Description
name                           name of diagnostic method
namecode                       namecode
param1                         value of 1st parameter        (meanings of parameters
param2                         value of 2nd parameter        depend on the details
                                         rd
param3                         value of 3 parameter          of the diagnostic method
                                         th
param4                         value of 4 parameter          being used)
param5                         value of 5th parameter
param6                         value of 6th parameter
param7                         value of 7th parameter
priority                       0=ignore, 1=manual mode, 2=manual and auto modes


                                                  10
The GUI configuration file describes the settings on the "visual", "statistics", and
"diagnostic" verification pages of the GUI (see section 5). It specifies which products to
display (automated and manual mode) and compute (manual mode). GUI configurations
can be named, saved, and loaded during an interactive GUI session. The
default_config.xml file specifies the initial settings when the GUI starts up, and also when
RTFV starts up.
Table 12. GUI configuration (<config> in default_config.xml)
Variable (tag
                      Description                                          Source
name)
name                  name of configuration
mode                  verification mode (automatic or manual)
start_date            verification starting date (yyyymmdd)
start_time            verification starting time (hhmmss)
end_date              verification ending date (yyyymmdd)
end_time              verification ending time (hhmmss)
quantity              meteorological quantity                              rtfv_quantity.xml
fcst_name             forecast name                                        rtfv_fcst.xml
projection            forecast projection (min); 9999 to specify 'all'     rtfv_fcst.xml
temporal_treatment    'instantaneous', 'accumulated' or 'average'
                      accumulation period if averaged or
accum_period
                      accumulated (min)
obs_name              name of verifying observations                       rtfv_obs.xml
domain                name of domain                                       rtfv_domain.xml
                      loop maps and plot time series by either:
                          single start time, varying lead times
approach                       ("model" view)
                          varying start times, single lead time
                               ("nowcast" view)
palette               color palette to use                                 rtfv_palette.xml
map1_type             left map display type                                rtfv_image.xml
map2_type             right map display type                               rtfv_image.xml
time_series           name of site for time series                         rtfv_site.xml
corresp_plot          type of correspondence plot                          rtfv_image.xml
product               statistical display (box-whisker or aggregate)
statistic             name of statistic to display                         rtfv_stat.xml
versus                x-axis of statistics plot (lead-time or intensity)
                      threshold used in categorical statistics; 9999
threshold                                                                  rtfv_quantity.xml
                      to specify 'all'
                      1 to include persistence forecast for
incl_persis           comparison, 0 to not (persistence not working
                      yet)
                      1 to make statistics plot and also compare
plot_stats            performance of all systems that used these
                      observations, 0 to not
method                diagnostic verification method to use                rtfv_diag.xml
param1                value of 1st parameter
param2                value of 2nd parameter


                                              11
        param3                value of 3rd parameter
        param4                value of 4th parameter
        param5                value of 5th parameter
        param6                value of 6th parameter
        param7                value of 7th parameter


 3.2     Internal data structures
        RTFV uses several internal data structures to carry system, configuration, observation,
        forecast, and image data. The structures are initialized in the init_rtfv.pro procedure.

3.2.1    System data
        The system data is held in the rtfv structure. The rtfv_* elements hold the possible values
        that the configuration values may take, and are read from the configuration files
        described in section 3.1. The remainder of the system information is read from the
        rtfv_startup.dat file described in section 3.1.
        Table 13. System data structure rtfv
        Element                     Description
        rtfv_quantity               (meteorological) quantity configurations
        rtfv_fcst                   forecast configurations
        rtfv_obs                    observation configurations
        rtfv_domain                 domain configurations
        rtfv_site                   site configurations
        rtfv_palette                palette configurations
        rtfv_vis                    visual product configurations
        rtfv_stat                   statistics configurations
        rtfv_diag                   diagnostic method configurations
        rtfv_glossary               blank observation structure defining contents of obs
        rtfv_dir                    name of directory to hold system configuration files
        config_dir                  name of directory to hold users' configuration files
                                    name of directory to hold forecast and observation files copied
        data_dir
                                    from their source
        image_dir                   local directory to hold recent verification graphical products
        statistics_dir              local directory to hold recent verification statistics data
        text_dir                    local directory to hold recent verification tables
        archive_dir                 archive directory to hold older verification products
                                    how often to check for new data (sec) - used by
        data_check_frequency
                                    trigger_rtfv.pro
        maxfcsts                    maximum # forecasts or observations to hold in memory
        maximages                   maximum # images of each type to hold in memory
        data_life                   maximum lifespan of data in memory (minutes of data)
        tolerance                   maximum time separating forecast and obs valid times
        aggregation_time            time over which statistics are aggregated (minutes)




                                                   12
3.2.2    Meteorological data
        All of the meteorological data are stored in a data structure called data. The metadata for
        all forecasts and observations are stored in registers described later in this section.
        Table 14. Contents of meteorological data structure data
        Element                      Description
        fcst                         pointer array containing forecast data structures
        fcst_register                pointer to forecast register
        fcst_counter                 number of forecasts currently held in memory
        current_fcst                 index of forecast being manipulated or verified
        blank_fcst                   blank forecast structure defining contents of fcst
        obs                          pointer array containing observation data structures
        obs_register                 pointer to observation register
        obs_counter                  number of observations currently held in memory
        current_obs                  index of observations being manipulated or verified
        blank_obs                    blank observation structure defining contents of obs

        The forecast and observation data structures are held in the data structure in pointer arrays
        called fcst and obs, described below. Pointers are used to reference the data in the fcst and
        obs structures to allow for variable data size and to speed data transfer between routines.

        Each forecast contains a single spatial forecast valid at a single time. Each
        observation contains a single spatial observation valid at a single time. This makes it
        very convenient for plotting and verifying spatially, but less convenient for working with
        time series at individual locations. Probability forecasts are read in as individual forecasts
        (e.g. prob_ge_1, prob_ge_5, etc.)
        Table 15. Contents of forecast data structure fcst
        Element                      Description
        name                         name of forecast
        forecast_type                forecast type (nowcast, NWP)
        quantity                     meteorological quantity
        spatial_type                 spatial type (point, cell, grid, area, line)
        projection                   forecast projection (min)
        accum_period                 accumulation/averaging period (min)
        grid                         pointer to gridded data
        cell                         pointer to cell data
        point                        pointer to point data
        area                         pointer to area data
        line                         pointer to line data
        match                        pointer to matched data (can be any spatial type)
        QC                           quality control done (0=no, 1=yes)
        base_date                    base date (yyyymmdd)
        base_time                    base time (hhmmss)
        valid_date                   valid date (yyyymmdd)
        valid_time                   valid time (hhmmss)
        read_date                    date of reading data into RTFV (yyyymmdd)
        read_time                    time of reading data into RTFV (hhmmss)


                                                       13
filename                      name of file that data was read from

Table 16. Contents of observation data structure obs

Element                       Description
name                          name of observation(s)
quantity                      meteorological quantity
spatial_type                  spatial type (point, cell, grid, area, line)
accum_period                  accumulation/averaging period (min)
grid                          pointer to gridded data
match_grid                    pointer to obs/analysis data regridded to match forecast
cell                          pointer to cell data
point                         pointer to point data
area                          pointer to area data
line                          pointer to line data
match                         pointer to matched data (can be any spatial type)
QC                            quality control done (0=no, 1=yes)
valid_date                    valid date (yyyymmdd)
valid_time                    valid time (hhmmss)
read_date                     date of reading data into RTFV (yyyymmdd)
read_time                     time of reading data into RTFV (hhmmss)
filename                      name of file that data was read from

When site data are read from file, they are put in the same order as the site list from
rtfv_site.xml. This eases the matching of observations and forecasts at sites.
The spatial data are loaded into the appropriate grid, cell, point, area or line elements,
which are themselves structure variables as described below.
Table 17. Description of grid structure variable (refer to load_grid_structure.pro)
Element                       Description
spatial_type                  'grid'
match_name                    name of data that this data is matched to
units                         units of data
spatial_units                 'km' or 'deg'
nx, ny                        grid dimensions in x and y directions
slon,slat                     starting longitude and latitude (southwest)
dx, dy                        grid increments in units of spatial_units
data                          2-dimensional array of data




                                                14
RTFV currently represents thunderstorm cells in keeping with the TIFS system, which
fits ellipses to cell areas determined by more sophisticated cell trackers such as TITAN or
WDSS. TIFS allows for forecaster modification of cell tracks, so both original and
modified speed and direction are included in the data structure, as well as a "visible" flag
to indicate whether the forecaster has culled the cell from the nowcast display (visible=0).
It is desirable in future to use a more general representation of those cell properties that
can be verified.

Table 18. Description of cell structure variable (refer to load_cell_structure.pro)
Element                       Description
spatial_type                  'cell'
match_name                    name of data that this data is matched to
nc                            number of cells in this data
ID                            array of cell ID numbers
clat                          array of cell center latitudes
clon                          array of cell center longitudes
major_axis                    array of ellipse major axes (km)
minor_axis                    array of ellipse minor axes (km)
orientation                   array of ellipse orientation angles (degrees true)
                              array of cell intensity magnitudes
magnitude
                              (e.g. 1=very weak, …, 5=very strong)
orig_speed                    array of unmodified cell speeds (km/hr)
orig_dir                      array of unmodified cell directions (degrees true)
mod_speed                     array of modified cell speeds (km/hr)
mod_dir                       array of modified cell directions (degrees true)
visible                       array of 'visible' indicators


Table 19. Description of point structure variable (refer to load_point_structure.pro)
Element                       Description
spatial_type                  'point'
match_name                    name of data that this data is matched to
units                         units of data
ns                            number of sites in this data
site                          array of site names
lat                           array of site latitudes
lon                           array of site longitudes
data                          array of site data

Table 20. Description of area structure variable (refer to load_area_structure.pro)
Element                       Description
spatial_type                  'area'
match_name                    name of data that this data is matched to
units                         units of data
name                          name of area (e.g., 'Severe thunderstorms likely')
magnitude                     area magnitude (threat index or something like that)
boundary                      array of vertices for the polygon bounding the area


                                                 15
        Table 21. Description of line structure variable (refer to load_line_structure.pro)
        Element                       Description
        spatial_type                  'line'
        match_name                    name of data that this data is matched to
        name                          name of line (e.g., 'Gust front')
        boundary                      array of vertices for the line

        Because spatial transformation may be required to verify the forecast the fcst and obs
        structures may contain more than one spatial representation of the same data. If a spatial
        transformation has been done the variable match_name stores the name of the data that the
        transformed data has been matched to.
        For example, suppose we want to verify a gridded forecast against a set of observations at
        points. The original forecast data would be stored in *fcst.grid, but the values interpolated
        to the locations of the observations would be stored in *fcst.point. match_name would be
        set to the name of the observation data, where the name is stored in the obs.name variable.
        See also Section 4.2.
        The observation and forecast data held in memory are indexed using registers,
        fcst_register and obs_register, that hold the most critical information (metadata) about the
        forecasts and observations. This is for both convenience and speed.
        Table 22. Information in fcst_register structure
        Element                       Description
        name                          name of forecast
        quantity                      meteorological quantity
        projection                    forecast projection (min)
        base_date                     base date (yyyymmdd)
        base_time                     base time (hhmmss)
        valid_date                    valid date (yyyymmdd)
        valid_time                    valid time (hhmmss)
        Table 23. Information in obs_register structure
        Element                       Description
        name                          name of observation
        quantity                      meteorological quantity
        valid_date                    valid date (yyyymmdd)
        valid_time                    valid time (hhmmss)



3.2.3     Image data
        Most verification generates graphical products which are saved in image files. Five types
        of images are produced by RTFV: map, TS (time series), corr (correspondence plot), stat
        (statistics), and diag (diagnostic method). For rapid display, including looping and
        zooming (not available yet) the map, TS, and corr images are held in memory in a date
        structure called image. As with the meteorological data, the metadata concerning the
        images are held in registers for each image type. To speed searching for images when



                                                         16
        running RTFV from the GUI, inventories of the contents of each image directory are
        maintained.

        Table 24. Information contained in the image data structure
        Element                       Description
        map_xsize, map_ysize          size of map images in pixels
        TS_xsize, TS_ysize            size of time series image in pixels
        corr_xsize, corr_ysize        size of correspondence plot image in pixels
        stat_xsize, stat_ysize        size of statistics plot image in pixels
        diag_xsize, diag_ysize        size of diagnostic plot image in pixels
        rgb                           256x3 array of rgb values in palette
        levels                        number of color levels in the palette
                                      maximum number of each type of image (map, corr, TS, stat,
        maximages
                                      diag) held in memory
        vis_inventory                 pointer array of visual image file lists
        stat_inventory                pointer array of statistics image file lists
        map                           pointer array of map images
        map_register                  pointer to map image register
        map_counter                   number of map images held in memory
        current_map                   index of current map image
        blank_map                     blank map image
        district_overlay              district borders overlay images
        TS                            pointer array of TS images
        TS_register                   pointer to TS image register
        TS_counter                    number of TS images held in memory
        current_TS                    index of current TS image
        blank_TS                      blank TS image
        corr                          pointer array of corr images
        corr_register                 pointer to corr image register
        corr_counter                  number of corr images held in memory
        current_corr                  index of current corr image
        blank_corr                    blank corr image


3.2.4     Statistics data
        A statistics data structure stats holds the quantitative verification results in memory for
        use in rapid plotting, tabulation, and aggregation. The data are also written to statistics
        files in the stats subdirectory.
        Table 25. Contents of statistics data structure stats
        Element                       Description
        fcst_name                     name of forecast
        quantity                      meteorological quantity
        spatial_type                  spatial type (point, cell, grid, area, line)
        projection                    forecast projection (min)
        temporal_treatment            instantaneous, average, accumulation
        accum_period                  accumulation/averaging period (min)
        obs_name                      name of observation used to verify forecast


                                                        17
        domain                       name of verification domain (ex: Beijing)
        valid_date                   valid date (yyyymmdd)
        valid_time                   valid time (hhmmss)
        statistics                   pointer to statistics data


 3.3     Global data structure - info
        The internal data structures described above are loaded into a global data structure called
        info which is passed between the major software components. The info structure is
        initialized in the init_rtfv.pro procedure.
        Table 26. Information contained in global info structure
        Element                      Description
        rtfv                         system data
        config                       configuration data
        data                         meteorological data
        image                        image data
        stats                        statistics data
        test                         0 to run for real, 1 to test system (prints diagnostic info)
        save                         0 to not save output, 1 to save output to rtfv.dat for testing
        log                          log information



 3.4     GUI data structure
        The GUI data structure encapsulates the main info structure, and adds to it several
        variables used to define the widget states. For details refer to the code for rtfv_gui.pro..

 3.5     Database description

3.5.1    Input data
        The input data normally reside on a data server, i.e., an external machine networked to
        the client machine running RTFV. The input directories on the server should be mapped
        to the client, and specified as obs_dir and fcst_dir in the start-up configuration file,
        rtfv_startup.dat (see Section 3.1). These directories should contain named subdirectories
        according to forecast and observed products, similar to the local fcst and obs directories
        shown here. RTFV expects that the product subdirectories will be further subdivided into
        date directories (ex: BJANC/20070717/).
        When testing RTFV apart from the data server, forecast and observation data must be
        read from files in named subdirectories of the local data/fcst and data/obs directories.
        The name of the file should ideally contain information about the product contained
        within, and the date and time corresponding to the data. An example might be
        swirls_qpf_20060731_063000.nc. The prefix (ex: swirls_qpf_ in this example) should be
        specified as the infile_prefix element in the configuration file rtfv_fcst.xml (or rtfv_obs.xml
        if the file contains observations) to indicate to RTFV the file name to search for. The file
        must end with the date/time and file type given as <yyyymmdd_hhmmss>.<extension>.


                                                      18
        The file formats of the data should conform to the standards specified for B08FDP, that
        is, netCDF for site-based observations and for gridded products, and XML for site-, line-,
        cell-, and area-based forecasts (see
        http://www.bom.gov.au/bmrc/wefor/projects/B08FDP/NowcastFormatSpecification.2.6.p
        df for more information, especially regarding the XML format structure).
        Each forecast file should contain a forecast for a single meteorological variable (e.g.
        precipitation) made at a single initial (base) date and time. It may contain forecasts for
        several lead times (10 min, 20 min, etc.), and possible also initial values at t=0. All points
        in space, whether the forecast corresponds to a grid, cells, lines, areas, or sites, are
        included in a single file.
        Each observation file contains data at a single (valid) date and time. All points in space,
        whether the forecast corresponds to a grid, cells, lines, areas, or sites, are included in a
        single file.
        An exception to the above file conventions is storm cell (track) data. In this case the
        observation file contains information on each cell concerning its ID, current location,
        strength, motion, and other characteristics. Because each cell's future location can be
        easily determined from the information in the observation file it is not necessary to create
        a forecast file. This means that both cell forecast and cell observations are read from
        observation files. In addition, the file may contain information on the past position of the
        cells.

3.5.2    Output
        RTFV writes output to files (as opposed to, say, a relational database). Graphical
        products are written to Portable Network Graphics (PNG) files in the image subdirectory
        in a directory structure reflecting the type of graphical product (vis, stat, or diag), the
        forecast name and quantity. Text tables of statistics are written to the text subdirectory.
        For example, map showing forecasts from SWIRLS precipitation forecasts are written to
        the <yyyymmdd>/image/vis/SWIRLS/precipitation directory, verification statistics plots
        written to the <yyyymmdd>/image/stat/SWIRLS/precipitation directory, diagnostic
        verification plots written to the <yyyymmdd>/image/diag/SWIRLS/precipitation
        directory, statistics tables written to the <yyyymmdd>/text/SWIRLS/precipitation
        directory, and so on. Whenever a new product file is written the appropriate product
        inventory is updated.
        Statistical data are written to ascii files in the stats subdirectory using a similar directory
        structure as used for graphical products. For example, statistics data files from CARDS
        thunderstorm forecasts are written to the <yyyymmdd>/stats/CARDS/thunderstorm
        directory. Numerical (statistical) products from the diagnostic verification methods are
        also written to the stats subdirectory (this may change).
        Forecast and observation data given at or interpolated to the sites listed in the
        rtfv_site.xml configuration file (see Table 7) are written to file as part of the verification
        processing. The directory structure is similar to that used for statistical products, e.g.
        <yyyymmdd>/TS/ SWIRLS/precipitation.
        The output file content is indicated by the name structure of the files, which takes the
        form:



                                                     19
      ff_qu_pj_tt_ap_ob_dm_sp.yyyymmdd.hhmmss.pc.png            and (graphics)
      ff_qu_pj_tt_ap_ob_dm_sp.latest.pc.png
      ff_qu_pj_tt_ap_ob_dm_sp.yyyymmdd.hhmmss.pc.txt           and    (tables)
      ff_qu_pj_tt_ap_ob_dm_sp.latest.pc.txt
      ff_qu_pj_tt_ap_ob_dm_sp.stats                                   (statistics)
The original graphic files are copied to "latest" graphics files for automatic display
purposes. The elements of the name structure are defined in the table below. The
namecodes are given in Section 3.2.
Table 27. Elements in output name structure.
Element       Description                               Examples
ff            forecast namecode (from rtfv_fcst.xml)    TIFS, SWIRL), etc.
                                                        Note: same as full name
qu            quantity namecode (from                   RR (precipitation), T (temperature),
              rtfv_quantity.xml)                        etc.
pj            forecast projection                       30min, 3hr, etc. or all if all
                                                        projections are included on one plot
tt            temporal treatment                        in (instantaneous) or ac
                                                        (accumulated)
ap            averaging/accumulation period (min)       0, 60, etc.
ob            observation namecode (from                aws (AWS), etc.
              rtfv_obs.xml)
dm            domain namecode (from                     BJ (Beijing), etc., or station ID for
              rtfv_domain.xml)                          time series plots
sp            spatial type                              gr (grid), po (point), ce (cell), li
                                                        (line), or ar (area)
yyyymmdd date for which product is valid                20060323
hhmmss   time (UTC) for which product is valid          091500
pc       graphical product information code,            map: for (forecast), obs
         depends on product type                           (observations), dif (difference),
                                                           ove (overlay)
                                                        time series: TS
                                                        correspondence plot: sca (scatter
                                                           plot), QQ (quantile-quantile
                                                           plot), con (conditional QQ plot)
                                                        statistic: composite of namecode of
                                                           statistic (from rtfv_stat.xml), plot
                                                           type (agg or box) and versus
                                                           projection or threshold (vp or vt)
                                                           (see stat_extension.pro)
                                                        diagnostic: namecode of diagnostic:
                                                           IS, PP, etc. (from rtfv_diag.xml)
The file contents of the local image and text directories are held in inventories in memory
to speed up the loading of imagery in the GUI.
For examples of RTFV output files refer to the RTFV User Manual.


                                               20
 4     Architectural and component-level design
      Only some components have descriptions written in this section. For others please refer
      to the comments in the source code.

4.1    Architecture diagrams
      A pictorial representation of the architecture is presented in the following flowcharts.

                                                              Start-up
                                                              rtfv.pro


                                                                               active_filter.pro
                                                       Read configuration,     load_base_map.pro
                                                       fill data structures,   load_district_map.pro
                                                      create data registers    load_site_map.pro
                                                            init_rtfv.pro      map_gshhs.pro
                                                                               count_products.pro



                                             no                                   yes
                                                            start GUI?



                           (background)                                                     (foreground)
                             do_rtfv.pro

                                                                                       (foreground)
                           Look for new files
                                                                                   Read image inventory
                           read_new_files.pro
                                                                                     get_inventory.pro




                         New observation data            no                               Load GUI      load_config.pro
                                                                   STOP
                          data files available?                                          rtfv_gui.pro   read_rtfv_palette.pro


       read_ncdf_grid.pro           yes        load_grid_structure.pro
      read_ncdf_point.pro                      load_cell_structure.pro
                                               load_point_structure.pro              GUI event handler
         read_xml_cell.pro  Read obs data                                                                     (See 4th page)
                                                                                   responds to user input
       read_xml_area.pro get_obs_from_file.pro load_area_structure.pro               rtfv_gui_event.pro
        read_xml_line.pro                      load_line_structure.pro
        read_lightning.pro                     write_ts_to_file.pro

                       Make input files for RTFV match_times.pro
                                                                                     Verify forecasts for
                        make_rtfv_inputfiles.pro match_fcst_to_obs.pro
                                                                                     specified forecast
                        write_rtfv_inputfile.pro check_verif_log.pro                                         (See next page)
                                                                                      and observation
                                                                                          run_rtfv.pro


                          Verify forecasts for
                          every forecast valid
                              at obs time         (See next page)
                               run_rtfv.pro



                                                 make_rtfv_statistics.pro
                         Make statistics plots plot_stats_ts.pro
                         run_rtfv_statsplots.pro clean_comparison.pro
                                                 plot_stats.pro


                                 STOP




                                                              21
           Main program for RTFV: run_rtfv.pro

                 Read input info
                  for verification
               read_rtfv_inputfile.pro



                Get obs data from
                 memory or file
                  get_obs.pro            match_times.pro
                                         get_obs_from_file.pro
                Get fcst data from       get_fcst_from_file.pro
                 memory or file          track_fcst.pro
                  get_fcst.pro
                                                 Transformation routines:
                                                  cell_to_point.pro
               Spatial transform if               grid_to_point.pro
               necessary to match                 area_to_point.pro
                  fcst and obs                    cell_area_to_grid.pro
               spatial_transform.pro              cell_to_grid.pro
                                                  grid-to_grid.pro
                                                  area_to_grid.pro
                                                  point_to_grid.pro
                 Match forecasts
                 and observations           domain_grid.pro
                 match_data.pro



              Make visual products
               make_rtfv_visual.pro
Loop
on input     Compute verif. statistics
files            verify_fcst.pro
                                            (see next page)
             Make statistical products
              make_rtfv_statistics.pro

             Make diagnostic products
             make_rtfv_diagnostics.pro



                 Clear some space
                  in memory                  free_stats_ptrs.pro
                 dump_old_data.pro



     no               finished
                  looping through
                    input files?

                           yes

           Put statistics and time series
            files in chronological order
                reorder_statsfiles.pro
                  reorder_tsfiles.pro



               Wait for more input...



                                            22
write_image_to_file.pro       Make visual products:
 product_directory.pro         make_rtfv_visual.pro

      match_data.pro
  read_rtfv_palette.pro        Make difference field
       plot_points.pro         make_difference.pro
      plot_ellipses.pro                                     Graphics files
         plot_area.pro         Plot data on map
        mask_grid.pro            plot_map.pro
         plot_wind.pro
        wind_barb.pro

                            Get point data to match sites
                                get_point_data.pro
    get_ts_from_file.pro                                    Graphics files
                               Plot time series
                                   plot_ts.pro




                            Get matched fcst & obs data
          scatterplot.pro        match_data.pro
         condqqplot.pro                                     Graphics files
                            Plot scatter or QQ diagram
                                  plot_corr.pro




                                 Verify forecasts:
                                  verify_fcst.pro

                                Compute statistics
    accumulate_rain.pro            verif_stats.pro
                                                            Statistics files
                                    verif_cell.pro
                                   verif_prob.pro



                            Make statistics products:
                             make_rtfv_statistics.pro

                            Aggregate scores over time
  get_stats_from_file.pro          get_stats.pro
  write_table_to_file.pro      aggregate_scores.pro
                                aggregate_jdist.pro


                                                            Graphics files
                                Draw statistics plots
                                    and tables
     stat_extension.pro           plot_stats.pro
                                  plot_pstats.pro             Table files




                                             23
                               Make diagnostic products:
                                make_rtfv_diagnotics.pro


                              Get matched fcst & obs data
                                    match_data.pro




                              Advanced diagnostic methods
                                           psd2.pro                        Results files
                                    object_oriented.pro
                                        cra_verif.pro
                                     intensity_scale.pro
                                  practically_perfect.pro               Graphics files
                               (not all code is written yet!)




                                                                read_rtfv_palette.pro match_filename.pro
         GUI event handler: rtfv_gui_event.pro                  check_compatibility.pro cleanup_rtfv.pro

                                                                                    file_code.pro
                 Load/save configuration       Load image(s) to display            get_image.pro
                      config_eh.pro            update_rtfv_images.pro         get_image_from_file.pro
                                                                                read_rtfv_table.pro

                                                 Interrupt calculations
                 Verification on demand
                                                      interrupt.pro
                    manual_verify.pro
                                                 (might not be possible)



      Add: accumulate_aws_rain.pro, get_hist_from_file.pro, qc_obs.pro, qc_fcst.pro

4.2    Utility routines
      Procedure                            Description
      read_simple_xml.pro                  Read data from XML file
      make_labels.pro                      Make string labels for numerical values
      projection_labels.pro                Convert between numerical and string representations
                                           of forecast projections
      convert_ddmonhhmm.pro                Convert between string DD Mon HH:mm YYYY and
                                           integer datetime
      ntimes.pro                           Counts number of time steps between two times
      nminutes.pro, nseconds.pro           Counts number minutes or seconds between two times
      addtime.pro                          Add days, hours and minutes to get a new time
      bootstrap_ci.pro                     Confidence interval estimated using bootstrap method
      color_bar.pro                        Makes a color bar (color scale)
      cw_bgroup2.pro                       IDL button group widget in two columns or rows
      image2gif.pro                        Converts image from window to gif file (Windows PC)
      set_pc.pro                           Sets 256 colors (Windows PC)
      regrid.pro                           Regrid data from one spatial resolution/domain to
                                           another
      replace_.pro, replace_sp.pro         In string replace space with underscore, replace



                                                  24
                                          underscore with space
      same_structure.pro                  Tests for equality between two structure variables
      set_qpf_colors.pro                  Loads color palette used by many applications
      set_plotstyle.pro                   Sets colors and symbols for each nowcast
      jday.pro, yyyymmdd.pro,             Convert between date (YYYYMMDD) and Julian day
      jdayx.pro                           (YYYYDDD)
      xy2latlon.pro                       Convert grid units from km to lat/lon
      convert_xml_datetime.pro            Converts YYYY-MM-DDThh:mm:ss to
                                          YYYYMMDD HHMMSS
      display_data.pro                    Reads data from files and plots to map
      count_products.pro                  Estimate the number of verification products
      openlog.pro, closelog.pro,          Open, close, and write to an output log file
      log.pro


4.3    Spatial transformation
      In order to verify a forecast with a given spatial type against observations of a different
      spatial type (for example, verifying a gridded precipitation forecast against point rain
      gauge observations) it may be necessary to spatially transform the forecasts to match with
      the observations. The routine that does this is spatial_transform.pro. It calls a number of
      specific routines that transform from gridded data to points, cells to grid, etc. The
      transformations are given in the table below.
      Table 28. Spatial transformations in RTFV

      Observa-    Forecast    Match
                                         Nowcast/forecast example                    Action
      tion type     type      type(s)
        point       point      point    Point forecast of wind speed      None required
                                        WDSS nowcast of Tstorm            Sample forecast cell to
        point       cell       point
                                        verified against spotter report   observation points
                                        Rainfields rainfall analysis      Sample forecast grid to
        point       grid       point
                                        verified against gauge obs        observation points
                   threat               Threat area for Tstorms           Sample forecast threat area
        point                  point
                    area                verified against spotter report   to observations points
         cell       point       X                                         STOP
                               cell,    TITAN cell track forecast
         cell       cell       grid     verified against TITAN cell       None required
                              (point)   analysis
         cell       grid        X                                         STOP
                   threat               TIFS threat area verified         Sample forecast threat area
         cell                  grid
                    area                against TIFS cell analysis        to cell gridpoints
                                        Point rain forecasts verified     Sample observation grid to
        grid        point      point
                                        against Rainfields analysis       forecast points
                               grid,    TIFS rain cells verified          Project forecast cell on
        grid        cell
                              (point)   against Rainfields analysis       observation grid




                                                   25
                                grid,    NWP verified against
          grid        grid                                             Possible regridding
                               (point)   Rainfields analysis
                     threat              TIFS threat area verified     Sample forecast threat area
          grid                  grid
                      area               against lightning analysis    to grid




 4.4     Memory management
        When there are a lot of different forecast products and/or observations, the memory in
        RTFV fills up rather quickly. In the rtfv_startup.dat file the data_life element specifies
        how many minutes worth of data should be kept before being released from memory (for
        example, 24 hrs is 1440 minutes). If the memory fills up too fast, decrease the value of
                                                           th
        data_life. In run_rtfv.pro there is a check every 5 verification job to see whether any data
        can be released from memory, and if so, the values referenced by the relevant data
        pointers are reset to 0.
        When the number of forecast data reaches 99% of the value of maxfcsts specified in
        rtfv_startup.dat then the oldest 20% of the data are removed from memory and the most
        recent 80% of the data is brought forward. Similar memory management occurs when the
        number of observation data reaches 99% of the value of maxfcsts.
        Memory management for image data is done similarly. When the number of images
        reaches 80% of the value of maximages specified in rtfv_startup.dat then the oldest 20%
        of images are dumped and the remaining ones moved forward. This applies only to
        images (potentially) displayed on the VISUAL page of the GUI. Images of statistics and
        diagnostic products are not held in memory but rather retrieved from file upon request.

 4.5     GUI
        (incomplete description)

4.5.1    Control panel
        The GUI has three tabbed pages, one each for visual, statistical, and diagnostic
        verification (see Section 5.2 for examples). Most of the controls for the configuration and
        display are in a control panel at the left, with remaining controls located at the top and
        bottom of the page. The upper part of the control panel is the same for all three pages
        while the lower part of each panel contains controls specific to each page.
        The controls for the visual, statistics, and diagnostics pages are synchronized, that is,
        when the user makes a change to an item (ex: Forecast quantity) in one page the change
        follows through to the other two pages. The uvalue of the button, droplist, etc. for the
        linked items is a structure in which the partners tag holds the widget IDs of the same
        button, droplist, etc. on the other two pages. That makes it easy to identify which widgets
        need to have their values changed when a change is made to the widget on one page.

4.5.2    Image referencing
        The images displayed in the GUI are referenced by a code that includes information
        about the forecast, observation, quantity, domain, etc. The image_code is used in the file



                                                   26
        name and also stored in the image register that keeps track of which images are stored in
        memory (see Section 3). When a new set of images is called for by either changing the
        GUI configuration while in automated mode, or selecting Verify now in manual mode, the
        routine update_rtfv_images.pro is called. This routine first checks the image register to
        see whether the images are already held in memory, and if not, tries to read the
        appropriate image files in the image/vis, image/stat, and image/diag directories.

4.5.3    Verification approach
        When there are multiple verification images to display on the visual page, two
        approaches to forecast timing can be taken. For nowcasts and other forecasts that are
        made quite often it is probably more typical to view the verification results for a given
        projection, for several base times. For forecasts like NWP that are made only infrequently
        it is more desirable to view the results for several projections from a single forecast run
        started from a single base time.
        Specifying a forecast_type of 'nowcast' in the rtfv_fcst.xml configuration file (see Section
        3.1) defaults to the varying forecasts, single projection option. Specifying a forecast_type
        of 'NWP' defaults to the single forecast run, varying projection option*. The user can
        override the default choice. The options refer only to the display, and not to the way in
        which forecasts are verified in automated mode.
        * This is currently disabled – all are varying forecasts, single projection.

 4.6     Software Interface Description
        RTFV is normally a stand-alone application. An IDL development license is required if
        changes are to be made to the code.
        There are a few interfaces to the outside world. One is via the schema used to validate
        forecast data files in XML format (normally this can be done using a local schema rather
        than an external URL).

4.6.1    External machine interfaces
        The startup file, rtfv_startup.dat, specifies the location of directory systems used by
        RTFV. These directory systems may contain input data (forecasts and observations), and
        may receive output products from RTFV (images, statistics, time series and tables). Some
        directory systems may be on the machine running the application, and some may be
        mounted drives on networked machines. See Section 3.5 for more information on the
        input and output directory systems.

4.6.2    External system interfaces
        RTFV verified forecasts produced by nowcast and forecast systems, against observations
        from a variety of sources. The directory structure and file names of the forecast and
        observation files should conform to the descriptions given in Section 3.5.1. Input data
        files are expected to be either in netCDF format (gridded forecasts and point
        observations) or XML format (point, cell, area forecasts) following the format
        specifications in
        http://www.bom.gov.au/bmrc/wefor/projects/B08FDP/NowcastFormatSpecification.2.6.p
        df.


                                                    27
 5     User interface design

5.1    Description of the user interface
      RTFV can be operated via a graphical user interface (GUI). The GUI contains interactive
      controls for the verification process, including which forecast products will be verified
      against which observations, what map, time series, and correspondence plots to display,
      which statistics to generate, and which diagnostic methods to apply. It also contains
      display panels for the verification products. The GUI has three tabbed pages, one each for
      visual, statistical, and diagnostic products (refer to section 2.1).
      The GUI is written in IDL and makes use of the IDL widget libraries. See section 2.2 for
      how to start the RTFV GUI.
      RTFV can also be operated from the command line without invoking the GUI. See
      section 2.2 for how to start RTFV from the command line.

5.2    Screen images
      The RTFV GUI opens with the visual page on top. Examples of the visual, statistics, and
      diagnostics pages are shown below. (These need to be streamlined! Too much space is
      taken up with controls. This may be done when recoding to Python.) REPLACE WITH
      NEW PICTURES!




                                                28
                                  1
                                                                    radar anal   LAPS05

                                 0.1
Probability of exceedance




                                0.01



                               0.001



                              0.0001



                             0.00001



                            0.000001
                                       0        10   20   30      40        50     60     70
                                                          Rain (mm/h)




                                           29
 5.3     Objects and actions
        All screen objects and actions are identified… (not written yet)

5.3.1    Keyboard shortcuts
        Exit                           cntl-Q
        Print                          cntl-P
        Go to VISUAL page              cntl-V
        Go to STATISTICS page          cntl-S
        Go to DIAGNOSTICS page         cntl-A

  6      Testing
        Test strategies and preliminary test case specifications are presented in this section. See
        also the next section on adding new components.

 6.1     Classes of tests

6.1.1    Testing background and foreground mode
        When the rtfv command finishes the data is normally removed from memory. However,
        the data can be saved in an IDL save file called rtfv_test.sav in the data subdirectory, and
        reread it for later use in testing. This is much faster than reading large amounts of original
        data from file. The save file can get fairly large (hundreds or even thousands of MB) so
        it's best not to keep too many copies around.
        The saved data contains a global data structure called info that is used by many RTFV
        routines (see Sections 3.2 and 3.3). It holds the configuration as well as all of the system,
        meteorological, and image data. It can be initialized using the init_rtfv command,
        described later. info is the input for the GUI procedure.
        For code testing it is advantageous to run the routines that are normally started inside the
        rtfv command. This way the data are kept in memory and can be used again and again
        for testing.

        To save the routines and data in rtfv_test.sav you can include /save in the call to rtfv:

        IDL> rtfv, restore=0, /test, /save

        If the main aim is to save the data then rtfv can be exited right away. The test
        keyword enables the printing of debug messages during the running of RTFV. /test
        also tells RTFV to read the data from the save file unless specifically told not to using
        restore=0.

        To test RTFV in foreground mode, first restore the data from the save file using the IDL
        restore command, then start the GUI:

        IDL> restore, '..\data\rtfv_test.sav'

        IDL> rtfv_gui, info




                                                   30
        To test RTFV in background mode without using the saved data (for example, suppose
        new data types have been added and you want to test the automated file reading routines):

        IDL> init_rtfv, 'rtfv_startup.dat', info, /test

        IDL> rtfv, /background, restore=0, /test

        Init.rtfv.pro initializes the system but the meteorological and image data structure will
        contain only blank data. When starting, RTFV will read all of the files in the data/fcst and
        data/obs directories. To speed things up for testing you may wish to move some of the
        data that you're not using into the more_fcsts or more_obs directories inside the data
        directory.
        To test RTFV in background mode using data from the save file, and write the existing
        (modified) data into the save file upon exiting:

        IDL> rtfv, /test, /save

6.1.2    Testing data reading and product generation routines
        Data reading and product generation routines can be tested without invoking the
        background or foreground RTFV, instead going straight to the run_rtfv.pro driver
        routine. Some test input files can be found in the test subdirectory of RTFV.
        To test new/modified code do something like the following:
        IDL> init_rtfv, 'rtfv_startup.dat', info
        IDL> inputfile='../test/rtfv_input.testswirls.txt'
        IDL> run_rtfv, inputfile, info
        Code can be debugged this way without going through the time consuming saving and
        restoring of the session. Test input files may be constructed for a variety of verification
        configurations.
        To more explicitly test the data reading routines for sample data files, the
        get_fcst_from_file.pro or get_obs_from_file.pro procedure can be run:

        IDL> get_fcst_from_file, info, <filename>

        The RTFV User Manual describes command-line procedures for running RTFV. In
        particular the routines do_rtfv.pro, run_rtfv_case.pro, and run_rtfv_statsplots.pro may
        be useful for testing the product generation routines.

6.1.3    GUI
        (No test routines written yet for the GUI!  )

 6.2     Expected software response
        RTFV should ideally run without error. Diagnostic messages are printed to the screen
        when run in development mode from an IDL session, and are printed to a log file when
        running either in development mode or as a stand-alone application.




                                                   31
6.3    Output logging
      RTFV prints information about its processing steps to three types of log files that are used
      to monitor the progress and health of the verification system.

      6.3.1 Observation log files
      Whenever a set of observations for a particular time are used to verify forecasts, the
      observation file name added to the observation log for that day. The name of the
      observation log file is RTFV/data/input/obs_files_yyyymmdd.txt, where yyyymmdd is the
      date of the observations.
      When RTFV runs in automated mode it normally checks the observation log file to see
      whether a set of observations has already been used. If so then new verifications are not
      required. This makes the verification processing more efficient. Manual running of RTFV
      performs verification even if the observations have already been used.

      6.3.2 Verification log files
      Every verification pair of observations and forecasts is logged in a verification log file.
      There is one verification file per day called verification_log.txt, located in the output
      directory RTFV/yyyymmdd. When running RTFV in automated mode a check is first
      made to the verification file to see whether the verification has already been done; if it
      has then it is not repeated.

      6.3.3 Process log files
      Output is sent to process log files to help with process monitoring and debugging. An
      entry in the startup file rtfv_startup.dat specifies the level of logging:
             0       don't log any output
             1       log only the most important information
             2       log all information
      (see Section 3.1). Output cycles through four log files according to the valid time
      (hhmmss) of the data being verified:
             000000 - 055959         data/log/log0.txt
             060000 - 115959         data/log/log1.txt
             120000 - 175959         data/log/log2.txt
             180000 - 235959         data/log/log3.txt
      When RTFV is started the log file corresponding to the current or specified valid time is
      started afresh. The log file is appended with process information during the six hour
      verification period and may grow to many thousands of lines. When one log file changes
      over to the next (for example, log0.txt changing to log1.txt for verification valid times
      later than 060000), the new log file is begun anew.
      A code snippet for logging process information, which must be a string or string array, is
      as follows:
             message='This is the message'
             log, info.log, 1, message




                                                 32
6.4    Performance bounds
      RTFV must "keep up" with the ingestion of new data. If the required products have not
      finished being generated before large amounts of new data comes along then the system
      may crash. Should this become a problem there are a few strategies that could be tried to
      make everything run faster:
      (a) Change the value of data_check_frequency to a larger number in the rtfv_startup.dat
      startup file. This will only work if the data overload occurs on an occasional rather than a
      regular basis, and will not work for scheduled automated running of RTFV.
      (b) Deactivate some of the less important statistics by changing their value of priority in
      the rtfv_stat.xml configuration file from 1 to 2. They will still get computed in the
      statistics code, but plots will not be made. Do not deactivate mean_forecast or
      mean_observed because they are used in the calculation of some other statistics.

      (c) If the confidence intervals are turned on in the statistics plots, these can be turned off
      by setting CI=0 in make_rtfv_statistics.pro. The bootstrap method for computing
      confidence intervals is time consuming. Binomial approximations could be used for some
      statistics.
      (d) Deactivate some of the less important forecast or observations sources by changing
      the value of priority in the rtfv_fcst.xml and rtfv_obs.xml configuration files from 1 to 2.

6.5    Critical components
      Data reading routines seem particularly sensitive to errors due to forecast and observation
      files not quite meeting the standard formats, having incomplete data, or being somehow
      corrupted. When new data becomes available it is critical to test the data reading routines
      (get_obs_from_file.pro, get_fcst_from_file.pro, and subroutines called within) before
      allowing the new data to become "operational". (See Section 6.1.2.) Debugging can be
      done by commenting out the error-catching code at the top of these routines.

 7     Appendices

7.1    Packaging and installation
      Table 29 summarizes the build and installation instructions for RTFV on Windows and
      Linux/Unix machines with and without IDL. The instructions are given in greater detail
      below.

      7.1.1 Building the application (executable) files
      RTFV should have two executable files, rtfv.sav, which runs RTFV from a GUI, and
      do_rtfv.sav, which is used for scheduled automated running of RTFV. The executable
      files are created in the IDL development environment from the RTFV project files,
      rtfv.prj and do_rtfv.prj. These project files must be located in the RTFV directory. The
      steps for building RTFV are as follows for both the Windows and Linux implementations
      of IDL:
         1. In the RTFV directory double-click on rtfv.prj to open up the project in an IDL
            session.



                                                  33
   2. In the Project menu select Options… Make sure that the Project Type is set to
      Save File (.sav)
   3. In the Project menu select Compile  All Files
   4. In the Project menu select Build. This will ask you if you want to reset the
      IDLDE session – click OK. This will create a file called rtfv.sav in the RTFV
      directory which is used to run the application.
Build do_rtfv.sav following the procedure above. Steps 1-4 only need to be done if there
have been modifications to the IDL code. It is not necessary to rebuild the application if
changes have been made only to the configuration files.
To run the application directly, select Run in the Project menu, or double-click on the
rtfv.sav file.
For further information consult the IDL Help information under Programming in IDL /
Creating Applications in IDL.

7.1.2 Exporting the RTFV application to a machine with no IDL license
RTFV can be run as a stand-alone application on a machine without an IDL license, but
not as a scheduled job. The new machine must have the same operating system as the
machine on which the application was created, e.g., an application created on a Windows
machine will not run on a Linux PC.
To export RTFV to a second computer follow these steps:
   1. Build the application following steps 1-4 in the previous section.
   2. In the Project menu select Export. Select the directory to which you want to
      export the project (you may need to create the directory first) and click OK. A
      dialog asks you if you want to export an IDL runtime distribution – select Yes.
      The default IDL export files should be fine – you do not need to change this.
   3. Copy all other input directories required by RTFV to the export directory. These
      will include the config, data, GUI, schema, and text directories described in
      Section 3. The src directory is not required but it is a good idea to include it so
      that errors may be traced to particular routines. The contents of the export
      directory are what will be copied to the second machine.
For further information consult the IDL Help information under Programming in IDL /
Creating Applications in IDL.
When the application is launched by double-clicking the rtfv.sav icon, an IDL splash
screen will appear. Click on this to begin RTFV.

7.1.3 Installation of RTFV on Windows PC
Copy the rtfv.sav file and all supporting RTFV and IDL directories from the export
directory (see previous section) to the RTFV directory on the machine to run RTFV.
Map the network drives for any external file systems to be used by RTFV. Edit the
rtfv_startup.dat configuration file to have the appropriate directories for the input


                                           34
forecasts and observations and output verification products. See Section 3 for information
on the file systems and directory structure.
If the machine to run RTFV has an IDL license specify the RTFV/src directory as a
library by selecting File/Preferences/Path in the IDL Development Environment.
Running RTFV as a scheduled job requires some further steps to be taken to produce a
batch file that the Windows scheduler will run. The batch file run_do_rtfv.bat contains a
single record instructing the IDLDE to execute a short procedure called run_do_rtfv.pro:
D:\PROGRA~1\RSI\IDL61\bin\bin.x86\idlde.exe "d:\\RTFV\\run_do_rtfv.pro"

The run_do_rtfv.pro procedure compiles do_rtfv, runs it, and exits IDL:
      !PATH +=";D:\\RTFV\\src"
      .compile do_rtfv
      do_rtfv
      exit, /NO_CONFIRM
Include the correct paths to the idlde.exe executable file and the RTFV\src directory in
these two short programs.
Use the Windows Scheduled Tasks wizard to schedule regular running of
run_do_rtfv.bat.

7.1.4 Installation of RTFV on Linux/Unix
Copy the rtfv.sav file and all supporting RTFV and IDL directories from the export
directory (see previous section) to the RTFV directory on the machine to run RTFV.
Edit the rtfv_startup.dat configuration file to have the appropriate directories for the input
forecasts and observations and output verification products.
If running on a machine with an IDL license, then the run_do_rtfv.sh script is used to
initiate do_rtfv.sav for scheduled automated running of RTFV:
      #!/bin/sh
      export IDL_DIR=/usr/local/rsi/idl
      export PATH=$PATH:/usr/local/rsi/idl/bin
      idl -rt='/home/eee/RTFV/do_rtfv.sav'
Edit run_do_rtfv.sh to have the appropriate paths to the IDL installation and the RTFV
directory containing the do_rtfv.sav executable.
Prepare any relevant binary masks for no-data regions.
Make an entry in the Cron Table to initiate regular running of run_do_rtfv.sh.




                                           35
Table 29. Modes of running, building, and installing RTFV
                                                                  Starting and
                       Operating Starting and ending from outside ending from
RTFV mode                                                                             Build instructions                                      Installation instructions
                        system   IDL                              an IDL
                                                                  session
                                                                                                                                            The RTFV directory should
                                                                                      The RTFV directory should contain the do_rtfv.prj
                                                                                                                                            contain the do_rtfv.prj,
                                                                                      file as well as the config, data, GUI, schema, src,
                                                                                                                                            do_rtfv.sav, and rtfv_startup.dat
                                                                                      and text directories.
                                                                                                                                            files as well as the config, data,
                                                                                      1. Double-click on do_rtfv.prj to open the project
Automated                         Schedule the task to run every 6                                                                          GUI, schema, src, and text
                                                                                      in an IDLDE session.
(background)           Windows      minutes by running the                                                                                  directories.
                                                                                      2. In the Project menu select Options… Make
verification via a     with IDL     Windows Scheduled Tasks                 ---                                                                 In rtfv_startup.dat set
                                                                                      sure that the Project Type is set to Save File (.sav)
scheduled job           license     wizard to start run_do_rtfv.bat                                                                         data_check_frequency to
                                                                                      3. In the Project menu select Compile  All Files
(do_rtfv)                                                                                                                                   99999.
                                                                                      4. In the Project menu select Build. This will ask
                                                                                                                                                Specify the RTFV/src
                                                                                      you if you want to reset the IDLDE session – click
                                                                                                                                            directory as a library by
                                                                                      OK. This will create a file called do_rtfv.sav which
                                                                                                                                            selecting File/Preferences/Path
                                                                                      is used to run the application.
                                                                                                                                            in the IDLDE.
                                                                     At the IDL                                                             The RTFV directory should
Automated                         Double-click on the rtfv.sav icon.
                       Windows                                       prompt type:     The RTFV directory should contain the rtfv.prj file contain the rtfv.prj, rtfv.sav, and
(background)                        Select Start automated
                       with IDL                                      IDL> rtfv        as well as the config, data, GUI, schema, src, and    rtfv_startup.dat files as well as
verification via GUI                verification. To end select End
                        license                                      To end type:     text directories.                                     the config, data, GUI, schema,
(rtfv)                              automated verification.
                                                                     IDL> exit        1. Double-click on rtfv.prj to open the project in an src, and text directories.
                                                                                      IDLDE session.                                            In rtfv_startup.dat set
                                                                                      2. In the Project menu select Options… Make           data_check_frequency to the
                                  Double-click on the rtfv.sav icon.   At the IDL     sure that the Project Type is set to Save File (.sav) number of seconds between
Manual (interactive)   Windows    Select Start manual verification     prompt type:   3. In the Project menu select Compile  All Files checks for new data (6
verification via GUI   with IDL   (GUI). To end select Exit from the   IDL> rtfv      4. In the Project menu select Build. This will ask    minutes=360 sec).
(rtfv)                  license   file menu or by clicking the Exit    To end type:   you if you want to reset the IDLDE session – click        Specify the RTFV/src
                                  button at the bottom of the GUI.     IDL> exit      OK. This will create a file called rtfv.sav which is  directory as a library by
                                                                                      used to run the application.                          selecting File/Preferences/Path
                                                                                                                                            in the IDLDE.
Manual verification    Windows
from the command       with IDL                   ---                                 as above                                                as above
line                    license




                                                36
                                                                     Starting and
                     Operating Starting and ending from outside ending from
RTFV mode                                                                           Build instructions                                      Installation instructions
                       system    IDL                                 an IDL
                                                                     session
Automated                        Cannot do this because IDL virtual
                      Windows
(background)                      machine throws up a splash screen
                     without IDL                                           ---                               ---                                           ---
verification via a                 that must be clicked in order to
                       license
scheduled job                            start the application
                                 Drag the rtfv.sav icon over the IDL                Follow the instructions above for building the          Install the IDL Virtual Machine
Automated
                      Windows Virtual Machine icon. Select Start                    executable rtfv.sav. Then:                              on the computer (see
(background)
                     without IDL automated verification. To end            ---      1. In the Project menu select Export. Select the        http://www.ittvis.com/idlvm/).
verification via GUI
                       license   select End automated                               directory to which you want to export the project           Copy the contents of the
(rtfv)
                                 verification.                                      (you may need to create the directory first) and        export directory (see Build
                                                                                    click OK. A dialog asks you if you want to export       Instructions) into the RTFV
                                 Drag the rtfv.sav icon over the IDL                an IDL runtime distribution – select Yes. The           directory on the machine that
                                 Virtual Machine icon. Select Start                 default IDL export files should be fine – you do not    will run RTFV.
Manual (interactive) Windows                                                        need to change this.                                        In rtfv_startup.dat set
                                 manual verification (GUI). To
verification via GUI without IDL                                           ---      2. Copy all other input directories required by         data_check_frequency to the
                                 end select Exit from the file menu
(rtfv)                 license                                                      RTFV to the export directory. These will include        number of seconds between
                                 or by clicking the Exit button at the
                                 bottom of the GUI.                                 the config, data, GUI, schema, src, and text            checks for new data (6
                                                                                    directories.                                            minutes=360 sec).
                                  In the cron table add entries to start            The RTFV directory should contain the do_rtfv.prj       The RTFV directory should
                                  the do_rtfv.csh script every 6                    file as well as the config, data, GUI, schema, src,     contain the do_rtfv.prj,
                                  minutes:                                          and text directories.                                   do_rtfv.sav, and rtfv_startup.dat
                                  6 * * * *                                         1. Open an IDLDE session                                files as well as the config, data,
Automated                         /$HOME/RTFV/src/script/                           2. In the File menu select Open Project. Locate         GUI, schema, src, and text
(background)                      do_rtfv.csh                                       and open do_rtfv.prj.                                   directories.
                      Linux with
verification via a                12 * * * *                               ---      3. In the Project menu select Options… Make                 In rtfv_startup.dat set
                      IDL license
scheduled job                     /$HOME/RTFV/src/script/                           sure that the Project Type is set to Save File (.sav)   data_check_frequency to
(do_rtfv)                         do_rtfv.csh                                       4. In the Project menu select Compile  All Files       99999.
                                  18 * * * *                                        5. In the Project menu select Build. This will ask          Specify the RTFV/src
                                  /$HOME/RTFV/src/script/                           you if you want to reset the IDLDE session – click      directory as a library by
                                  do_rtfv.csh                                       OK. This will create a file called do_rtfv.sav which    selecting File/Preferences/Path
                                  (etc.)                                            is used to run the application.                         in the IDLDE.




                                                  37
                                                                 Starting and
                     Operating Starting and ending from outside ending from
RTFV mode                                                                            Build instructions                                      Installation instructions
                       system    IDL                             an IDL
                                                                 session
                                 To start from the linux prompt: At the IDL                                                                  The RTFV directory should
Automated                                                                            The RTFV directory should contain the rtfv.prj file
                                 /$HOME/RTFV/src/script/ prompt type:                                                                        contain the rtfv.prj, rtfv.sav, and
(background)         Linux with                                                      as well as the config, data, GUI, schema, src, and
                                 rtfv.csh                        IDL> rtfv                                                                   rtfv_startup.dat files as well as
verification via GUI IDL license                                                     text directories.
                                 To end type:                    To end type:                                                                the config, data, GUI, schema,
(rtfv)                                                                               1. Open an IDLDE session
                                 IDL> exit                       IDL> exit                                                                   src, and text directories.
                                                                                     2. In the File menu select Open Project. Locate
                                                                                                                                                 In rtfv_startup.dat set
                                                                                     and open rtfv.prj.
                                                                                                                                             data_check_frequency to the
                                                                                     3. In the Project menu select Options… Make
                                 To start from the linux prompt:      At the IDL                                                             number of seconds between
                                                                                     sure that the Project Type is set to Save File (.sav)
Manual (interactive)             /$HOME/RTFV/src/script/              prompt type:   4. In the Project menu select Compile  All Files
                                                                                                                                             checks for new data (6
                     Linux with
verification via GUI             rtfv.csh                             IDL> rtfv      5. In the Project menu select Build. This will ask
                                                                                                                                             minutes=360 sec).
                     IDL license                                                                                                                 Specify the RTFV/src
(rtfv)                           To end type:                         To end type:   you if you want to reset the IDLDE session – click
                                 IDL> exit                            IDL> exit                                                              directory as a library by
                                                                                     OK. This will create a file called rtfv.sav which is
                                                                                                                                             selecting File/Preferences/Path
                                                                                     used to run the application.
                                                                                                                                             in the IDLDE.
Automated
                                 Cannot do this because IDL virtual
(background)            Linux
                                 machine throws up a splash screen
verification via a   without IDL                                           ---                                ---                                            ---
                                  that must be clicked in order to
scheduled job          license
                                        start the application
(do_rtfv)
                                 Launch the IDL Virtual Machine                      Follow the instructions above for building the          Install the IDL Virtual Machine
                                 from the command line:                              executable rtfv.sav. Then:                              on the computer (see
Automated                        idl –vm=C:/RTFV/
                        Linux                                                        1. In the Project menu select Export. Select the        http://www.ittvis.com/idlvm/).
(background)
                     without IDL rtfv.sav. Select Start                    ---       directory to which you want to export the project          Copy the contents of the
verification via GUI
                       license   automated verification. To end                      (you may need to create the directory first) and        export directory (see Build
(rtfv)
                                 select End automated                                click OK. A dialog asks you if you want to export       Instructions) into the RTFV
                                 verification.                                       an IDL runtime distribution – select Yes. The           directory on the machine that




                                               38
                                                                   Starting and
                     Operating Starting and ending from outside ending from
RTFV mode                                                                         Build instructions                                     Installation instructions
                       system    IDL                               an IDL
                                                                   session
                                 Launch the IDL Virtual Machine                   default IDL export files should be fine – you do not   will run RTFV.
                                 from the command line:                           need to change this.                                      In rtfv_startup.dat set
                                 idl –vm=C:/RTFV/                                 2. Copy all other input directories required by        data_check_frequency to the
Manual (interactive)    Linux
                                 rtfv.sav. Select Start manual                    RTFV to the export directory. These will include       number of seconds between
verification via GUI without IDL                                         ---
                                 verification (GUI). To end select                the config, data, GUI, schema, src, and text           checks for new data (6
(rtfv)                 license
                                 Exit from the file menu or by                    directories.                                           minutes=360 sec).
                                 clicking the Exit button at the
                                 bottom of the GUI.




                                              39
 7.2     Adding and removing components
        RTFV is designed to make it easy (or at least not too difficult) to modify and extend. In
        most cases it is simply a matter of adding new information to the configuration files and
        setting its priority to 1 (use in automated and manual mode) or 2 (use in manual mode
        only). If new data types are sufficiently different from existing types then it may be
        necessary to modify the code.
        To remove components simply change its priority to 0 in the relevant configuration file.

7.2.1     New forecast information
        Add the relevant information to the rtfv_fcst.xml configuration file. Test reading of new
        forecast data from file using get_fcst_from_file.pro. Routines that may require editing
        include get_fcst_from_file.pro, read_new_files.pro.

7.2.2     New observation information
        Add the relevant information to the rtfv_obs.xml configuration file. Test reading of new
        observation data from file using get_obs_from_file.pro. Routines that may require editing
        include get_obs_from_file.pro, read_new_files.pro.

7.2.3    New meteorological quantities
        Add the relevant information to the rtfv_quantity.xml configuration file. Test reading of
        new meteorological quantities from file using get_fcst_from_file.pro and
        get_obs_from_file.pro.

7.2.4     New spatial domains
        Add the relevant information to the rtfv_domain.xml configuration file. Test by selecting
        the new domain in the GUI and performing a manual verification on the VISUAL page.

7.2.5     New sites
        Add the relevant information to the rtfv_site.xml configuration file. Test by selecting the
        new site(s) in the GUI and performing a manual verification on the VISUAL page. The
        time series displays should reflect forecast performance at the relevant site(s), provided
        that both forecast and observation data are available at the site(s).

7.2.6    New color palettes
        Add the relevant information to the rtfv_palette.xml configuration file. Test by setting one
        of the quantities in rtfv_quantity.xml to use the new palette, and performing a manual
        verification of this quantity on the VISUAL page.

7.2.7     New statistics
        Add the relevant information to the rtfv_stats.xml configuration file. Add/modify code in
        the appropriate statistics handling routines (verify_stats.pro, aggregate_scores.pro,
        bootstrap_ci.pro and plot_stats.pro). If the new statistics are sufficiently different from
        existing statistics it may be desirable to write one or more new statistics routines.




                                                   40
7.2.8     New diagnostic methods
        Add the relevant information to the rtfv_diag.xml configuration file. Add/modify code in
        the appropriate diagnostic methods handling routines

 7.3     Adding and updating code
        To install new code in RTFV, copy the procedure(s) to the rtfv/src directory. Add any
        new routines to the rtfv.prj and do_rtfv.prj projects in the group "src". Then build the
        application as instructed in Section 7.1.

 7.4     Supplementary information

7.4.1    Machine at Beijing Meteorological Bureau
        RTFV runs on the PC in room 1008. It reads data from the product server which has IP
        number of 172.18.9.202 when referenced from within BMB. For RTFV the relevant
        subdirectory is /data2/b08fdp/RTFV.
        The RTFV machine has the following login and password:
              Login: fdp_rtfv
              Password: rtfv.
        This information is required when setting up the scheduled RTFV task on the RTFV
        machine. It may also be required for logging in – this hasn't been checked.




                                                   41

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:2/1/2012
language:
pages:42