NOAA’s National Weather Service Accessing the GRIB2 encoded by ejy64045

VIEWS: 103 PAGES: 26

									NOAA’s
National Weather Service
Accessing the GRIB2 encoded
National Digital Forecast Database

Arthur Taylor
NOAA/NWS/Meteorological Development Laboratory

NOAA WebShop 2004

July 28, 2004
                NDFD: What is it?

     • The National Digital Forecast Database (NDFD), is a mosaic of
       the National Weather Service’s (NWS) digital forecasts
     • It is available free to all.
     • It is updated hourly with the best forecast available to the NWS.
     • It contains the following variables at various temporal steps out
       to 7 days:
        – Maximum Temperature, Minimum Temperature, Probability of
          Precipitation, Temperature, Wind Direction, Wind Speed, Dew Point
          Temperature, Sky Coverage, Quantity of Precipitation, Snow Amount,
          Wave Height, and Weather (which includes visibility).
   Advantage?
     • The NDFD provides a single place to go in order to get the best
       forecast available from the NWS each hour.



NDFD / Degrib
                NDFD: How do I get it?

     • The NDFD is available in the World Meteorological
       Organization’s (WMO) GRIB (version 2) format at:
        – http://weather.noaa.gov/pub/SL.us008001/ST.expr/DF.gr2/DC.ndfd/
        – ftp://tgftp.nws.noaa.gov/SL.us008001/ST.expr/DF.gr2/DC.ndfd/
     • It is broken up into various geographic areas which include:
        – The Continental U.S. (CONUS),
        – Various predefined sectors which are: Pacific North West, Pacific South
          West, Northern Rockies, Central Rockies, Southern Rockies, Northern
          Plains, Central Plains, Southern Plains, Upper Mississippi Valley, Central
          Mississippi Valley, Southern Mississippi Valley, Central Great Lakes,
          Eastern Great Lakes, North East, South East, and Mid Atlantic sectors.
        – Puerto Rico.
     • Inside a sector it is broken up into the various weather
       elements.



NDFD / Degrib
                NDFD: Which Sector?




NDFD / Degrib
                NDFD: Custom Sector?

     • The predefined sectors are created hourly, and are already
       waiting to be downloaded.
   Advantage: Speed.
     • Data is already available and ready to transmit when the request
       for the data arrives on the server.
   Disadvantage: Flexibility.
     • If a user is interested in a specific area, he/she will either need
       to download multiple sectors, or download excess information.
   Alternate solution?
     • Provide a web service which allows the user to specify the area
       they are interested in. The service encodes just that portion of
       the NDFD in GRIB2 and sends it back.



NDFD / Degrib
                NDFD: Custom Sector.

     • The experimental custom sector web service is available at:
        – http://ndfd.weather.gov/cgi-bin/ndfd/gribcut
     • The following call gets the maximum temperature in a subgrid which is
       defined with its lower left corner at lat=30, lon=-100, and its upper right
       corner at lat=40, lon=-90:
        – gribcut?var=maxt&lat1=30&lon1=-100&lat2=40&lon2=-90
     • For more information see:
        – http://www.nws.noaa.gov/ndfd/grib.htm
   Advantage: Flexible.
     • The user can choose any latitude longitude box. Note the box is not
       allowed to be larger than 15 degrees on a side.
   Disadvantage: Speed.
     • The data is encoded upon request, which means some work on the
       server side.
     • In a high demand period this method may slow down.
     • If this becomes an issue, various methods can be used to speed it up.


NDFD / Degrib
                NDFD: How do I use it?

   Since NDFD uses the WMO’s GRIB2 format, how do I
   read a GRIB2 file?
   The NWS has provided NDFD users a program (“degrib”) to
   easily download and locally decode GRIB2 files see:
     • http://www.nws.noaa.gov/mdl/NDFD_GRIB2Decoder/
     • “degrib” can be installed on most local operating systems (Windows,
       Linux, UNIX).
     • It provides a graphical user interface for specifying products to
       download and decode, along with some image generation capability.
   The software is based on the GRIB2 library available at:
     • http://www.nws.noaa.gov/mdl/iwt/grib2/decoder.htm
     • http://www.nws.noaa.gov/mdl/iwt/grib2/encoder.htm


NDFD / Degrib
                Degrib: What can it do?

     • Download NDFD Data
     • Inventory GRIB and GRIB2 files
     • Convert GRIB and GRIB2 data to other formats
        –   ESRI .shp (point, small polygon, large polygon) files
        –   .flt files (used by GrADS and ESRI Spatial Analyst)
        –   netCDF files
        –   .csv files
     • Probe GRIB and GRIB2 data at a given point.
     • Interpret the meta-data provided in the GRIB and GRIB2 files.

     Note: Both a Graphical User Interface (GUI) and command line
      interface are provided. See “c:/ndfd/degrib15/docs/degrib.txt”
      for information on the command line interface.


NDFD / Degrib
                Degrib: Installation

   •Degrib can be downloaded from:
        – http://www.nws.noaa.gov/mdl/NDFD_GRIB2Decoder/register.htm
     • On the download page, MS-Windows users should get “ndfd-
       demo.exe”, which contains:
           • An installation wizard
           • Compiled code
           • Source code for the libraries, and the “degrib” and “tkdegrib” programs
             (superImageGen and htmlmaker source code are not provided)
           • Documentation
     • Unix users should get “degrib-unix.tar.gz”, which contains:
           • Source code for the libraries and the “degrib” and “tkdegrib” programs
           • Documentation.
     • To compile the code you need:
           • FORTRAN (g77 should work) : for the decoder library
           • C (gcc should work) : for the “drivers” and projection library
           • Optional : Tcl/Tk (free from http://scriptics.com) : for the Graphical User Interface



NDFD / Degrib
                Degrib: Data download

  Step 1: Download some data.
   • The GUI version of degrib,
     “tkdegrib”, provides an
     interface to get NDFD either
     via ftp or http.
      – Highlight the desired sector
      – Click on either “Download by ftp”
        or “Download by http”
   • Alternatively, you can get the
     data directly from:
      – http://weather.noaa.gov/pub/SL.u
        s008001/ST.expr/DF.gr2/DC.ndf
        d/
      – ftp://tgftp.nws.noaa.gov/SL.us00
        8001/ST.expr/DF.gr2/DC.ndfd/




NDFD / Degrib
                Degrib: Data download

  Step 1 continued:
   • The GUI version of degrib now provides a
     “custom” sector, which you control by:
      – Use the “file->configure” menu.
      – Choose the “Custom” tab.
      – Enter in the desired area, press ok.


   • In addition there is a command line method
     to download various sectors and variables,
     using:
      – cd C:\ndfd\degrib15\bin\
      – tcldegrib web.tcl




NDFD / Degrib
                Degrib: Convert to .shp file

  To convert to .shp format:
      – Double click on the GRIB2 file (top
        pane)
      – Select message to convert (middle
        pane)
      – Choose an “output file name”
      – Click on “Generate .shp file”
   • “Small Polygon” creates better
     images in ESRI and is easier to
     manipulate, but it is larger than
     the “Point .shp”
   • “Large Polygon” merges
     polygons together to create
     smaller files, but makes grid to
     grid comparison more difficult.
   • “Include Missing Values” allows
     you to study the entire grid, or
     limit the result to where data
     exists


NDFD / Degrib
                  CONUS MaxT in ArcView




          After converting the grid, start up ESRI ArcView, then load the layer
          (.shp file) that you just created. Use the color legend found in:
          C:/ndfd/degrib15/arcview/poly_legend (for polygon shapefiles)
          C:/ndfd/degrib15/arcview/point_legend (for point shapefiles)


NDFD / Degrib
                Polygon vs Point, DC area




NDFD / Degrib
                NDFD CONUS Map Projection

   For CONUS (continental U.S.), NDFD uses a Lambert Conformal
   Conic Map Projection:
     • Tangent latitutude 25, orientation longitude -95, mesh latitude 25, mesh
       size 5079.406 m.
     • In ESRI ArcMap that would be:
        – 1st Parallel = 2nd Parallel = 25 degrees north (tangent latitude)
        – Lon of center of projection = -95 degrees (orientation longitude)
        – Lat of center of projection = 25 degrees (tangent latitude)
        – False easting = 0 m, False northing = 0 m, X Shift = 0, Y Shift = 0
   In addition, NDFD uses a spherical earth with radius 6371.2 km
   To assist ESRI ArcView 3.x, degrib creates a .ave (ArcView Avenue
   script) to set the projection and radius correctly.
   For ArcGIS, please see /degrib/arcview/prj.adf, and ndfd.prj



NDFD / Degrib
                Un-projected vs Projected




NDFD / Degrib
                Degrib: Convert to .flt file

   To convert to .flt file (for use
   with GrADS or ESRI Spatial
   Analyst):
        – Double click on the GRIB2 file
          (top pane)
        – Select message to convert
          (middle pane)
        – Choose an “output file name”
        – Click on “Generate .flt file”
     • “Grid” allows you to select
       what kind of interpolation to
       do (or keep the original
       projection).
     • “GrADS .ctl file”: creates a
       control file which GrADS can
       use to access the .flt file
     • “M.S.B. first”: creates the .flt
       files in “Big Endian” format.


NDFD / Degrib
                CONUS MaxT in GrADS




NDFD / Degrib
                Degrib: Convert to netCDF file

   To convert to netCDF file:
        – Double click on the GRIB2 file
          (top pane)
        – Select message to convert
          (middle pane)
        – Choose an “output file name”
        – Click on “Generate NetCDF
          (.nc) file”
     • This creates a netCDF file
       according to the “CF”
       metadata conventions.
     • This converts 1 GRIB
       message to 1 netCDF file.
     • An improvement would be
       for multiple GRIB messages
       to go to 1 netCDF file.



NDFD / Degrib
                CONUS MinT in NetCDF




NDFD / Degrib
                Degrib: Convert to .csv file

   To convert to CSV file:
        – Double click on the GRIB2 file
          (top pane)
        – Select message to convert
          (middle pane)
        – Choose an “output file name”
        – Click on “Generate .csv files”
     • This creates a comma
       separated text file which can
       be worked with in excel.
     • Note: Excel has problems
       loading the CONUS NDFD, as
       it appears to be limited to
       65,536 records, while the
       NDFD typically has 739,297
       records



NDFD / Degrib
                CONUS MinT in .csv in Excel




                             X




NDFD / Degrib
                Degrib: Probe point

   Degrib also provides the ability to probe a GRIB file at a
   set of latitude longitude locations. It computes the
   value at that spot, and returns a text message. Note:
   this is not available in the GUI. To do so from the
   command line:
        – C:\ndfd\degrib\bin\degrib maxt.bin –P –pnt 38.99,-77.03
        – C:\ndfd\degrib\bin\degrib maxt.bin –P –pntFile point.txt
     • The first example probes just 1 point, while the second reads a
       file which is of the form: StationID, lat, lon per line. The second
       form allows for numerous stations.
     • By default, probe chooses the “nearest” grid cell, but using the
       “-Interp” option, causes it to perform bi-linear interpolation.
   See (/degrib/docs/degrib.txt for more help)


NDFD / Degrib
                   Probe point (MaxT, MinT, PoP)

 element, unit, refTime, validTime, (38.993600,-   element, unit, refTime, validTime, (38.993600,-
 77.022400)                                        77.022400)
                                                   PoP12, [%], 200307212300, 200307220000, 39.000
 MaxT, [F], 200307212300, 200307230000, 86.882
                                                   PoP12, [%], 200307212300, 200307221200, 39.000
 MaxT, [F], 200307212300, 200307240000, 82.375     PoP12, [%], 200307212300, 200307230000, 50.000
 MaxT, [F], 200307212300, 200307250000, 85.233     PoP12, [%], 200307212300, 200307231200, 50.000
                                                   PoP12, [%], 200307212300, 200307240000, 50.000
 MaxT, [F], 200307212300, 200307260000, 87.753
                                                   PoP12, [%], 200307212300, 200307241200, 50.000
 MaxT, [F], 200307212300, 200307270000, 89.395     PoP12, [%], 200307212300, 200307250000, 20.000
 MaxT, [F], 200307212300, 200307280000, 87.775     PoP12, [%], 200307212300, 200307251200, 20.000
                                                   PoP12, [%], 200307212300, 200307260000, 14.000
 MaxT, [F], 200307212300, 200307290000, 86.800
                                                   PoP12, [%], 200307212300, 200307261200, 5.000
                                                   PoP12, [%], 200307212300, 200307270000, 5.000
 element, unit, refTime, validTime, (38.993600,-   PoP12, [%], 200307212300, 200307271200, 11.000
 77.022400)                                        PoP12, [%], 200307212300, 200307280000, 11.000
 MinT, [F], 200307212300, 200307221200, 72.625     PoP12, [%], 200307212300, 200307281200, 21.000
                                                   PoP12, [%], 200307212300, 200307290000, 21.000
 MinT, [F], 200307212300, 200307231200, 73.165
 MinT, [F], 200307212300, 200307241200, 69.025
 MinT, [F], 200307212300, 200307251200, 65.965
 MinT, [F], 200307212300, 200307261200, 67.945
 MinT, [F], 200307212300, 200307271200, 69.745
 MinT, [F], 200307212300, 200307281200, 69.025


NDFD / Degrib
                NDFD, GRIB2 & Weather

   To encode Weather (Wx), NDFD disseminates “weather
   strings”.
     • Advantage: A flexible format that can encode a description of
       weather such as “Chance of thunderstorms and chance of
       heavy rain showers”
   Difficulty: A “weather string” is not a number, so it is
   challenging to store in GRIB2.
     • Solution: Use GRIB2 section 2 to provide an ASCII look up table,
       and store the numeric values in the regular GRIB2
        – Note: each weather grid, for each forecast projection, has a different
          ASCII look up table, so without section 2 it has no meaning.
     • Some Questions:
        – What does “Chc:T:<NoInten>:<NoVis>:^Chc:RW:+:<NoVis>:” mean, and
          how, for example, is an Emergency Manager supposed to know that?
        – What can be done for the .flt file?


NDFD / Degrib
                Degrib: Future?

   Maintain the program by keeping up with NDFD as more
   variables / sectors are introduced.
        – In order to inform users of updates, degrib has an “announcement
          mailing list”, which at the last release had 434 people on it.
   Improvements: Based on user feedback, current
   thoughts are:
        –   Create some form of graphical probe point capability
        –   Add some way for UNIX users (without GrADS) to plot the images
        –   Add some way for users to zoom in on a map
        –   Improve the NetCDF and GrADS file outputs to contain multiple GRIB
            messages.




NDFD / Degrib

								
To top