ESP w GFS and CFS by dfhdhdhdhjr


									CBRFC and OHD Efforts to Advance
 Long Lead Hydrologic Prediction
                  Andy Wood
      NWS Colorado Basin River Forecast Center

               Julie Demargne
        NWS Office of Hydrologic Development

 SI/Y2 Climate and Streamflow Forecast Workshop
       Salt Lake City, UT – March 21-22, 2011
                   Let’s not forget...
 CBRFC/NRCS official forecasts have plenty of skill at
  times, but…
      Applications of Probabilistic Flow Forecasts
…little of that skill
comes from
traditional climate
forecasts, currently

(JFM Precip correlation
with Nino 3.4)
                    Main Motivations
 We’re not always confident about our forecast confidence
    forecast method deficiencies
    ad hoc framework

 We may not be capturing all potential prediction skill
   (rationale for this workshop)

 Some Users want “more”
   (not naming any names)
             Areas of Progress / Topics
 Hydrologic Ensemble Forecast System
    Leverages CFS

 Objective combination of multi-agency forecasts

 Tighter integration with USBR on reservoir regulation
                    An ESP Upgrade: The NWS Hydrologic
                      Ensemble Forecast Service (HEFS)
                                                                                   Verification products
                      Pre-                                   Verification
                   processor                                   system
Weather                            Input Ensembles

                                     Ensemble                               Post
                                     Streamflow      “Raw”                  processed
                                     Prediction                                             Product
 Input flow data                       System                                              generation
    Streamflow                          Data
                     Operational     Assimilator
                      Forecast                                     Post
                       System                                   processor

                                      reforecaster                                         Ensemble
             Uncertainty integration in HEFS
                Uncertainty in hydrologic forecast
Uncertainty in future forcings + Uncertainty in everything else
precipitation,                        initial soil moisture conditions,
temperature,                                             model errors,
potential evaporation,                              observation errors,
etc.                                                               etc.

•   Note, progress in DA may carve off an uncertainty piece from the
    latter category…but for now…

          Uncertainty integration in HEFS
             Uncertainty in hydrologic forecast
Uncertainty in future forcings + Uncertainty in everything else

Atmospheric Pre-Processor:        Hydrologic Post-Processor:
  retain skill contained in         lump all hydrologic
  single-valued input               uncertainties into one
  forecasts and generate            and model it
  unbiased ensembles                stochastically
 based on modeling joint          based on techniques TBD,
  probability distribution          but including basic
  between observations and          regressive approaches
  forecasts and Schaake Shuffle
  for space-time variability
      Strategy depends on Re-forecasts
• A hindcast = a numerical prediction for a date in the
  past using the model and data assimilation system
  that is currently operational.
• Uses:
   – (1) Post-processing of ensemble weather and climate
     forecasts; correcting for systematic bias, spread
     deficiencies, downscaling. Crucial for implementing
     reliable uncertainty forecasts in NWS.
   – (2) Data assimilation: first-guess forecasts corrected by
     observations; forecasts assumed unbiased; reforecasts can
     help adjust to make sure they are.
   – (3) Diagnosing model errors: sometimes model
     deficiencies aren’t obvious without large sample.

                                            Credit: Tom Hamill
   NOAA’s 1st-generation reforecast
               data set
• Model: T62L28 NCEP GFS, circa 1998

• Initial States: NCEP-NCAR Reanalysis II plus 7 +/- bred modes.

• Duration: 15 days runs every day at 00Z from 19781101 to now.

• Data: Selected fields (winds, hgt, temp on 5 press levels, precip, t2m,
  u10m, v10m, pwat, prmsl, rh700, heating). NCEP/NCAR reanalysis
  verifying fields included (Web form to download at Data saved on 2.5-degree grid.

• Experimental precipitation forecast products: .
                                                     Credit: Tom Hamill
 Comparison against NCEP / CPC forecasts
at 155 stations, 100 days in winter 2001-2002


               Reforecast calibrated
              Week-2 forecasts more
              skillful than operational
               NCEP/CPC 6-10 day,
               which was based on
             human blending of NCEP,
               ECMWF, other tools.


                     Credit: Tom Hamill
     For a new Climate Forecast System (CFS) implementation
                           Two essential components:

 A new Reanalysis of the atmosphere, ocean, seaice and land over
  the 31-year period (1979-2009) is required to provide consistent
                       initial conditions for:
  A complete Reforecast of the new CFS over the 28-year period
     (1982-2009), in order to provide stable calibration and skill
        estimates of the new system, for operational seasonal
                         prediction at NCEP

Like GFS reforecast, we leverage calibration capability from CFS reforecast
CFS Seasonal Forecast Skill - Precipitation
CFS Seasonal Forecast Skill - Precipitation
CFS Seasonal Forecast Skill - Tmin
         Atmospheric Pre-Processor: calibration
Off line, model joint distribution between single-valued forecast and
   verifying observation for each lead time
Archive of observed-forecast pairs
                                   PDF of Observed                        PDF of Obs. STD Normal
       Joint distribution
                  Sample Space
              Y                                                    NQT

                                                                         Joint distribution
                                          Correlation (X,Y)                         Model Space

                  PDF of Forecast         PDF of Fcst STD Normal

                                        NQT                                                       X

NQT: Normal Quantile Transform                         Schaake et al. (2007), Wu et al. (2011)        17
                     Atmospheric Pre-Processor:
                      ensemble generation (1)
In real-time, given single-valued forecast, generate ensemble traces
         from the conditional distribution for each lead time
                                       Conditional distribution
            Joint distribution                given xfcst
             Y   Model Space


                                                                  …      NQT-1      for that
                      xfcst                                       xn               particular
                                                                                   time step
                                             x1 xi           xn
                                                 Ensemble forecast

  Given single-valued forecast,
 obtain conditional distribution
                                                     Schaake et al. (2007), Wu et al. (2011)    18
              Atmospheric Pre-Processor:
               ensemble generation (2)
In real-time, string together lead-time specific ensemble values
               across lead times to generate traces
                  Schaake Shuffle (Clark et al. 2004)

X1 (1992)     X1(1991)      X1 (1993)       X1 (1991)     Member 1 (1991)

X2 (1991)     X2 (1995)     X2 (1994)       X2 (1995)     Member 5 (1995)

X3 (1993)     X3 (1994)     X3 (1995)       X3 (1994)     Member 4 (1994)

X4 (1994)     X4 (1992)     X4 (1991)       X4 (1992)     Member 2 (1992)

X5 (1995)     X5 (1993)     X5 (1992)       X5 (1993)     Member 3 (1993)

Time step 1   Time step 2   Time step 3     Time step 4

       Obtain forecast ensembles that behave similarly to
         historical ensembles in space-time variability
           Atmospheric Ensemble Processor:
                 calibration strategy
Known: CFS-scale
forecasts have substantial                                       East R. at Almont, CO
                                                                 Nov 29 2010 Forecast
skill variation and bias
relative to finer time-space
scale climatologies
Downscaling/calibration                                            CFS precipitation
is an essential step for               1           2       3       4       5        6
follow-on applications                                     Lead Months

                                                                    GFS precipitation
Precipitation forecast skill for
forecast events w/ variable
period lengths

                                   1       2   3   4   5   6 7     8 9   10 11 12 13 14
                                                            Lead days                    20
                Current Operations (2010)

  ESP run four different ways to support water supply and
    peak flow forecasts:

ESP 1                   Climatology temperature (T) and precipitation (P)
              Single Value T   Single Value
ESP 2         and P forecast   T forecast;                       Climo T and P
                               Climo P

XEFS 3        Ensemble T and P forecasts from GFS        Climo T and P

XEFS 4        Ensemble T and P forecasts from GFS        Ensemble T and P forecasts from CFS

         Forecast           5            10           14
         Issuance        Days out      Days out     Days out
                           Time into the future
Example: CFS climate forecasts, late 2010

Examples of Experimental Ensembles
                 •   CFS-based ensemble forecasts for
                     Apr-Jul 2011 for upper Colorado
                     river basins issued in Dec. 2010
                     show deficits compared to
                     climatology-based forecasts

                 •   Working on verification, diagnosis
                     of WY2011 results during
                     experimental implementation

                 Average contribution to Lake Powell
                 Apr-Jul inflow:
                      Green River             34%
                      Colorado River 50%
                      San Juan River          13%
         Example of Experimental Ensembles
                                          Flow into Lake Powell
GFS and/or CFS based
ensembles: CBRFC & CNRFC
experimental products updated daily

                                                 GFS              CFS

Contact: Andy Wood (
 Hydrologic Ensemble Forecast System:
        planned enhancements
                        Weather and Climate        GEFS, CFSv2

                                                       Ensemble Verification System
Hydrologic              Atmospheric Ensemble
Uncertainty                  Processor
  Land Data            Hydrologic, Hydraulic,
  Assimilator         Water Resources Models

  Observations          Hydrologic Ensemble
  (forcing, flow)            Processor

                    Hydrology & Water Resources
                         Product Generator

                       Water Products & Services
                                Atmospheric Ensemble Processor:
                                      calibration strategy
                         2 in                                             Archive of observed-forecast pairs

                                            Precipitation forecast (mm)
Precipitation forecast

                         1 in

                         0 in

                                                                            Observed precipitation (mm)

                  Bias-corrected ensemble mean
                  Spread representing skill
                                                                           Precipitation ensemble forecast     26
          Atmospheric Ensemble Processor:
           current & new forecast sources
          HPC/RFC         Day 1-5
          single-valued   ensembles

Medium- GFS/GEFS          Day 1-14
Range   ens. mean         ensembles              Calibrated
Long-     CFS/CFSv2       Day 15~                and
Range     forecast        ensembles   Merging    seamless
                                      Joining    short- to
        Other ensembles               Blending
        (SREF, NAEFS…)                           forcing
Under development                                input
   Other info             Day 15~
   (climate indices…)     ensembles
    Atmospheric Ensemble Processor:
         CFSv2 reforecast strategy
                         Reforecast Configuration for CFSv2 (T126L64)
•   9-month hindcasts initiated from every 5th day and run from all 4 cycles of that day, beginning from Jan
    1 of each year, over a 29 year period from 1982-2010. This is required to calibrate the operational CPC
    longer-term seasonal predictions (ENSO, etc)
•   In addition, a single 1 season (123-day) hindcast run, initiated from every 0 UTC cycle between these
    five days, over the 12 year period from 1999-2010. This is required to calibrate the operational CPC first
    season predictions for hydrological forecasts (precip, evaporation, runoff, streamflow, etc)
•   In addition, three 45-day (1-month) hindcast runs from every 6, 12 and 18 UTC cycles, over the 12-year
    period from 1999-2010. This is required for the operational CPC week3-week6 predictions of tropical
    circulations (MJO, PNA, etc)    Insufficient record for hydrologic users?

Jan 1            Jan 2              Jan 3            Jan 4              Jan 5             Jan 6
0 6 12 18        0 6 12 18          0 6 12 18        0 6 12 18          0 6 12 18         0 6 12 18
    USEFUL                                                                                    USEFUL
                                            LESS USEFUL

                   9 month run                          1 season run                   45 day run
          Atmospheric Ensemble Processor:
         coordination w/ NCEP and partners
•   CFSv2 User Needs Assessment Workshop on Mar.8, 2011

•   NCEP to continue CFS runs at 0 UTC to support current
    experimental hydrologic ensemble forecasts: after June 2011?

•   NCEP and partners to provide access to CFSv2 reforecasts
      Priority: get CFSv2 reforecasts out to 10/11 months (6-hr/daily
       time steps)
      In future: expand reforecast period for 45-day and seasonal
       CFSv2 runs to cover all 29 years?

•   OHD to upgrade Atmospheric Ensemble Processor prototype to
    use CFSv2 and implement new prototype at pilot RFCs for testing
      CFSv2 data processing coordinated with Princeton and CPC
      Initial prototype ready for testing: planned for Fall 2011
 Hydrologic Ensemble Forecast System:
        planned enhancements
                        Weather and Climate        GEFS, CFSv2

                                                       Ensemble Verification System
Hydrologic              Atmospheric Ensemble
Uncertainty                  Processor
  Land Data            Hydrologic, Hydraulic,
  Assimilator         Water Resources Models

  Observations          Hydrologic Ensemble
  (forcing, flow)            Processor

                    Hydrology & Water Resources
                         Product Generator

                       Water Products & Services
                                          Uncertainty integration in HEFS:
                                              hydrologic uncertainty
Precipitation ensemble forecast

                                  2 in
                                                                                                                                                             Archive of obs-simulated pairs

                                                Hydrologic, hydraulic models

                                                                               Raw flow ensemble forecast

                                                                                                                                Simulated streamflow (cms)
                                  1 in                                                                                Skill?

                                  0 in

                                                                                                                                                               Observed streamflow (cms)

                                  Uncertainty-integrated                                                    Repeat for all individual                                Post-processed flow
                                  flow ensemble forecast                                                      raw flow ensemble                                       ensemble member
                                                                                                                  members                                                                  31
Strategy for hydrologic uncertainty modeling
•   Current approach for the short range
      Hydrologic Post-Processor: lump all hydrologic uncertainties into
       one and model it stochastically (combining probability matching
       and recursive linear regression; Seo et al. 2006)
     Hydrologic Model Output Statistics: model total uncertainty
       (input + hydrologic) in single-valued operational flow forecasts
       using QPF information for first few days (combining Model Output
       Statistics and Adjust-Q; Regonda et al, in preparation)

•   Future approach
     Include other statistical post-processing techniques for longer range
      and for regulated flows (HEPEX post-processing testbed and workshop
      in June 2011)
     Improvement with ensemble data assimilation, parametric uncertainty
      processor and multi-model ensembles                               32
Uncertainty integration in HEFS:
       data assimilation

Assimilation window                       Observations

                                          Model trajectory
                                          w/ DA

                                          Model trajectory
                                          w/o DA

  Immediate      Present         Future
  past           forecast time

      Improvements of initial conditions
              Strategy for data assimilation
•   Current approach:
      develop data assimilation (DA) tools with single-valued forecasts
       and extend to ensemble DA

•   Techniques currently tested
     Hydrologic routing DA to assimilate observed streamflow into 3-
      parameter Muskingum routing model (under testing for future HEFS)
     Streamflow DA to assimilate streamflow, precipitation and PE for
      Sacramento and Unit Hydrograph models (used for deterministic
     Snow and streamflow DA (including different techniques, e.g.,
      Ensemble Kalman Filter and Maximum Likelihood Ensemble Filter)
     DA for distributed modeling to assimilate streamflow and soil moisture
      for gridded Sacramento model and kinematic-wave routing in
      Research Distributed Hydrologic Model (RDHM)
              Ensemble Verification System
•   Current software
      Verifies numerical time-series for individual points/areas
      Supports flexible conditional verification (e.g., > flood threshold)
      Computes several key metrics (deterministic and ensemble) to
       describe forecast reliability, resolution, discrimination, and skill
      Java tool with structured GUI
      Fully documented and freely available (

•   On-going
      Enhancements (e.g., estimation of confidence intervals)
      New metrics and products to communicate meaningful
       information to forecasters and end users
      Training

EVS interface, graphics
and free download                                        36
      Hydrologic Ensemble Forecast System:
             product dissemination
• Need for reliable probabilistic
    unbiased
                                     Flood Stage

                                                        Least Likely

    accurate spread
                                                        Most Likely

                                                   _   Median Fcst

                                                   ?    Observed

• Need for wide range of products:
    probabilistic products and
     ensemble traces
    forcing inputs and hydrologic
    real-time forecasts and
    meta data and verification
     information                                                       37
Current forecast framework

Target, multi-forecast framework

              What the framework means
 The water supply forecast approach may become ‘messier’,
  but also, we hope:
    more transparent
    better informed by climate forecasting

 Important to note that a single ‘better’ forecast must fit into a
  complex institutional framework and process

Experimental Ensemble Prototypes:
CBRFC Experimental Ensemble Products:

Papers on Atmospheric Ensemble Processor:
Wu et al, 2011. Generation of ensemble precipitation forecast from single-valued
quantitative precipitation forecast for hydrologic ensemble prediction, JoH.
Schaake et al, 2006. Precipitation and temperature short-term ensemble forecasts
from existing operational single-value forecasts, HESSD.
Seo et al, 2006. A statistical post-processor for accounting of hydrologic uncertainty in
short-range ensemble streamflow prediction, HESSD.
Brown et al, 2010. The Ensemble Verification System (EVS): a software tool for
verifying ensemble forecasts of hydrometeorological and hydrologic variables at
discrete locations, EMS.
Extra Slides
      Example Correlation Plot for
     North Fork American River, CA
                                                                      Forecast Period

                                                                      Period     Days
                                                                         1         1
                                                                         2        1-2
                                                                         3        1-3
                                                                         4        1-4
                                                                         5        1-5
                                                                         6        1-7
                                                                         7       1-10
                                                                         8       1-14

This plot shows how the coefficient of correlation between GFS forecast precipitation and
observed precipitation varies during the year, depending on the event being forecast. This
plot was constructed for each of the 24 selected MOPEX basins. Separate correlation
plots were made for precipitation, tmin and tmaax.
            Selected MOPEX Basins
USGS ID       Long       Lat     SqMi        Station Name
01076500   -71.6861    43.7592    622   PEMIGEWASSET RIVER AT PLYMOUTH, NH
01567000   -77.1294    40.4783   3354   JUNIATA RIVER AT NEWPORT, PA.
02135000   -79.2472    34.0569   2790   LITTLE PEE DEE R. AT GALIVANTS FERRY, S.C.
02236000   -81.3828    29.0081   3070   ST. JOHNS RIVER NR DELAND, FLA.
02349500   -84.0439    32.2981   2900   FLINT RIVER AT MONTEZUMA, GA.
02486000   -91.0267    30.4042   3171   PEARL RIVER AT JACKSON, MS
03251500   -84.2667    38.5978   2326   LICKING RIVER AT MCKINNEYSBURG, KY.
05520500   -87.6686    41.1600   2294   KANKAKEE RIVER AT MOMENCE, IL
06192500   -110.5653   45.5972   3551   YELLOWSTONE RIVER NEAR LIVINGSTON, MT.
06340500   -101.6217   47.2853   2240   KNIFE RIVER AT HAZEN, ND
06600500   -96.3119    42.5767    886   Floyd River at James, IA
06847000   -99.8931    40.1200   1650   BEAVER CREEK NEAR BEAVER CITY, NEBR.
07068000   -90.8475    36.6219   2038   CURRENT RIVER AT DONIPHAN,MO.
07196500   -94.9233    35.9228    959   ILLINOIS RIVER NEAR TAHLEQUAH, OK
08055500   -96.9442    32.9658   2459   ELM FORK TRINITY RIVER NR CARROLLTON, TX
09132500   -107.4336   38.9258    526   NORTH FORK GUNNISON RIVER NEAR SOMERSET, CO.
09292500   -110.3408   40.5119    132   YELLOWSTONE RIVER NEAR ALTONAH, UTAH
09497500   -110.4992   33.7981   2849   SALT RIVER NEAR CHRYSOTILE, ARIZ. MILE 34.8
11138500   -120.1672   34.8397    281   SISQUOC RIVER NEAR SISQUOC, CALIF.
11427000   -121.0228   38.9361    342   NF AMERICAN R A NORTH FORK DAM CA
11532500   -124.0750   41.7917    613   SMITH R NR CRESCENT CITY CA
12413500   -116.3069   47.5639   1220   COEUR D'ALENE RIVER NR CATALDO, IDAHO
12449500   -120.1150   48.3653   1301   METHOW RIVER AT TWISP, WA
14321000   -123.5542   43.5861   3683   UMPQUA RIVER NEAR ELKTON, OREG.
Skill of GFS Cumulative Pecipitation Forecasts
Skill of GFS Average Tmin Forecasts
Skill of GFS average Tmax Forecasts
    Atmospheric Ensemble Processor: strategy
•   Current situation
     NWP outputs need to be calibrated for RFC hydrologic forecasting
     NWP ensembles are generally biased in the mean and spread
     Additional skill for short range in HPC/RFC operational single-valued

•   Current prototype
     Uses single-valued forecasts from multiple models to cover short- to
      long-range (HPC/RFC, GFS/GEFS, CFS/CFSv2)
     Generates conditional ensemble values given real-time forecasts
      based on modeling forecast uncertainty/skill from an archive of
      forecast-observed pairs
     Reconstructs space-time variability of all forcing input ensembles
      based on historical climatological values (Schaake Shuffle)
     Uses multiple temporal scales to capture/preserve forecast skill at
      longer scales (e.g. several days/weeks)
      Requirements for Hydrologic Services

•   Generate reforecasts of calibrated forcing input ensembles and
    hydrologic ensembles for several decades to support
      forecast calibration and verification
      user decision making, including calibration of downstream
       applications and DSS

•   OHD and RFCs need to
      select forcing input forecasts with long enough archive of
       reforecasts to support DSS
      coordinate with NCEP and partners (NCDC, NCAR…) for new
       model implementation and reforecast availability

    Example of verification results: HEFS flow ensembles
        compared to climatological ESP ensembles

Skill Score for Mean Continuous
  Rank Probability Score (CRPSS):             Higher scores: better
  GFS-based flow ens. generated w/ pre-
  and post-processing compared to
                                                   gain from Atm Pre-Proc.
    GFS-based flow ens. w/o post-                 using GFS ensemble
     processing                                    means
                                                        gain from Hydro
    climatology-based flow ens.                               Post-Proc.
     (operational ESP) (w/o pre- and

    Very large improvement w/ pre-
   processing and GFS ens. means over
   climatological ESP                     North Fork of American River, CA
    Significant improvement w/ post-     Ensembles from Jan. 1979
   processing                             to Sep. 2005
                   Product Generator

• Current prototype
    Basic functionality to generate operational AHPS products
    Enhanced functionality based on RFC and users feedback

• On-going
    On-demand product sharing across operating clients
    Additional default template products
    Coordinating product generation and dissemination with AHPS
    Training

                     Product Generator
                                           Editor Tool Bar

Interactive Viewer Panel

                Parameters Editing Panel

                                              Chart Display Panel


To top