Models and Modeling in FEWS Part 2 by wulinqing

VIEWS: 0 PAGES: 57

									Models and Modelling in FEWS
Part II

Micha Werner
Deltares & UNESCO-IHE
       Error correction
       ARMA & ADJUST-Q

In this section we will discuss two methods used for correcting the
outputs of a hydrological model. The method used widely in the
NWS is ADJUST-Q, which typically requires manual interaction
during the forecast run. The second is the ARMA method in FEWS.
This is a statistical error model that is widely used in forecasting.
This does not need interaction during the forecast process
Improving the Forecast




  A: Input correction
  B: State Updating (data assimilation)
  C: Parameter Updating
  D: Postprocessing (including Error Correction)



                                    3
  Output Processing


This can be done using very simple approaches as well as with more
   complex methods that canb also provide an estimate of uncertainty

Simple methods:
• Adjust Q (correction at start forecast)
• AR or ARMA type error correction

More “complex” methods:
• Quantile regression
• Bayesian Output Processor (HUP)




                                        4
 Overview of error correction models/methods


Available methods for error correction in FEWS

Internal
• AdjustQ type operation
• ARMA Error correction method

External (models) – run using the adapter approach
• MCRM/DODO Error Correction approach
• CEH ARMA Module
• PDM Error correction/State updating
• Implementations of Quantile Regression & HUP




                                     5
Overview of available error correction methods


ADJUST-Q: Empirical error correction
• Parameter steps determines convergence speed
• steps may be changed interactively during forecast




 Example: simple model with constant bias

                                            steps




                                       6
Overview of available error correction methods


Statistical model of error
• Time series modeling
• ARMA: Auto Regressive – Moving Average

Concept
• Error is typically highly correlated in time
• Establish model of error – predict future error
• Correct model simulation in forecast period with predicted error




Model Order
Model Parameters

                                     7
ARMA module Delft-FEWS - 1

Autoregressive Moving Average Models used for forecasting of stationary
   timeseries – in this case applied to modelling the time evolution of the
   model error

AR: This part of the model describes how each observation (error) is a
  function of the previous k observations (errors). For example, if k = 1,
  then each observation is a function of only one previous observation. That
  is,


   where Qres(t) represents the observed residual (error) value at time t,
   Qres(t−1) represents the previous observed residual (error) at time t − 1,
   e(t) represents some random error and c and a1 are constants. Other
   observed values of the series can be included in the right-hand side of the
   equation if k > 1:




                                          8
ARMA module Delft-FEWS - 2


MA: This part of the model describes how each observation is a
  function of the previous y errors. For example, if y = 1, then each
  observation is a function of only one previous error. That is,




   Here e(t) represents the random error at time t and e(t−1)
   represents the previous random error at time t − 1. Other errors
   can be included in the right-hand side of the equation if y > 1.




                                      9
ARMA Model




 Example of error correction using ARMA. Corrected time series (red)
 will converge to uncorrected time series (pink) as lead time
 increases
                                     10
ARMA Model


Simple example of ARMA model




See also spreadsheet…
                               11
AR module Delft-FEWS - 3

What is required for setting up an ARMA Model
• Simulated trace (typically SQIN)
• Observed trace (typically QIN)

Parameterisation of error model
- Model Order –
- Model parameters




Three ways of defining error model in FEWS
- Automatic:     Establish both order & parameters dynamically (AR only)
- Defined order: Order defined by user; Dynamic parameter identification
- Define all:    Order & parameters defined by user




                                       12
Establishing ARMA model order and parameters



               Window length




 Statistical behavior of error in window of defined length used to
 identify order and/or parameters of error model.
 Rule of thumb: Window should be > 50 x order of AR model

                                                   13
Establishing ARMA model order and parameters


  Length of window will influence the estimation of AR parameters.
  As window increases autocorrelation of errors will decrease for
  most hydrological time series


                       Window length




When estimating order of model: Define maximum order
Typical AR orders vary in range 1-3   14
Error Correction using FEWS ARMA model


FEWS ARMA Error model

Additional Features
• Error correction using AR and MA
• pre-processing methods to normalize errors (Log, Box-Cox etc)

•   Additional options
     • Interpolation of observed data to remove “small’ gaps
     • Data hierarchy for simulated inputs
     • Constraints on outputs
     • Constraints on inputs


                                     15
Error Correction using FEWS ARMA model

Options for ARMA model

      Free order & Free                   Fixed order & Free
      parameters                          parameters

      This allows the error model to      Order is now fixed – but parameters
      establish both order & parameters   dynamically - order
      dynamically                         established/calibrated offline



      Free order & Fixed                  Fixed order & Fixed
      parameters                          parameters

      not applicable                      Everything is now fixed –
                                          parameters & order
                                          established/calibrated offline




                                             16
Error Correction using FEWS ARMA model

Options for ARMA model – pro’s and con’s
       Free order & Free                     Fixed order & Free
       parameters                            parameters

       Pro: may utilize full potential       Pro: utilize potential of dynamic
       Con: statistical optimization with    orders
       many degrees of freedom –small        Con: very small risk of coming
       risk of coming unstuck                unstuck
       Con: behavior with strange data/bad   Con: behavior with strange data/bad
       model unpredictable                   model unpredictable
                                             Con: need to establish - order. 3 is
                                             good working max.
       Free order & Fixed                    Fixed order & Fixed
       parameters                            parameters

       not applicable                        Pro: controlled, predictable,
                                             behavior
                                             Con: need to establish order &
                                             parameters. Calibration required
                                                17
Error Correction using FEWS ARMA model


Notes on inputs to Error model
• 2 Traces are required
   • Simulated trace – shoud cover historical & forecast period
   • Observed trace – normally ends at T0
•   When there is missing data in simulated time series – failure L

•   Error correction module allows multiple simulated time series to be
    allocated
     • Simulated – Forecast
     • Simulated – Historical
     • Simulated – Backup (use in case problems with cold start!)

                                     18
Error Correction using FEWS ARMA model

Additional options – manipulating inputs

Range check on input can be defined (min/max)
   • This is like validation – values beyond range become Missing
   • Better to apply a more stingent validation berfore going in to error model
      (e.g rate change checks etc)

•   Interpolation of input data
     • Avoid spurious results due to small gaps
     • Same function as in InterpolationModule: Linear Interpolation for defined
        gap length

•   Ignore Doubtfull. Doubtfull data can be set to be ignored
     • Be very careful – as rated flows often doubtful beyond range of rating –
        but we do want these to be used



                                           19
Error Correction using FEWS ARMA model


Additional options – manipulating outputs

•   Range check on outputs can be defined (min/max)
     • This is NOT a validation – values are constrained to min-max
     • Typically used for constraining discharge values to zero (or a
       minimum flow, e.g. as input to HD model)




                                     20
Application of Error correction


General notes
• Error correction is a form of modeling!

•   Careful thought of the nature of errors being corrected

•   Calibration & validation
     • Calibration required if orders are fixed
     • Validation required in both cases !




                                       21
Application


Typical application of error model
• Rainfall-runoff model calculates flow to catchment outlet (C)
    • Error correction applied to flow at C
• Routing-model calculates propagation of flow in steep river
    • Uses error corrected flow as input
    • Error correction applied to flow at B
• HD model calculates levels & flows in
   reach from B to A
    • Uses error corrected flow as input        C
                                              B

                                     A                     Legend:
                                                                     Main River
                                                                     Small River
                                                                     Sub-Catchment


                                         22
Application

Error model cannot be applied to tidal signal as is!
• Periodic signal requires different approach

•   Approach 1: Correction of surge residuals
     • Possible – but…
     • Forecast surge may be very different from observed surge (bias)

•   Approach 2: Correction Frequency domain (Prosymfo2)
     • Significant training periods (several months data)
     • If to be considered – integrate as external module




                                          23
Setting up the ARMA Model in FEWS


Configuration when using automatic estimation methods is very easy
• Identify inputs and outputs
• If fixing order - set order of AR to e.g. 3 (typically maximum order)
• Typically MA can be ignored – as AR dominates

If fixing both order AND parameters: Recommended approach
• Set up models & ARMA in UpdateStates workflow
      • Configure ARMA to estimate parameters
• Run UpdateStates for extended period (e.g. 1 year)
• Run ARMA in DEBUG mode for 1 year of data (through e.g. cold
     state selection).




                                     24
Configuration


ARMA model run in DEBUG mode – allow parameters to be
  estimated
• Read AR (and MA values if relevant from DEBUG message
• Copy values as fixed




                                25
Comparison of ADJUSTQ to AR




                  1
                  12
 Blending steps = 100




                        26
Comparison of ADJUSTQ to AR




                  30
                  12
                  1
 Blending steps = 100




                        27
Calibrating and Validating ARMA models

Calibration of ARMA models using e.g. FEWS inernal routines, or other statistical
    packages
Validation
• Run series of hindcast runs
• Plots of lead time accuracy




        Fig. 3. (a) Lead time accuracy of the discharge forecast expressed as RMSE at the gauging stations of Cochem on the
        Mosel River, and Maxau on the River Rhine. Both the accuracy with and without error correction are shown. (b) Shows
        an example of the corrected and simulated flows at the gauge of Maxau in the Rhine for the forecast of 24th of
        December 2002
                                                                       28
Calibrating and Validating ARMA models


FEWS can be easily applied in setting up such hindcast runs




                                   29
ARMA versus ADJUST-Q


Pros;
• ARMA allows for an automated approach to adjusting errors –
   reduces need for interactivity
• ARMA makes statistical sense – errors typically have structure
• ARMA provides an objective method – can be verified using
   hindcasts
• ADJUST-Q supports changing interactively when not behaving
   properly
Cons;
• ARMA is a statistical model – not a hydrological model – statistical
   sanity is not always hydrologically correct
• ADJUST-Q is subjective – difficult to apply in verification



                                     30
Questions…
       Routing models in FEWS
       Hydrodynamic models

In this section we will discuss the application of routing models in
FEWS – focusing primarily on the use of hydrodynamic models such
as HEC-RAS. Some of the particular aspects of using HD models in
real time are discussed.
Routing models


Objective: Calculate propagation of flood wave through river system
• Simple Hydrological Routing (KW, Lag-K, Muskingum, …)
• Complex with 1-D hydrodynamic model (ISIS, Mike11, SOBEK,
  HEC)
• Potentially more complex – 2D models (Delft3D, Telemac, Flow2D
  etc)




                                   33
Routing models linked to FEWS (Examples)


Hydrological   LAG-K                 NWS                      US
               TATUM                 NWS                      US
               Kinematic Wave (KW)   CEH-Wallingford          England & Wales, Scotland
               2-Lyr Muskingum       Deltares                 -



Hydrodynami    SOBEK-1D              Deltares                 Rhine basin, Waterboards
c              ISIS                  HR Wallingford/Halcrow   England & Wales, Scotland
               Mike-11               DHI                      England & Wales, Italy, Spain
               HEC-RAS               USACE                    US, Italy, Sudan
               Delft3D               Deltares                 Scotland
               SOBEK-1D2D            Deltares                 Thailand




                                             34
Differences between model approaches


Kinematic Wave




Diffusive Wave




Dynamic Wave (all other cases) Full Equations

Most models are derivations of the shallow water equations – ignoring
different terms that are insignificant: Depends on the hydraulic situation

                                            35
Hydrodynamic vs. Hydrological Models


Typical set-up
                           Simple routing – often in hydrological model
                           e.g. UNIT-HG



         Hydrological routing
         e.g. LAG-K, Kinematic Wave


                                                  C
    Hydrodynamic routing
    e.g. HEC-RAS                              B

                                    A                      Legend:
                                                                     Main River
                                                                     Small River
                                                                     Sub-Catchment


                                        36
Hydrodynamic vs. Hydrological Models


Pros;
• Hydrodynamic routing provides more realistic simulation of flood
   wave propagation
• Deals well with backwater effects, change in flood wave
   propagation when flow goes out of bank
• Allows incorporation of structures and control of structures
• Allows outputs at intermediate locations (not gage à gage)

Cons;
• More complex models, data intensive
• Computationally more demanding
• Risk of instability



                                    37
Hydrodynamic vs. Hydrological Models


Apply HD models only when really required
• Extensive floodplains
• Reaches with structures
• Tidal Reaches
• Confluences



Mixing models
• Hydrological à Hydrodynamic
• Hydrodynamic à Hydrological




                                  38
HD model in a forecast workflow




                       39
Exchange between HD models & FEWS

•   All HD models integrated with FEWS using standard adapter approach
•   Inputs (typical)
      • Flows at upstream boundary and tributary inflows
      • Level at downstream boundary – may be a tidal boundary
         (not required when internal rating curve boundary is used)
•   Inputs (less common)
      • Gate settings
      • Temperature
•   Outputs (typical)
      • Water Level & Flow
         (point – or – longitudinal)




                                           40
Exchange between HD models & FEWS


•   Location of boundaries needs careful thought to avoid “reading” a
    defined boundary as the result of a HD model

                                        Reach influenced by d/s boundary condition

                                                                     Ignore results
                                                                     from this point

Upstream boundary – Q(t)



                                                Downstream boundary – Q-h

                       Flow direction




                                           41
Exchange between HD models & FEWS

Tidal boundaries offer a specific problems
• Astronomical constants to derive astronomical tide
• Difficult to work with harmonic constants
• Work with surge residuals (interpolate, ARMA modeling etc) – then add back to
    astronomical tide
• ADJUST-T (NWS operation addresses similar issue)
• Other option – link with coastal shelf model (see case study…)




                                           42
Hydrodynamic models & Error correction
 •   Hydrodynamic models typically cover long reaches of river, which means
     that intermediate gages are not utilized for error correction

 Options
 • State updating: e.g. Ensemble/Extended Kalman Filter; Particle Filter)
      • Particle filter applied in Rhine for updating
 • Simple “nudging” techniques
      • Available in Mike11 & ISIS
    à These are computationally intensive

 •   Splitting model in sections – use error correction at each gage
      • Assumes rating curve is reliable!




                                            43
Model cascades


Hydrodynamic – Hydrodynamic model cascade

Complex interaction
• State in d/s hydrodynamic model
  affects state in u/s hydrodyanic model
                                                                        I
• Overlap models                    Model 1                  III


                                                                   II

                      Model 2                               IV
                                           VI

                                   VII



                                Region of influence of d/s boundary




      Model 1                              Model 2
                                           44
Model cascades


Connecting two hydrodynamic models

Error correction on flow from u/s model
• Note that this does assume rating curve is reliable!! May not
   include hysterisis


       Gauge u/s of model transition
       Calculate error ε
                                            Add error to flow at d/s
                     Q                      boundary Model 1
                                            u/s boundary Model 2
                                            Read Levels from d/s model!!!
         Q+ε

      Model 1                          Model 2
                                       45
   Burn-in profiles

   Avoid “abrupt” shock on startup
   Mainly relevant to HD modules (stability)
   Only applied when starting from a cold state
      • Identify start value in cold state
      • Gradual “climb” to actual value




Burn-in section




                                       46
Inundation Mapping

Inundation maps provide spatial view of extent of inundation

Two main approaches in integrating these maps in FEWS

•   Running external (2D) hydrodynamic model – importing resulting grid data
    to view dynamic inundation profile
     • HEC-RAS (ID + Interpolation)
     • TUFlow
     • SOBEK-1D2D
•   Running a 1D hydrodynamic model
     • Export levels at cross sections to FEWS Flood Mapping Module
     • Interpolate water surface profile in GIS (PCRaster)
     • Import dynamic flood map to FEWS


                                        47
Inundation Mapping using a 2D model


•   Model runs through General
    Adapter – as does any model
•   Time series of grid data
    returned – map stack
•   Imported to FEWS database –
    displayed as any other grid




Example:
SOBEK 1D2D model of the Barotse Floodplain
Zambezi River, Zambia

DEM extent 303 x 541 cells; 720m resolution (resampled from 90m SRTM data)
SOBEK model using 1D for main stem rivers


                                                48
Inundation Mapping using a 1D model + Interpolation




                           49
                    Example: Modeling of bifurcation/Confluence
                    1D: Modeler decides division
                    2D: Division depends on water level




                          ?




Pannerdensche Kop
Forecasting using 1D & 2D HD models in the Firth of Clyde,
Scotland


            Low Pressure




     increased tide
        levels




                                51
  Forecasting using 1D & 2D HD models in the Firth of
  Clyde, Scotland

Firth of Clyde (FoC) Flood Forecasting
Model setup in Delft3D-FLOW
• Hydrodynamics module of Delft3D
framework, applied for the modelling of
surface water systems


FoC Model provides
• Tidal surge forecasts at locations
distributed in Firth of Forth
• Downstream boundary to 1D river models




                                           52
      Firth of Clyde model development


Model setup - computational grid
• Orthogonal curvilinear grid, aligned with local
geometric features
• Spatially varying resolution (1 km – 100 m)
• Run in 2D, 3D effects are secondary
• Based on a time step of 1 minute, a 1 day
simulation takes approximately 6 minutes


•Model does not run often (4x per day) when
forcings are updated. Provides d/s boundary for
river models

•Runs on dedicated server to avoid conflicting with
other resources




                                                      53
Firth of Clyde model development

Model setup - boundary forcing
• Tidal boundary conditions (harmonic constituents) for 50 tidal components
• External surge conditions by time-varying, spatially uniform water level elevation
• Meteorological forcing by time-varying, spatially uniform wind speed and direction
• Assuming one-way coupling at rivers (model provides d/s level boundary), no river
discharge taken into account




                                               54
Questions…
Wrap-up
Wrap up of models in FEWS


•   Variety of different types of models available for running in FEWS
     • All integrated using the same “adapter” concept
     • Models can be mixed in a single workflow – extremely useful
       for creating “integrated modeling structures”

•   Increasing use of distributed & physically based models in
    forecasting
     • Issues: speed, database sizes, complexity, …
•   Variety of models & adapters available and used operationally
     • Actual availability depends on model & supplier (licences)
     • Adapters to new models can be readily developed

                                      57

								
To top