Ensemble Data

Document Sample
Ensemble Data Powered By Docstoc
					                                                              1




Recent Developments with the NCEP GEFS


                           Yuejian Zhu,
     Zoltan Toth, Richard Wobus, Qingfu Liu and Mozheng Wei

                 Environmental Modeling Center
                      NOAA/NWS/NCEP

                      Acknowledgements:
    Bo Cui, DingChen Hou, Mark Iredell and Stephen Lord EMC
         David Michaud, Brent Gordon and Luke Lin NCO
                 GEFS Changes – (Aug. 2005)                               2


 Extend T126 portion of forecast after 180 hours
   •   This change is intended to improve ensemble support for 5-10
       days and week-2 forecast by providing high resolution (T126)
       and continue (no resolution change) forecast
   •   Results:
       •   Increased spread for week-2 forecast
       •   Improving probabilistic skill beyond 180 hours
 Initial perturbations (breeding cycle), 6-hour instead of 24-
  hour cycle (see next slide):
   •   This change is intended to enable for relocation of perturbed
       tropical storm. Tuning initial perturbation size is for reducing
       spread for short-range forecast
   •   Results:
       •   Decreased spread for short-rang (1-3) forecast
       •   Improving forecast skill for first 3 days
       •   Improving probabilistic forecast skill for short lead-time
             Current breeding cycle                                              New breeding cycle               3

                           24 hours                                                       6 hours
                                                                    Re-scaling
              24hrs                              Up to 16-d          6hrs             Next T00Z     Up to 16-d
T00Z                                                          T00Z
                       Re-scaling
10m                                                           40m




                   24hrs                         Up to 16-d                 Re-scaling
      T06Z                                                          T06Z                             Up to 16-d
                             Re-scaling
      10m                                                           40m




                           24hrs                 Up to 16-d                     Re-scaling
            T12Z                                                          T12Z
                                    Re-scaling
            10m                                                                                      Up to 16-d
                                                                          40m




                                   24hrs         Up to 16-d                            Re-scaling
               T18Z                                                             T18Z
                                           Re-scaling
               10m
                                                                                40m
                                                                                                     Up to 16-d
                                                                    4

        GEFS Changes – (Aug. 2005, Cont.)

 Relocation of Perturbed tropical storm
  – Add 5% of TS to initial perturbation
  – This change is intended to reduce track forecast error
    and uncertainty for short lead-time (1-3 days)
  – Results:
     • Reducing mean track errors by 10% for 12-48 hours
     • Reducing the ensemble track spread (uncertainty) for short
       lead-time
     • Improving track forecast skills
                                                                                             5



        GFS TS relocation                  Ensemble TS relocation
                                                             6hrs fcst
Fcst/guess    3hrs    6hrs       9hrs
                                              P                                   N
             Use GFS Track information                           C
                                                                           Use ens. Track
                                         Use ens. Track
                                         information                       information

               Relocated TS to                        Use GFS Track information


              Observed position
                                            To separate into env. Flow (EF)
                                            And storm perturbation (SP)

                     GDAS
                     (SANL)              Ens. Rescaling      Combined       Ens. Rescaling
                                          For EF (p+n)                       For SP (p+n)



                     FCST                                      FCST
                                                                       6


   Hurricane Track Plots (case 1)
                 Frances (08/28)


                                              With relocation



Without relocation




                                                  Reduced initial spread
                           Large initial spread
                                                           7



Hurricane Tracks Plots (case 2)
                          Ivan (09/14)




     Without relocation


                                         With relocation
                                                                   8
      Track errors and spreads
               2004 Atlantic Basin (8/23-10/1)

       opr-errors         exp-errors     opr-spread   exp-spread
400
      From Timothy Marchok (GFDL)
350
300
250
200
150
100
                                         Reduced mean track
50
                                         errors and spreads
 0
      24h           48h            72h         96h       120h
                                                                            9

                     Hurricane track errors
                            2 basins (Atlantic and e-Pacific)

          GFS        OPR    REL                            GFS   REL
350                                           15
300
                                              10
250
200                                            5
150                                            0
100
                                              -5
 50
  0                                          -10
      0         24     48      96                  0       24    48    96
                                       Percentage improvement to
          Track errors (miles)         operational ensemble
                Period: 20040824-20040930 (53-103 cases)
                                                                                            10
                        GEFS Changes (May 2006)
1.   Increase the number of perturbed ensemble members
     •   14 (in place of current 10) perturbed runs for each cycle (20 by early 2007)
         • NAEFS requirement
     •   This change is intended to improve ensemble based prob. forecasts
         • Results: improved probabilistic skill, slightly improved ensemble mean skill
             (seasonally dependent)
2.   Add control runs for 06, 12 and 18Z cycles
     •   This change is intended to enable for relocation of perturbed tropical storm
     •   Facilitates comparison of high & lower resolution ensemble controls
         • If lores control and ensemble mean differ – indication of nonlinearities
         • If high & lores controls differ – indication for possible effect of resolution
3.   Introduce Ensemble Transform (ET) into GEFS breeding method
     •   ET breeding method creates globally orthogonal initial perturbations
     •   Uses simplex method to create individual (not paired) perturbations
     •   This change is intended to improve probabilistic forecast skill
         • Results: Improved probabilistic forecast skill; Slightly reduced ensemble mean
             hurricane track errors for 12-96 hours
                                                                      11


                    GEFS configurations
                              Past                 Current
      Model                   GFS                    GFS
Initial uncertainty            BV                    ETBV
Model uncertainty            None                    None
  Tropical storm        Relocation(2005)             same
 Daily frequency       00,06,12 and 18UTC            same
  Hi-re control        T382L64 (d0-d7.5)             same
     (GFS)             T190L64 (d7.5-d16)
  Low-re control        T126L28 (d0-d16)      T126L28 (d0-d16)
(ensemble control)        00UTC only         00,06,12 and 18UTC
Perturbed members       10 for each cycle    14 (20) for each cycle
  Forecast length      16 days (384 hours)           same
  Implementation        August 17th 2005        May 30th 2006
                                                                                                    12
              Bred Vector                       Ensemble Transform Bred Vector
               (Current)                                    (New)
                            Rescaling                       P1 forecast               Rescaling
                                                   P2 forecast


         P1
ANL                                                 ANL
         N1

                                                  P3 forecast
           t=t0      t=t1        t=t2                            P4 forecast
                                                                t=t0           t=t1          t=t2

P#, N# are the pairs of positive and negative    P1, P2, P3, P4 are orthogonal vectors
P1 and P2 are independent vectors                No pairs any more
Simple scaling down (no direction change)        To centralize all perturbed vectors (sum of all
                                                 vectors are equal to zero)
                                                 Scaling down by applying mask,
         P2
ANL                                              The direction of vectors will be tuned by ET.
         N2
   Summary of Retrospective and Parallel Runs                                  13


• Period:
   – 08/20/2005 – 09/30/2005 (retrospective runs)
   – 03/01/2006 – 04/26/2006 (NCO real time parallel)
• Statistics for
   – Hurricane track errors (retrospective runs only)
       • Atlantic-, East Pacific-, West Pacific- basins, total basins
   – RMS errors and AC scores for ensemble mean
       • NH and SH ex-tropic
   – Probabilistic verifications (ROC)
       • NH and SH ex-tropic
• Conclusions
   – Tropical – mean of track error (slightly improved)
       • Improved (48-, 72-, 96-hours over all)
   – Northern hemisphere
       • Mean -improved from retrospective runs;-similar from NCO real time
       • Probabilistic (improved)
   – Southern hemisphere
       • Mean –similar from retrospective runs; -improved from NCO real time
       • Probabilistic (improved)
                                                                                                                   14
                   Hurricane Track Errors (Period: 08/20-09/30/2005)
                        ENSs    ENSx   GFSs                                      ENSs    ENSx   GFSs
   350                                                      350
                        Atlantic Basin                                     East Pacific Basin
   300                                                      300
   250 ENSs-operational ensemble                            250
   200 ENSx-retrospective runs                              200
       GFSs-operational GFS
   150                                                      150
   100                                                      100
    50                                                        50
        0                                                        0
Hours        12    24      36    48    72     96   120                12    24      36     48    72    96    120
Cases       174   157     141    128   101    75   48    Cases       181   165     149    135   109    85     69
                        ENSs    ENSx   GFSs                                      ENSs    ENSx   GFSs
   250                                                     300
                  West Pacific Basin                                         All Basins
   200                                                     250

   150                                                     200
                                                           150
   100
                                                           100
    50
                                                            50
        0
             12    24      36    48    72     96   120        0
                                                                      12    24      36    48    72     96    120
Cases       177   161     145    129   101    66   44 Cases          532   483     435    392   311    226   161
                                    Ensemble mean evaluations                                      15

                  Improving skills from/after day 3
                       65% AC scores – useful skill        ENS_s – opr. ensemble
                       Ens. extended add. 20 hours         ENS_x – real time parallel


                                            Northern Hemisphere
                                                                               RMSs are very similar
       ENS_s – opr. ensemble
       ENS_x – retrospective runs

               Much better than GFS after 72 hours




                                                                      Right – Real Time Parallel
Left – Retrospective Runs


                                           Southern Hemisphere
                          RMSs are very similar
                                                                  Improved
                  Probabilistic Evaluation (ROC)                                     16




           ENS_s – operational ensemble
           ENS_x – retrospective runs
                                                                          Improved
                              Northern Hemisphere
Improved                                     ENS_s – operational ensemble
                                             ENS_x – real time parallel runs

Left: Retrospective Runs                               Right: Real Time Parallel




                                                                        Improved
                              Southern Hemisphere

Improved
     First Implementation of NAEFS – Summary                                            17

1. Bias corrected members of joint MSC-NCEP ensemble
      •   Decaying accumulated bias (~past 50 days) for each var. for each grid point
      •   For selected 35 of 50 NAEFS variables
      •   32(00Z), 15(06Z), 32(12Z) and 15(18Z) joint ensemble members
      •   Bias correction against each center’s own operational analysis
2. Weights for each member for creating joint ensemble
   (equal weights now – unequal weights to be added later)
      •   Weights don’t depend on the variables
      •   Weights depend on geographical location (low precision packing)
      •   Weights depend on the lead time
3. Climate anomaly percentiles for each member
      •   Based on NCEP/NCAR 40-year reanalysis
          •   Used first 4 Fourier modes for daily mean,
          •   Estimated climate pdf distribution (standard deviation) from daily mean
      •   For selected 19 of 50 NAEFS variables
      •   32(00Z), 15(06Z), 32(12Z) and 15(18Z) joint ensemble members
      •   Adjustment made to account for difference between oper. & re-analysis
      •   Provides basis for downscaling if local climatology available
          –   Non-dimensional unit
                                                                                             18
             Bias Correction Method & Application
 Bias Assessment: adaptive (Kalman Filter type) algorithm
   decaying averaging mean error = (1-w) * prior t.m.e + w * (f – a)

   For separated cycles, each lead time and individual grid point, t.m.e = time mean error



                                          6.6%

                                                        • Test different decaying weights.
                                                          0.25%, 0.5%, 1%, 2%, 5% and
                                                          10%, respectively
                                          3.3%

                                                        • Decide to use 2% (~ 50 days)
                                                 1.6%
                                                          decaying accumulation bias
                                                          estimation

             Toth, Z., and Y. Zhu, 2001

 Bias Correction: application to NCEP operational ensemble 15 members
                                                    19
   List of Variables for Bias Correction, Weights
and Forecast Anomalies for CMC & NCEP Ensemble
     Summary of NAEFS First Implementation                   20


• Period:
   – 04/10/2006 – Current (NCO real time parallel)
• Maps comparison for bias (before and after)
   – 500hPa height, 2m temperature
• Statistics for
   – Bias reduction in percentage
       • Height, temperature, winds
   – RMS errors
   – Probabilistic verifications (ROC)
       • NH, SH and tropic
• Conclusions
   – Bias reduced (approximately 50% at early lead time)
   – RMS errors improved by 9% for d0-d3
   – Probabilistic forecast
       • Improved for all area, all lead time
       • Typically for NH, 20-24 hours improvement from d7
       500hPa height: 120 hours forecast (ini: 2006043000)            21

Shaded: left – raw bias               right – bias after correction
      2 meter temperature: 120 hours forecast (ini: 2006043000)        22

Shaded: left – raw bias                right – bias after correction
                                                                        23
 Bias Improvement (absolute value) after Bias correction


                     Overall bias
                     reduction:
500hPa height                                      850hPa temperature
                     (globally)
                     D0-3: 50%
                     D3-8: 40%
                     D8-15: 30%




                     There is daily
Sea level pressure   variation after              2m Temperature
                     bias correction,
                     more bias
                     reduced for
                     valid 12Z cycle
Bias Improvement (absolute value) after Bias correction             24




10m U-component                                   10m V-component
                      Overall bias
                      reduction:
                      (Tropic)
                      D0-3: 50%
                      D3-8: 45%
                      D8-15: 40%


 Sea level pressure




                                                 2m temperature
                   Evaluation after bias correction (16 cases)                25




             Probabilistic skill
             Extended 20-h for d-7




Northern Hemisphere                                     Southern Hemisphere
                             Black-operational ensemble (10m)
                             Red-real time parallel ensemble (14m)
                             Green-real time parallel ensemble after
                             bias correction (14m)

  RMS errors for ensemble mean
  reduced for 48-h forecast (~9%)                       Tropics
                            NAEFS Performance Review                                                         26

Appendix 6

KEY PERFORMANCE MEASURES

                                          Improvement in Ensemble Forecasts


                            Requirement                        Threshold       Actual        Variance
                                                                               25Apr-
                                                                              10May06

                                                                                          Met or exceeded
                                          Bias Reduction (%)     50%           30-70%      in Tropics & up
                                                                                          to D3 elsewhere;
         Ensemble Mean                                                                      slightly below
       3-14 Day Lead Time                                                                     otherwise
                                     RMS Error Reduction (%)
                                                                 10%          Up to 10%    Met up to D3,
                                                                                          below expected
                                                                                          D4 and beyond


                                                   3 Day        6 Hours        12 hrs
                                                                                             Exceeded


    Improvement in Ensemble-based
                                                   7 Day       12 Hours        16 hrs
         Probabilistic Forecasts                                                             Exceeded




                                               10 – 14 Days    24 Hours        48 hrs
                                                                                             Exceeded
         NAEFS Configuration Review (NCEP)                                                      27


Appendix 8
MINIMAL (PREFERRED) CONFIGURATION FOR THE GLOBAL ENSEMBLE FORECAST SYSTEMS OPERATIONAL AT CMC
AND NCEP




                 FEATURE                         2005 Plan   2008 Plan    May 2006 Actual /
                                                                         Feb    2007     Plan
                                                                         (NCEP)

                 Forecast lead time (days)       16          16 (35)     16



                 Number of cycles per day        2 (4)       4           4



                 Number       of      ensemble   10 (20)     20 (50)     14 / 20
                 members



                 Model resolution (km)           120 (90)    80 (60)     120 / ?



                 Number of vertical levels       28 (42)     42 (64)     28 / ?
    GEFS Major Implementation Plan (FY07)                             28


• Upgrade vertical resolution from 28 to 64 levels for 20 perturbed
  forecasts
   – 4 cycles per day
   – T126L64
   – Up to 384 hours (16 days)
• Real-time generation of hind-cast at T126/L64 resolution.
   –   4 cycles per day
   –   27 hind-casts for each cycle since 1979
   –   Using reanalysis II initial conditions (T62L28 resolution)
   –   Add random noise to high frequency (T63-T170) by using
         • Cycling (6-hr T170 model forecast)
         • Other method?
• (Alternate) – upgrade both horizontal and vertical resolution to
  T170/L64
• Introduce ESMF scheme that allows concurrent generation of all
  ensemble members.
• Add stochastic perturbation scheme to account for model errors
  (tentative plan)
               NAEFS upgrade plan (FY07)                                            29


• Add approximately 15 new variables to current 51 pgrba for NAEFS
  data exchange.
   – Such as vertical shear, helicity, u,v, t, RH for 100, 50hPa, LH, SWR, LWR at
     surface, and etc..
• Add GFS high resolution control bias correction by using current
  method for ensemble.
   – There is a problem when we estimate bias after GFS change resolution after
     180 hours
• Set up GFS low resolution (ensemble) control run on NCO’s real
  time parallel prior to GFS upgrade in the future.
   – As bias estimation of GFS major/minor implementation
   – Need to compare the bias of ensemble mean and control
• Improve bias correction algorithm.
   – Pending on hind-cast information
   – Two weights: one from real-time (analysis and forecast) bias estimation
     (mainly for week-1), another one from hind-cast (mainly for week-2)
                                                                                       30
            NAEFS Expansion and Future Plan
• Plans to be coordinated with THORPEX
    – Links with Phase-2 TIGGE archive and beyond (GIFS)
• Expansion
    – FNMOC
        • Experimental data exchange by Dec 2006
        • Preliminary evaluation by Dec 07
        • Operational implementation by Dec 08 (subject to improved performance)
    – UK Metoffice
        • Decision on going operational & possibly joining NAEFS - by 2008
    – KMA, CMA, JMA
        • Expressed interest, no detailed plans yet
• Data exchange with MSC
    – Replace current ftp with more reliable telecom by Dec 08
• Statistical post-processing
    – Continual enhancements to current methods (2nd moment correction, addtnl vars)
    – Testing (Dec 08) & possible implementation (09) of advanced methods
• Products
    – Week-2 – experimental by Nov 06
    – Web graphics
        • MSC – Nov 06
        • NCEP – Mar 07
                                                                       31
                                       THORPEX LINKS
                                   PRODUCT DEVELOPMENT

•   Goals:
     – Develop new numerical modeling applications
     – Develop new product generation tools and products

•   Participants / Contributions
     – Scott Jacobs et al. (NCO)
         • NAWIPS ensemble functionalities
     – Richard Verret et al. (Meteorological Service of Canada, MSC)
         • NAEFS web-based products
     – David Unger et al. (CPC) and Richard Verret et al. (MSC)
         • Week-2 NAEFS products
     – Bob Grumbine (EMC)
         • Sea ice ensemble application
     – Dingchen Hou (EMC)
         • River flow ensemble application
     – Steve Silberberg, Binbin Zhou (NCEP)
         • Aviation weather guidance
     – Yuejian Zhu (NCEP)
         • NAEFS coordination


•   Supported partially by NOAA THORPEX program
                   32




Background !!!!!
RPSS before/after bias correction                        33




                     RPSS performance for past 5 years
DGD FORECAST UNCERTAINTY ALTERNATIVES
• Current status (in NDFD):
   – Expected value (mean, median, or mode??) of distribution only
• Scenario 1 – Add 1 variable
   – Add spread to expected value (1 additional grid)
       • Workshop WG felt that was not enough info
       • Recommended adding 2 pieces of info
• Scenario 2 – Add 2 variables
   – Add info on spread on 2 sides of mean/median/mode
   – 10/90 or 20-80 percentile values
       • Preferred as opposed to variance (spread) info that is more abstract
   – NDFD Workshop recommendation
   NDGD FORECAST UNCERTAINTY QUESTIONS
• Use mean, mode, or median in NDGD?
   – Mean – Expected value
       • Can fall around minimum in pdf
       • Requires additional info (what percentile it corresponds with)
   – Mode – Most likely event
       • Appealing heuristically (well defined meaning)
       • Requires additional info (what percentile it corresponds with)
       • Use in future when multiple modes can be considered?
   – Median – 50 percentile
       •   Heuristic meaning (half below, half above)
       •   Consistent with 10/90 (or 20/80) percentile approach
       •   Verifies similarly to ensemble mean
       •   No need for additional info
       •   Used by HPC in PQPF context
• Use 10/90 OR 20/80 percentile?
   – 10/90 is more inclusive (covering explicitly 80% of forecast distribution)

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:3
posted:8/9/2011
language:
pages:33