GME – User′s Guide

Document Sample
GME – User′s Guide Powered By Docstoc
					                   -1-




HRM – User´s Guide
       For HRM Vrs. 2.8 and higher


      D. Majewski, DWD, FE 13
      Phone: + 49 69 8062 2728
        Fax: +49 69 8062 3721
   E-mail: detlev.majewski@dwd.de

            February 2010



                                    e
                                 uid
                           ´   sG
                      User
                   M-
                 HR
                                                                                -2-


                                                                         Table of contents

1. SHORT OVERVIEW OF THE HRM SYSTEM ................................................................ 4
  1.1 INTRODUCTION .............................................................................................................................................. 4
  1.2 DIFFERENTIAL FORM OF THE MODEL EQUATIONS ........................................................................................... 6
  1.3 HORIZONTAL GRID ......................................................................................................................................... 7
  1.4 VERTIKAL GRID .............................................................................................................................................. 8
  1.5 PHYSICAL PARAMETERIZATIONS .................................................................................................................. 13
     1.5.1 Radiation and clouds.................................................................................................................................. 13
     1.5.2 Grid-scale precipitation ............................................................................................................................. 13
     1.5.3 Convection ................................................................................................................................................. 14
     1.5.4 Turbulent fluxes in the ABL and the free atmosphere ................................................................................ 15
     1.5.5 Subgrid-scale orographic effects parameterization ................................................................................... 15
     1.5.6 Soil model................................................................................................................................................... 16
     1.5.7 Sea ice model ............................................................................................................................................. 18
  1.6 INITIAL STATE AND LATERAL BOUNDARY CONDITIONS ................................................................................ 18
  1.7 INITIALIZATION ............................................................................................................................................ 19
  1.8 SETTING UP THE HRM FOR A NEW REGIONAL DOMAIN ................................................................................ 20
     1.8.1 Definition of the HRM domain ................................................................................................................... 20
     1.8.2 Defintion of grid spacing ........................................................................................................................... 20
     1.8.3 Preparation of the topographical data file ................................................................................................. 21
     1.8.4 Setting up the GME data distribution......................................................................................................... 21
  1.9 CALLING TREE FOR MAIN PROGRAM HRMORG .............................................................................................. 22
  1.10 CALLING TREE OF THE TIME STEPPING ROUTINE PROGORG ......................................................................... 23
  1.11 CALLING TREE OF SUBROUTINE PHYSICS .................................................................................................... 24
  1.12 CALLING TREE FOR POST-PROCESSING SUBROUTINE PP_ORG ..................................................................... 25
2. NAMELIST INPUT OF THE HRM ................................................................................. 26
  2.1 INTRODUCTION ............................................................................................................................................ 26
  2.2 EXAMPLE OF AN INPUT_HRM FILE ............................................................................................................ 27
  2.3 EXPLANATION OF THE DIFFERENT CONTROL VARIABLES AND SWITCHES ..................................................... 28
     / hrm_ctl / - General control variables and switches .......................................................................................... 28
     / grid_ctl / - Grid definition ................................................................................................................................. 30
     / dyn_ctl / - Control variables and switches for the dynamics of HRM ............................................................... 31
     / ini_ctl / - Control variables and switches for the initialization ......................................................................... 32
     / phy_ctl / - Control variables and switches for the physical parameterizations ................................................ 33
     / dia_ctl / - Control variables and switches for diagnostics ................................................................................ 36
     / met_ctl / - Control variables for the meteograph print-out............................................................................... 37
     / gribout / - Control variables and switches for the HRM post-processing......................................................... 38
3. USING THE POST-PROCESSING FUNCTIONS .......................................................... 40

4. FILE NAME CONVENTIONS......................................................................................... 41
  4.1 NAMING CONVENTION FOR DIRECTORIES .................................................................................................... 41
  4.2 GENERAL FORM OF FILE NAMES OF THE HRM ............................................................................................. 41
     Analysis files ....................................................................................................................................................... 41
     Lateral boundary data files ................................................................................................................................. 41
     Forecast files ....................................................................................................................................................... 42
5. MULTITASKING OF THE HRM .................................................................................... 44
  5.1 INTRODUCTION ............................................................................................................................................ 44
  5.2 OPENMP-VERSION OF THE HRM FOR SHARED MEMORY COMPUTERS ......................................................... 45
  5.3 MPI-VERSION OF THE HRM FOR DISTRIBUTED MEMORY COMPUTERS ......................................................... 45
6. COMPUTER RESOURCES NEEDED BY THE HRM ................................................... 47
  6.1 NUMBER OF GRID POINTS PER LAYER ........................................................................................................... 47
  6.2 VERTICAL RESOLUTION ................................................................................................................................ 47
  6.3 TOTAL MEMORY REQUIREMENTS ................................................................................................................. 47
  6.4 TOTAL CPU REQUIREMENT OF THE HRM .................................................................................................... 48
                                                                             -3-


  6.5 TIMING EXAMPLES ....................................................................................................................................... 49
7. OVERVIEW OF THE GRIB1 VARIABLES ................................................................... 52
  7.1 INTRODUCTION ............................................................................................................................................ 52
  7.2 TABLES OF THE GRIB VARIABLES................................................................................................................ 52
  7.3 PRODUCT DEFINITION SECTION PDS AND GRID DESCRIPTION SECTION GDS OF GRIB1 FIELDS................. 61
8. VISUALISATION BASED ON THE GRADS PACKAGE ............................................ 65

9. VISUALISATION BASED ON THE VIS5D PACKAGE ............................................... 66

10. OPERATIONAL USE OF THE HRM BASED ON GME DATA ................................... 67
  10.1 INTRODUCTION .......................................................................................................................................... 67
  10.2 GME DATA NEEDED FOR THE HRM ........................................................................................................... 67
  10.3 NAMELIST INPUT OF THE PROGRAM GME2HRM ................................................................................... 68
  10.4 EXAMPLE OF AN INPUT_GME2HRM FILE ............................................................................................... 69
  10.5 EXPLANATION OF THE DIFFERENT CONTROL VARIABLES AND SWITCHES ................................................... 70
    / org_ctl / - General control variables and switches ........................................................................................... 70
    / grid_gme_ctl / - Grid definition of the GME..................................................................................................... 71
    / grid_hrm_ctl / - Grid definition of the HRM ..................................................................................................... 71
    / dia_ctl / - Control variables and switches for diagnostics ................................................................................ 72
    / gribin / - Control variables and switches of the input....................................................................................... 73
    / gribout / - Control variables and switches of the output ................................................................................... 73
  10.6 MEMORY REQUIREMENTS OF THE PROGRAM GME2HRM ......................................................................... 74
  10.7 OPERATIONAL SCHEDULER ........................................................................................................................ 74
11. INTERPOLATION HRM-X TO HRM-Y ........................................................................ 75
  11.1 INTRODUCTION .......................................................................................................................................... 75
  11.2 HRM-X DATA NEEDED FOR THE HRM-Y .................................................................................................. 75
  11.3 NAMELIST INPUT OF THE PROGRAM HMX2HMY................................................................................... 76
  11.4 EXAMPLE OF AN INPUT_HMX2HMY FILE............................................................................................... 77
  11.5 EXPLANATION OF THE DIFFERENT CONTROL VARIABLES AND SWITCHES ................................................... 77
    / org_ctl / - General control variables and switches ........................................................................................... 77
    / grid_hmy_ctl / - Grid definition of HRM-Y ....................................................................................................... 79
    / dia_ctl / - Control variables and switches for diagnostics ................................................................................ 80
    / gribin / - Control variables and switches of the input ....................................................................................... 81
    / gribout / - Control variables and switches of the output ................................................................................... 81
12. IMPLEMENTATION OF THE HRM SYSTEM .............................................................. 82
  12.1 INTRODUCTION .......................................................................................................................................... 82
  12.2 DOWNLOAD HRM PACKAGE FROM DWD‘S FTP-SERVER ........................................................................... 82
  12.3 CREATION OF THE LIBRARIES WHICH ARE NEEDED BY HRM AND GME2HRM.......................................... 83
  12.4 CREATION AND TEST OF THE BINARIES (HRM, GME2HRM AND HMX2HMY) ........................................ 86
  12.5 HRM ON GNU/LINUX CLUSTERS .............................................................................................................. 90
13. CURRENT USERS OF THE HRM .................................................................................. 93

14. REFERENCES ................................................................................................................ 121
                                               -4-


1. Short overview of the HRM system
1.1 Introduction
The High resolution Regional Model (HRM) is a flexible tool for Numerical Weather Prediction
(NWP). The Deutscher Wetterdienst (DWD) provides this comprehensive package to meteoro-
logical services, universities, and research institutes world-wide.

The HRM package consists of the following parts

   Topographical data sets for any region of the world at mesh sizes between 30 and 5 km; these
    data are prepared at the DWD on request.

   An interface program, called gme2hrm, which interpolates analysis and forecast data of the
    global model GME of the DWD to any HRM grid. The interpolated data serve as initial and
    lateral boundary data for HRM forecasts. Another transformation program, called hmx2hmy,
    allows to interpolate HRM data to another (usually higher resolution) HRM grid.

   The NWP model HRM itself; the main features of the HRM are described in Tab. 1.1.

   Several postprocessing programs which read the GRIB1 code forecast fields of the HRM and
    provide interfaces to the public domain graphic packages GrADS and VIS5D.

   A scheduler based on Korn shell scripts for setting up an operational HRM suite based on
    analysis and forecast data of the GME which are distributed by the DWD via the internet.

   Full scientific documentation of the HRM.

The use of the HRM system requires a written contract between the DWD and the institution in
question. There are three different contracts, one for scientific usage, one for official duty pur-
poses, and finally one for commercial applications. Another contract is necessary for the opera-
tional usage of GME data as initial and lateral boundary conditions for HRM forecasts.

For more information about the HRM system, please contact

General usage                 D. Majewski            e-mail: detlev.majewski@dwd.de
Operational scheduler         M. Gertz               e-mail: michael.gertz@dwd.de
GME-data (operational)        M. Gertz               e-mail: michael.gertz@dwd.de
GME-data (research)           N. Liesering           e-mail: norbert.liesering@dwd.de
Physical parameterization     B. Ritter              e-mail: bodo.ritter@dwd.de
Cloud microphysics            A. Seifert             e-mail: axel.seifert@dwd.de
Topographical data            H. Asensio             e-mail: hermann.asensio@dwd.de
Postprocessing                D. Liermann            e-mail: doerte.liermann@dwd.de
Data assimilation             G. Paul                e-mail: gerhard.paul@dwd.de

Legal contract                J. Christoffer         e-mail: juergen.christoffer@dwd.de

In Section 13, there is a list of the current users of the HRM system. These users may be con-
tacted, too, and are willing to help new users in setting up the HRM system on their local com-
puter environment. For more information: http://www.met.gov.om/hrm/index.php. Please register at
the google group for HRM support: http://groups.google.com/group/hrm_help.
                                              -5-


Table 1.1      Short overview of the hydrostatic High resolution Regional Model HRM

Prognostic variables                                Diagnostic variables
   Surface pressure          ps                       Vertical velocity         
   Temperature               T                        Geopotential              
   Water vapour              qv                       Cloud cover               clc
   Cloud water               qc                       Diffusion coefficients    tkvm/h
   Cloud ice                 qi
   Ozone (optional)          o3
   Horizontal wind           u, v
   Several surface/soil parameters


Numerics of HRM
   Regular or rotated latitude/longitude grid
   Mesh sizes between 0.25° and 0.05° (~ 28 to 6 km)
   Arakawa C-grid, second order centered differencing
   Hybrid vertical coordinate, 30 to 60 layers (Simmons and Burridge, 1981)
   Split semi-implicit time stepping (Burridge, 1975); t = 150s at  = 0.25°
   Helmholtz equation solved by a direct method (FFT and Gauss solver)
   Lateral boundary formulation due to Davies (1976)
   Radiative upper boundary condition as an option (Herzog, 1995)
   Linear fourth-order horizontal diffusion, slope correction for temperature
   Adiabatic implicit nonlinear normal mode initialization (INMI, Temperton, 1991) or
    diabatic (incremental) digital filter initialization (DFI, Lynch, 1997)


Physical parameterizations of HRM
   -two stream radiation scheme (Ritter and Geleyn, 1992) including long- and shortwave
    fluxes in the atmosphere and at the surface; full cloud - radiation feedback; diagnostic deriva-
    tion of partial cloud cover (rel. hum. and convection)
   Grid-scale precipitation scheme including parameterized cloud microphysics (Doms and
    Schättler, 2003)
   Mass flux convection scheme (Tiedtke, 1989) differentiating between deep, shallow and mid-
    level convection or (alternatively) Bechtold (2001) convection scheme
   Level-2 scheme (Mellor and Yamada, 1974) of vertical diffusion in the atmosphere, similar-
    ity theory (Louis, 1979) at the surface
   Subgrid scale orographic (SSO) effects (blocking and wave breaking) due to unresolved
    orography (Lott and Miller, 1997)
   Seven-layer soil model including snow and interception storage (Heise and Schrodin, 2002)
   Sea ice model (Mironov and Ritter, 2003; Mironov and Ritter, 2004)


Programming issues
   Coded in Fortran95; some C subroutines for GRIB encoding/decoding
   Parallelization based on OpenMP for shared memory multi-processors and on MPI for dis-
    tributed memory systems
                                                    -6-


1.2 Differential form of the model equations
The prognostic equations of HRM are expressed in differential form in terms of spherical coor-
dinates (,) and a hybrid (sigma-pressure) vertical coordinate  as follows:

Zonal wind u

u                 . u             
      f  v       
                              1
                                        K   RTv  ln p    u   K 4  4 u -  lbc u  u lbc 
                                                                  
t                       a cos             a cos          t  sub

Meridional wind v

v                 . v    1 
      f  u               K   RTv  ln p    v   K 4  4 v   lbc v  vlbc 
                                                            
t                       a              a             t  sub

Temperature T

T    1  T              T  . T  Lv        T 
          u    v cos            C vc        K 4  T  Tref  -  lbc T - Tlbc 
                                                              4
            
t a cos                   
                               c p c p       t  sub
                                                

Surface pressure ps

p s            1
                     p                  p 
                    u      v cos    d -  lbc  p s  p s,lbc 
           1
                                             
 t     a cos  0                             

Specific water vapour content qv

qv    1  qv             qv  . qv           qv 
              v cos        Cvc   t   K 4  qv -  lbc qv  qv ,lbc 
            u                                                4

 t a cos                    
                                                     sub

Specific cloud water content qc

qc    1  qc            q  . q     q 
           u
               v cos  c    c   c  -  lbc qc  qc,lbc 
                             
 t a cos                      t  sub

Specific cloud ice content qi

qi    1  qi             qi         . qi  qi 
              v cos  
            u                         
                                                  -  lbc qi  qi ,lbc 
 t a cos                                 t  sub

where (u,v) are the zonal (meridional) wind components, T is the temperature, ps is the surface
pressure, qv is the specific water vapour content, qc is the specific cloud liquid water content and
qi is the specific cloud ice content, t is the time and a is the mean radius of the Earth
(a=6371229m),  is the vorticity and f is the Coriolis parameter, is the vertical velocity in the
                                                                   
                                                                          


hybrid system and ω is the vertical velocity in the pressure system, α is the density of air, Φ is
the geopotential and K is the specific kinetic energy, p is the pressure and Tvv is the virtual tem-
                                                                                 v
perature, Tref is a reference temperature depending only on height, Lv is the latent heat of con-
densation, Cvc is the condensation rate and (..)sub is the sub-grid scale tendency due to paramete-
                                                -7-


rized processes like radiation, convection or turbulence, K4 is the constant coefficient of linear
fourth order diffusion.

Variables with the index ―lbc‖ are the lateral boundary conditions prescribed by the driving
model, e.g. the global model GME. In a lateral boundary zone of about six to eight grid rows the
HRM forecast is adjusted towards the one of the driving model with the help of a relaxation term
where lbc is the relaxation coefficient.


1.3 Horizontal grid
The horizontal discretization of the HRM is based on an Arakawa C-grid formulation.



                                         vi,j

                                       Ti,j      ui,j Ti+1,j




Fig. 1 Horizontal placement of variables in the Arakawa C-grid of the HRM


The wind component u is shifted half a mesh size (/2) to the east, the wind component v is
shifted half a mesh size (/2) to the north. The mesh sizes in both directions are the same, i.e.
 = .

The physical mesh size (km) is given by: Δx = a cos φ Δλ and Δy = a Δ φ with a: radius of the
Earth (6371229 m). The grid ―point‖ value is a representative area average over Δx * Δy, i.e. for
Δx = Δy = 28 km: 784 km2.

       (1, je)                                                               (ie, je)



          N
        
       j2 S



                                          WE

       (1,1)                                  j1                            (ie,1)

Fig. 2 Indexing of grid points in the HRM grid
                                                  -8-


HRM counts the grid points starting in the lower left corner to the upper right one (scanning
mode). The first index (j1 or i) is from west to east, the second one (j2 or j) is from south to
north.


1.4 Vertikal grid
The vertical discretiztion of HRM is based on a hybrid coordinate system (sigma-pressure) after
Simmons and Burridge (1981). The pressure ph at the layer interfaces (= half levels) is given by:

       ph (j1, j2, j3) = ak(j3) + bk(j3)*ps(j1, j2)

with   j3 = 1, i3e +1   where i3e is the number of HRM layers. (e.g. i3e = 40 or 60),
       ps               time-dependent surface pressure on the orography of the model,
       j1 = 1, ie       row index (west – east direction),
       j2 = 1, je       column index (south – north direction),

The pressure pf at the center of the layer (= full levels), where most variables of the HRM are
defined, is given by the arithmetic mean of the pressure values at the adjacent half levels:

       pf (j1, j2, j3) = 0.5*( ph (j1, j2, j3) + ph (j1, j2, j3+1) )

with   j3 = 1, i3e.


The vertical coordinate parameters ak and bk are stored in the Grid Description Section (GDS)
of each GRIB1-Record. If the 60-layer version of the HRM uses the same layers like the 60-layer
version of the GME the 2x61 vertical coordinate parameters ak and bk have the following values
(Tab. 1.2). For the former 40-layer version of GME the 2x41 vertical coordinate parameters ak
and bk have the values given in Tab. 1.4.
                                         -9-


Table 1.2   The vertical coordinate parameters ak and bk of the HRM (60-layer version)

   K         ak (Pa)          bk (-)           K           ak (Pa)           bk (-)
    1            0.0        0.00000            32         18970.0          0.28236
    2        1000.0         0.00000            33         18366.6          0.31497
    3        2000.0         0.00000            34         17676.2          0.34876
    4        3000.0         0.00000            35         16907.7          0.38354
    5        4000.0         0.00000            36         16070.4          0.41909
    6        5000.0         0.00000            37         15172.8          0.45526
    7        6000.0         0.00000            38         14227.4          0.49175
    8        7000.0         0.00000            39         13244.6          0.52835
    9        7998.4         0.00002            40         12235.5          0.56482
   10        8990.0         0.00010            41         11211.0          0.60094
   11        9969.2         0.00031            42         10182.2          0.63648
   12       10930.6         0.00076            43          9159.9          0.67123
   13       11875.7         0.00158            44          8154.6          0.70496
   14       12805.4         0.00293            45          7176.4          0.73749
   15       13722.0         0.00494            46          6234.6          0.76861
   16       14620.0         0.00782            47          5338.5          0.79817
   17       15493.2         0.01173            48          4495.9          0.82598
   18       16331.1         0.01687            49          3714.3          0.85192
   19       17122.8         0.02342            50          3000.0          0.87586
   20       17855.2         0.03157            51          2357.9          0.89771
   21       18516.6         0.04147            52          1792.5          0.91738
   22       19094.8         0.05326            53          1306.5          0.93483
   23       19579.7         0.06705            54           901.0          0.95005
   24       19962.4         0.08290            55           575.6          0.96304
   25       20236.3         0.10087            56           328.7          0.97384
   26       20397.0         0.12094            57           156.6          0.98254
   27       20441.8         0.14311            58             52.7         0.98927
   28       20370.4         0.16731            59              6.6         0.99420
   29       20183.6         0.19346            60              0.0         0.99763
   30       19884.9         0.22145            61              0.0         1.00000
   31       19478.5         0.25113
                                              - 10 -


Table 1.3      Pressure (hPa) and mean height (m) of the 60 HRM model layers (full levels)
               for the US-Standard-Atmosphere (US 1976)

    K          pf (hPa)           zf (m)               K         pf (hPa)            zf (m)
     1             5.00           35776                31         462.52              6145
     2           15.00            28368                32         489.31              5734
     3           25.00            25028                33         516.48              5335
     4           35.00            22856                34         543.92              4949
     5           45.00            21247                35         571.52              4576
     6           55.00            19971                36         599.18              4217
     7           65.00            18912                37         626.78              3872
     8           75.00            18004                38         654.17              3541
     9           85.00            17210                39         681.23              3225
    10           95.00            16505                40         707.84              2925
    11         105.04             15868                41         733.88              2639
    12         115.22             15281                42         759.23              2369
    13         125.69             14730                43         783.78              2114
    14         136.62             14201                44         807.44              1874
    15         148.18             13686                45         830.08              1650
    16         160.47             13181                46         851.64              1442
    17         173.61             12681                47         872.01              1248
    18         187.68             12187                48         891.12              1070
    19         202.75             11697                49         908.91                907
    20         218.87             11212                55         925.32                759
    21         236.05             10732                51         940.32                626
    22         254.33             10252                52         953.87                506
    23         273.68              9773                53         965.97                401
    24         294.09              9297                54         976.60                310
    25         315.54              8824                55         985.79                231
    26         337.97              8357                56         993.58                165
    27         361.33              7897                57        1000.02                111
    28         385.55              7445                58        1005.17                 67
    29         410.55              7001                59        1009.15                 34
    30         436.24              6567                60        1012.05                 10

Tab. 1.3 lists the pressure (in hPa) and mean height (in m) of the full levels (60-layer version) of
the HRM for the US-Standard-Atmosphere. The lowest full level is about 10 m above the model
orography, and the lowest model layer with a depth of about 20 m is defined as Prandtl layer
(constant flux layer) in the turbulence parameterization. The uppermost full level is placed at 5
hPa.
                                         - 11 -


Table 1.4   The vertical coordinate parameters ak and bk of the HRM (40-layer version)


                   K          ak (Pa)              bk (-)
                    1              0.0            0.00000
                    2          2000.0             0.00000
                    3          4000.0             0.00000
                    4          6000.0             0.00000
                    5          8000.0             0.00000
                    6          9976.1             0.00039
                    7         11902.1             0.00183
                    8         13722.0             0.00513
                    9         15379.8             0.01114
                   10         16819.5             0.02068
                   11         18045.2             0.03412
                   12         19027.7             0.05169
                   13         19755.1             0.07353
                   14         20222.2             0.09967
                   15         20429.9             0.13002
                   16         20384.5             0.16438
                   17         20097.4             0.20248
                   18         19584.3             0.24393
                   19         18864.8             0.28832
                   20         17961.4             0.33515
                   21         16899.5             0.38389
                   22         15706.5             0.43396
                   23         14411.1             0.48477
                   24         13043.2             0.53571
                   25         11632.8             0.58617
                   26         10209.5             0.63555
                   27          8802.4             0.68327
                   28          7438.8             0.72879
                   29          6144.3             0.77160
                   30          4941.8             0.81125
                   31          3850.9             0.84737
                   32          2887.7             0.87966
                   33          2063.8             0.90788
                   34          1385.9             0.93194
                   35           855.4             0.95182
                   36           467.3             0.96765
                   37           210.4             0.97966
                   38             65.9            0.98827
                   39              7.4            0.99402
                   40              0.0            0.99763
                   41              0.0            1.00000
                                              - 12 -


Table 1.5      Pressure (hPa) and mean height (m) of the 40 HRM model layers (full levels)
               for the US-Standard-Atmosphere (US 1976)

                      K           pf (hPa)              zf (m)
                       1             10.0               31055
                       2             30.0               23849
                       3             50.0               20576
                       4             70.0               18442
                       5             90.1               16842
                       6           110.5                15546
                       7           131.7                14436
                       8           153.8                13452
                       9           177.1                12555
                      10           202.1                11718
                      11           228.8                10930
                      12           257.4                10175
                      13           287.6                 9444
                      14           319.6                 8737
                      15           353.2                 8054
                      16           388.3                 7395
                      17           424.6                 6762
                      18           461.9                 6155
                      19           500.0                 5574
                      20           538.6                 5022
                      21           577.4                 4499
                      22           616.0                 4004
                      23           654.3                 3540
                      24           691.8                 3105
                      25           728.2                 2701
                      26           763.2                 2327
                      27           796.6                 1983
                      28           828.0                 1670
                      29           857.3                 1387
                      30           884.3                 1134
                      31           908.6                   910
                      32           930.4                   714
                      33           949.4                   546
                      34           965.6                   405
                      35           979.1                   289
                      36           989.9                   196
                      37           998.4                   124
                      38          1004.6                    72
                      39          1009.1                    35
                      40          1012.0                    10

Tab. 1.5 lists the pressure (in hPa) and mean height (in m) of the full levels (40-layer version) of
the HRM for the US-Standard-Atmosphere. The lowest full level is about 10 m above the model
orography, and the lowest model layer with a depth of about 20 m is defined as Prandtl layer
(constant flux layer) in the turbulence parameterization. The uppermost full level is placed at 10
hPa.
                                                - 13 -


1.5 Physical Parameterizations
Unresolved atmospheric processes interact with the large-scale flow but contain also essential
forecast information (e. g. cloudiness or precipitation) which can not be generated by the adia-
batic part of the model. The simulation of such processes in the HRM is handled by a set of ded-
icated parameterization modules.

1.5.1 Radiation and clouds
Radiative transfer of solar and thermal radiation in clear and cloudy atmospheres is based on
Ritter and Geleyn (1992). A full radiation step is performed every hour at all grid points, solar
fluxes are computed each time step taking the actual zenith angle into account but the atmospher-
ic transmission from the previous full radiation step is used. The longwave cooling rate is kept
constant during the one-hour period.

Cloudiness is derived from specific cloud liquid water and ice contents, relative humidity, con-
vective activity and stability. If the cloud liquid water content qc exceeds 0 (the grid box is then
saturated with respect to water) the cloud cover is set to 1. In the case of cloud ice q i the thre-
shold is set to 1 mg/kg to take thin cirrus clouds into account which are semi-transparent for
small cloud ice contents.

Partial cloud cover, i. e. in the case of qc = qi = 0, is based on an empirical relation using the rela-
tive humidity of the layer under consideration. In convective situations, i. e. if the convection
scheme diagnosed a convective cloud, the depth of the convective cloud is also taken into ac-
count. If a stable layer exists at the layer of the convective cloud top the cloud cover is increased
to simulate the formation of an anvil.

The computation of the high, medium and low cloud cover (CLCH, CLCM, CLCL) takes the
cloud cover of each model layer (CLC_RAD) into account. If two adjacent model layers are
cloudy, the total cloud cover is the maximum of the two layers (maximum overlap). If a cloud-
free layer is in between, the total cloud cover will be higher than the cloud cover of the single
layers (random overlap).

Radiadition is computed in the subroutine parrad, cloudiness is computed in the subroutine
parclc. The interval between two full radiation steps is controlled by the NAMELIST variables
hincrad_s (solar) and hincrad_t (thermal) in / phy_ctl /.

The heating rate due to solar radiation is stored in SOHR_RAD, the one due to thermal radiation
in THHR_RAD. The solar radiation balance (average value since start of the forecast) at the
top/bottom of the atmosphere is stored in ASOB_T/ASOB_S, the thermal radiation balance in
ATHB_T/ATHB_S.

1.5.2 Grid-scale precipitation
The grid-scale precipitation scheme includes parameterized cloud microphysics (Doms and
Schättler, 2003) with the three prognostic water species water vapour, cloud liquid water and
cloud ice. The precipitation phases (rain and snow) are treated diagnostically. These five phases
interact in many ways (e. g. aggregation, deposition, riming, shedding; see Fig. 3) described by
microphysical processes which are formulated depending on the mixing ratios of the different
water phases.
                                             - 14 -


Grid-scale precipitation is computed in the subroutine pargsp_qi (with prognostic cloud ice) or
pargsp (without prognostic cloud ice; this was the operational version until 16 Sept. 2003).

Grid-scale precipitation (rain and snow, accumulated since the start of the forecast) is stored in
RAIN_GSP and SNOW_GSP.




Fig. 3 Grid-scale parameterization scheme of the HRM including cloud microphysics


1.5.3 Convection
The parameterization of deep and shallow convection is based on a mass flux approach (Tiedtke,
1989) or (alternatively) on Bechtold (2001) scheme which is based on Tiedtke‘s scheme.
Both convection schemes distinguish three different convection types, namely shallow, mid-level
and penetrative (deep). While in the Tiedtke scheme the three-dimensional convergence of water
vapour is used as closure assumption for shallow and deep convection, CAPE (convective avail-
able potential energy) and its convective turnover time-scale (resolution dependent) instead of
the assumption of moisture balance at subcloud layer forms the closure in the Bechtold scheme.
Convective precipitation is initiated only if the cloud depth exceeds 3000 m over land and 1000
m over water.
The height of the base and top of convective clouds (above msl) is stored in the variables
HBAS_CON and HTOP_CON. The convective precipitation (rain and snow, accumulated since
the start of the forecast) is stored in RAIN_CON and SNOW_CON.

To save computing time, the convective parameterization is not computed each time step but
only each ninccon time step (ninccon is defined in NAMELIST / phy_ctl /).

Convection is computed in the subroutine parcon.
                                            - 15 -



                                                                      cloud top

                                                                  detrainment
                                        updraft




                                                                  entrainment
                                                                  and
                                                                  detrainment
               downdraft


           entrainment
           and
           detrainment

                                                     cloud base

     convective gust                                 moisture convergence


Fig. 4 Processes taken into account in the parameterization of convection in the HRM


1.5.4 Turbulent fluxes in the ABL and the free atmosphere
The parameterization of the vertical turbulent fluxes (Müller, 1981) is based on Louis (1979) in
the Prandtl-layer (which is the layer closest to the surface) and a diagnostic level-two scheme
based on Mellor and Yamada (1974) for the boundary layer and the free atmosphere.

Transfer coefficients for momentum, heat and moisture (and the roughness length over sea) at
the surface are computed in the subroutine parturs, diffusion coefficients for momentum, heat
and moisture are computed in the subroutine partura.

To save computing time, the calculation of turbulent diffusion coefficients in the atmosphere is
not computed each time step but only each ninctura time step (in NAMELIST / phy_ctl /).

The transfer coefficient for momentum is stored in TCM, the one for heat and moisture in TCH.
The turbulent diffusion coefficent for momentum is stored in TKVM, the one for heat and mois-
ture in TKVH.


1.5.5 Subgrid-scale orographic effects parameterization
Subgrid-scale orographic variability has a pronounced impact on the mean (resolved) flow of
HRM. Within one model grid box there are height differences in the order of several hundreds of
meters in many mountainous regions. Thus the mean orography alone does not fully describe the
influence of the mountains on the atmosphere. Subgrid-scale orographic features can block on
one hand the flow in the lowest model layers and on other hand give rise to gravity wave which
                                                - 16 -


break and dissipate higher up retarding the flow in the middle and upper troposphere. HRM uses
a subgrid-scale orography (SSO) scheme developed by Lott and Miller (1997). The scheme takes
into account not only the subgrid-scale variability of orography (via the standard deviation of
height) but the orientation, steepness and horizontal anisotropy of the subgrid-scale orographic
structures. These parameters can be derived from the 1 km2 global GLOBE orography data set.
To save computing time, the SSO scheme (subroutine parsso) is not called each time step but
only each nincsso time step (in NAMELIST / phy_ctl /).


1.5.6 Soil model
The multi-layer soil model (Heise and Schrodin, 2002) comprises seven active layers for the heat
(energy) and six for the soil moisture (water) budget.
Prognostic variables are soil temperature, total (liquid and frozen) soil water content and the soil
ice content. Additionally, a snow cover is decribed by a water content of snow, a mean snow
density and a snow ―freshness‖ factor which is used in the computation of the solar albedo. Fi-
nally, a small amount of water can be stored in the interception storage.
Soil temperatures are defined at nine levels: 0, 0.5, 2, 6, 18, 54, 162, 486 and 1458 cm in the
ground. The temperature at the interface soil – atmosphere is identical to the one at a depth of 0.5
cm. The temperature at a depth of 1458 cm is constant and taken from climatological values of
annual mean temperature at 2 m.
Soil moisture is calculated for the six active layers 0 – 1, 1 – 3, 3 – 9, 9 – 27, 27 – 81, 81 – 243
while for the layers 243 – 729 and 729 – 2187 cm it is kept constant.

The soil model is split into two parts parsoil_a and parsoil_b.

                                            Energy budget

                radiation    sensible and                radiation   sensible and
                             latent heat flux                        latent heat flux


                         snow
                         store



               layer 1      snow/soil
                            heat exchange

               layer 2




                                              soil
               layer n                      heat flux


Fig. 5 The processes taken into account in the thermal part of the HRM soil model
                                               - 17 -


                                                  Water budget

          transpiration          snow,                  evaporation
                                 rime                                         rain,
                                                                              dew




                                  snow cover
                                                                      intercepted
                                                                         water
                                                                                            surface
                                                                                            runoff
layer 1

layer 2




                                   capillary                          gravitational
                                   transport                            transport


layer n
                                                                                         subsurface
                                                                                           runoff

Fig. 6 The processes taken into account in the hydrological part of the HRM soil model

The temperature at the surface of a snow cover is stored in T_SNOW; the temperature at the in-
terface surface – atmosphere or surface – bottom of snow cover is stored in T_S. The tempera-
ture at the interface surface – atmosphere is stored in T_G; for land points without snow and wa-
ter points T_S = T_SNOW = T_G. The soil temperature is stored in T_SO.

The water content of the snow cover is stored in W_SNOW, the snow density in RHO_SNOW,
the snow freshness factor in FRESHSNW.
The total soil water is stored in W_SO, the frozen part in W_SO_ICE. Intercepted water is stored
in W_I.

ATTENTION: The temporal integration of the soil model is based on a 2-time level scheme with
the time levels ―t-t‖ (nt2m) and ―t‖ (nt2c), while the rest of the model is integrated by a 3-time
level scheme with the time levels ―t-t‖ (nt3m), ―t‖ (nt3c) and ―t+t‖ (nt3p).


The HRM parameterization modules are nearly identical to those of the global model GME.
                                              - 18 -


1.5.7 Sea ice model
The sea ice model (Mironov and Ritter, 2003; Mironov and Ritter, 2004) computes the thermo-
dynamical effects of an oceanic ice layer. The heat conduction through the ice layer is paramete-
rized via a bulk approach where the form of the temperature profile in the ice layer is prescribed
but variable with time. There are two prognostic variables, namely the temperature at the sea ice
– air interface and the ice thickness. Currently, a snow cover on the sea ice layer is not taken care
of. Ice cover FR_ICE (either 1 or 0, no fractional ice cover) is taken once per day (at 00 UTC)
from observations during the SST analysis. New grid boxes with sea ice are assigned an ice
thickness of 0.5 m. The maximum ice thickness is set to 3 m. The sea ice model can melt exist-
ing ice (by setting FR_ICE to 0) but cannot form new ice.


1.6 Initial state and lateral boundary conditions
Several options exist to derive the initial state of the HRM, namely
     a) Interpolation of the analysis of DWD‘s global model GME (grid spacing 40 km, 40 lay-
         ers until 31 January 2010, grid spacing 30 km, 60 layers from 1 February 2010) to the
         HRM grid,
     b) 3D-Var data assimilation, based on DWD‘s PSAS scheme, ported to the HRM by Hanoi
         National University, Vietnam,
     c) 3D-Var data assimilation, developed by M. Bonavita and L. Torrisi (CNMCA-UGM,
         Rome, Italy),
     d) LAPS analysis, adapted to HRM and used by INMET, Brazil.
Option a) is most suitable for new HRM users with little experience in numerical weather predic-
tion and use of observations. More advanced users should try to implement a data assimilation
system (options b) to d)) because the forecast during the first three to twelve hours will be im-
proved. There is less adaptation of HRM to the fine scale topography, and local observations, not
distributed on the global telecommunication system (GTS), may have a beneficial impact on the
initial state of the HRM.

Two options are available for lateral boundary conditions of the HRM, namely
    a) Forecasts of DWD‘s global model GME, distributed to HRM users free of charge, or
    b) Forecasts of the global model of the ECMWF (Reading, UK), available only to member
        states or associated ones.
DWD provides the analyses and forecasts of GME on all 60 model layers and seven soil layers at
a horizontal resolution of 30 km four times per day, namely
    a) Up to 78 (120) hours at 3-hourly intervals, based on the initial states for 00 and 12 UTC.
        These data are distributed by the DWD via the internet between 02:40 to 03:30 UTC for
        00 UTC and between 14:40 to 15:30 UTC for 12 UTC.
    b) Up to 48 hours at 3-hourly intervals, based on the initial states for 06 and 18 UTC. These
        data are distributed by the DWD via the internet between 08:40 to 09:10 UTC for 06
        UTC and between 20:40 to 21:10 UTC for 18 UTC.
Since the volume of the global data set (more than 30 GByte) is much too big for a timely trans-
fer, DWD provides to each HRM user GME data sets tailored to the respective local HRM do-
main. Lateral boundary data may even be given for a ―frame‖, i.e. not covering the full HRM
domain but only a frame with a width of about 10 HRM rows and columns. Thus the amount of
GME data which has to be transferred via the internet can be reduced drastically. For a typical
HRM domain of 4000x4000 km2 the GME data set (initial state plus 26 lateral boundary data
sets at 3-hourly intervals) needed for a 78-h HRM forecast is in the order of (1 + 26)*12.6
MByte = 341 MByte for the full version and about 110 MByte for the frame version. An internet
                                              - 19 -


connection with a band width between 512 to 1024 kbit/s is fast enough for a timely reception of
the GME data.


1.7 Initialization
To remove high frequency initial noise from the forecast HRM provides two different initializa-
tion schemes, namely
    a) An adiabatic implicit nonlinear normal mode initialization (INMI; Temperton and Roch,
        1991), or
    b) A diabatic (incremental) digital filtering initialization (DFI; Lynch, 1997).
Both initialization schemes remove unwanted ―noise‖ from the first few forecast hours. This
―noise‖ manifests e.g. as high frequency fluctuations of the surface pressure due to an adjustment
between mass and wind fields in the model.

If diabatic processes are the dominant forcing for the development of flow pattern, DFI should
be employed because it includes a short (<= 1800 s) forecast of the HRM with full physics (ex-
cept the soil model). The DFI consists of an adiabatic backward integration over the time range
tspan (in NAMELIST / ini_ctl /) and a diabatic forward integration over the same period. The
prognostic variables (surface pressure, wind components and temperature) are filtered during
these integrations to remove high-frequency oscillations. The humidity fields (relative humidity,
cloud liquid water and cloud ice) are kept unchanged during the initialization.

To reduce the impact of the DFI on the initial state, the filtering is performed in vertical normal
mode space; the number of vertical modes filtered is controlled via the variable nvmdfi (in
NAMELIST / ini_ctl /). Setting e.g. nvmdfi = 20 for a 60-layer HRM (i3e = 60), only the external
and the first 19 internal modes are filtered by the DFI, the higher 40 (21 to 60) modes are left
unchanged. Thus the structure of the atmospheric boundary layer will hardly be changed by the
DFI.

In data assimilation mode (see 1.6 b), c) or d)) it is not advisable to perform a full DFI after each
analysis step because the DFI could damp the meteorological pattern in the whole model domain
even in those regions where the model first guess (FG, i.e. the 3-h or 6-h HRM forecast valid at
analysis time) was not altered by the analysis scheme. In this case the incremental DFI (IDFI)
offers a better solution. For the IDFI the initialized FG (FG(DFI)) is subtracted from the initial-
ized analysis (ANA(DFI)) and this difference is added to the FG.

                          ANA (IDFI) = ANA (DFI) – FG (DFI) + FG

The IDFI modifies the analysis only in those regions where the balanced state of the first guess
fields has been modified due to the input of observations.
                                              - 20 -


1.8 Setting up the HRM for a new regional domain
If a national meteorological service (NMS) decides to use the HRM operationally for its region
of interest, the following steps have to be performed.


1.8.1 Definition of the HRM domain
This task is influenced by several factors which have to be taken into account. The user require-
ments with respect to the domain of interest are of main importance to specify the geographical
longitude and latitude of the lower left corner (startlon, startlat for the HRM) and upper right
corner (endlon, endlat) of the model domain. The actual HRM domain should be larger than the
domain of interest by a zone of 200 to 400 km around. In this zone, close to the lateral bounda-
ries of the HRM, the model adapts the coarser resolution (30 km) GME data to its higher resolu-
tion (between 20 and 5 km). If weather systems predominantly enter the model domain from the
west, the domain of interest could be moved a bit more to the east to provide a larger adaptation
zone in the western part of the HRM domain. Finally, it should be avoided to place the lateral
boundaries of the HRM domain in regions of steep topography because the mismatch between
the topographies of the coarse grid model (e.g. GME) providing the lateral boundary conditions
and the high resolution HRM might cause problems.

Example:
Kenya extends roughly between 34°E, 4°S and 42°E, 4°N. The HRM domain has been set to:
startlon = 26.0, startlat = -12.0 and endlon = 50.0, endlat = 12.0.


1.8.2 Defintion of grid spacing
The definition of the grid spacing (Δλ, Δφ, which must have the same value!) of the HRM
strongly depends on the computer power available and the expected run time of the HRM fore-
cast (e.g. 78-h forecast in 1h 30min wallclock time). Section 6 provides an overview of the com-
puter resources needed by the HRM and shows some timings of the HRM on different computer
platforms.
The number of grid points (ie for the west-east direction and je for the south-north direction) is
computed by

       ie = (endlon – startlon) / Δλ + 1 and je = (endlat – startlat) / Δφ + 1

where Δλ = Δφ is the grid spacing in degree, e.g. Δλ = Δφ = 0.125°.

The grid spacing in km (Δx for west-east direction, Δy for south-north direction) is given by

       Δx = a cos φ Δλ and Δy = a Δφ

where a is the radius of the Earth (a = 6371229 m), φ is the latitude of the grid point and the grid
spacing Δλ = Δφ is given in Radian (e.g. 0.125° corresponds to 0.125° * π / 180° = 0.002181661
Radian).
Thus a grid spacing of 0.125° corresponds approximately to a mesh size of 14 km.

ATTENTION:
Because a Fast Fourier Transform (FFT) is used in the HRM to solve the Helmholtz equation of
the semi-implicit time stepping scheme, the number of grid points in west-east direction has to
                                               - 21 -


fulfil the following condition: ie – 1 must be a power of 2, 3, and/or 5; see Section 2.3 for nam-
elist / grid_ctl /. Thus valid values for ie are e.g. 97, 101, 109, 121, 129, 145, 151, 161, 163, 181,
193, 201, 217, 241, 251, 257, 271, 289, 301, 321, 325, 361, 381, 385, 401, 433, 451, 481, 487,
501, 513, 541, 577, 601, 641, 649, 721, 751, 769, 801, 811, 865, 901, 961.

Since GME uses a grid spacing of 30 km, the range of grid spacings of the HRM is usually in
between 22 to 5 km (or Δλ = Δφ in between 0.20° and 0.05°).
For a given region, the computational cost of the HRM will increase by a factor of almost eight if
the grid spacing is halved. The reason for this rapid increase is due to the number of grid points
(factor 4) and the necessary reduction of the model time step (factor of 2). Thus the reduction of
the grid spacing from 20 to 5 km for a given domain requires almost 64 (= 4 x 4 x 4) times more
computer resources if the production times remain unchanged.

Example:
For the Kenya domain with startlon = 26.0, startlat = -12.0 and endlon = 50.0, endlat = 12.0.
At a grid spacing of 28 km (Δλ = Δφ = 0.25°) ie = 97 and je = 97.
At a grid spacing of 14 km (Δλ = Δφ = 0.125°) ie = 193 and je = 193.


1.8.3 Preparation of the topographical data file
If the HRM domain and grid spacing have been specified by the user, DWD will provide the
necessary topographical data file. High-resolution (usually 1 x 1 km2) data of orography, land
use, soil type and vegetation are accumulated on the HRM grid. The domain covered by the to-
pographical data is typically somewhat larger than the actual HRM forecast domain to allow the
user to extend or move the HRM domain later on a bit.


1.8.4 Setting up the GME data distribution
To reduce data transmission times considerably DWD sends to each HRM user only data of
those GME grid points which are necessary for a given HRM domain. If the HRM domain size is
e.g. 4000 x 4000 km2, only about 20556 GME grid points (from 655362 GME points, each point
covering about 778 km2) have to be transferred via the internet.
For a 78-h HRM forecast the initial data file plus 26 lateral boundary data files (78 h at 3-hourly
intervals) are necessary. Each file contains 6 fields on all 60 model layers and 35 surface and soil
fields (see Section 10.1). To reduce the amount of data even more, the files are compressed by
gzip or bzip2. Finally, the lateral boundary data can be provided for a frame of about 10 HRM
grid rows/columns. This reduces the amount of GME data to be transferred considerably!
To transfer the GME data via the internet to the national meteorological service (NMS) running
the HRM, DWD performs an ―ftp‖ to a computer at the NMS. This computer has to be connected
to the internet with a fixed IP address. Depending on the speed of the link and the size of the data
files the transfer of the GME data for a 78-h forecast may take less than one hour.
                                           - 22 -


1.9 Calling tree for main program hrmorg
hrmorg                            Main program of the HRM

Set up of HRM (grid, initial data, first and second lateral boundary data
(lbc) sets, initialization (INMI or DFI)
      mpi_init                 Initialize MPI communication
      mpi_comm_rank
      mpi_comm_size
      readnlst                 Reads the namelists; domain decomposition
            readnlpp           Read postprocessing namelists gribout
      allocate_fields          Allocate all fields used in HRM

      setup_vartab_1              Generate the table for GRIB-code variables
                                  contained in /comio/
            setup_vartab
      setconst1                   Setting physical, mathematical, and some
                                     parameterization constants
      get_sd                      Open the GRIB file(s) with the start data
      read_sd                     Read the start data file(s) of HRM
      gengrid                     Generate grid and calculate operators
            passco                Initialize the FFT in west – east direction
            recvreal1             MPI communication of ak, bk to all processors
            sendreal1             MPI communication of ak, bk to all processors
            gen_vert              Calculate the parameters related to the
                                     vertical grid;
      setconst2                   Set additional (grid dependent) constants
      setup_verint                Set up for output on pressure levels
      setup_meteograph            Set up the the meteograph files
      get_lb                      Open the GRIB file with the first lbc data
      read_lb                     Read the first lbc data file of HRM
      get_lb                      Open the GRIB file with the second lbc data
      read_lb                     Read the second lbc data file of HRM
      id_mix_lb                   Mix initial and lbc data in the boundary zone
      recv_lb                     MPI communication of lbc data to all processors
      inmi                        Perform a implicit normal mode initialization
                                     (INMI), if required or alternatively
      df_ini                      Perform a digital filter initialization
                                     (DFI), if required
             dolph                Calculate Dolph-Chebyshev filter window
                  cheby           Calculate Chebyshev polynomials
            progorg               Adiabatic backward integration
            condens               Effects of condensation/evaporation
            dolph
                  cheby
            progorg               Diabatic forward integration
            condens
      read_fg_incr                For IDFI, read FG increments

Time stepping loop
      get_lb                      Open the GRIB file with next lbc data
      read_lb                     Read next lbc data file of HRM
      recv_lb                     MPI communication of lbc data to all processors
      progorg                     Forward time stepping of HRM
      save_fg                     For IDFI, save FG fields

End of main program
      mpi_barrier
      mpi_finalize                End MPI communication
                                          - 23 -


1.10 Calling tree of the time stepping routine progorg
progorg                         Organize the time stepping of the HRM
    prog_s_01                   Called only for first time step
        geopot                  Calculate the geopotential at half and full
                                   levels for all layers
    recv_full                   For MPI version, send all prognostic/diagnostic
                                   variables from processor 0 to all processors
    prog_s_02                   Copy the lbc data to time step t+dt of all
                                   prognostic variables
    physics                     Parameterization of diabatic processes
    send_recv_physics           Extend variable qrs_gsp by two rows and columns
    prog_s_03
        progexp                 Calculate the explicit forecast of the HRM for
                                   ps, T, qv, qc, qi, u, v
            grad2s              Calculate pressure gradient
            div2v               Calculate the horizontal mass divergence
            vor2v               Compute relative vorticity
            hdiff_v             Horizontal diffusion of u, v
            grad2s              Gradient of T
            hdiff_s             Horizontal diffusion of T, qv
    send_recv_prog_03           Extend variables ps, t, v by one row
    prog_s_04
        sicor1                  Compute right-hand-sides of Helmholtz equations
                                   of the semi-implicit time stepping
              lap2s             Compute Laplacian of sec. temp. derivative of
                                   generalized potential
              div2v             Compute divergence of sec. temp. derivative of
                                   horizontal wind
    send_recv_prog_04           Extend variables rhs_si and exp_si by one row and
                                   column
    prog_s_05
        sicor2              Solve the 2-d Helmholtz equations for mode kj3
                               FFT in west – east direction; Gaussian
                               elimiation in south – north direction
             gaelco         Compute coefficients for solver
             solve_pois_helm Solve one 2D Helmholtz equation
    prog_s_06
         sicor3             Back-transformation of the solution of the 2D-
                               Helmholtz equations; add the SI corrections
                               to the values of the prognostic fields
             condens        Calculate effects of condensation/evapoaration
                               on T, qv, qc
    prog_s_07
         asselin            Asselin filtering of prognostic variables
         lb_relax           Perform lateral boundary relaxation (Davies)
         condens            Calculate effects of condensation/evapoaration
                               on T, qv, qc
    prog_s_08
         geopot             Calculates the geopotential at half/full levels
         nearsfc            Computes the near surface fields
             calqvs         Calculation of surface humidity over water
    send_recv_prog_08       Extend all prognostic/diagnostic variables by two
                               rows and columns
    send_full               Send all prognostic/diagnostic variables to
                               processor 0 (only every ninc_s_f time steps)
    rubc                    Compute radiative upper boundary condition (if
                               switched on)
    dia_phys                Compute area mean quantities for diagnostic
                               purposes and print in file DIAGNOSTICS
    meteograph              Print meteographs for stations selected
         date_time             Compute date and time of forecast
                                          - 24 -


    diagnos                     Compute area mean quantities for a quick check,
                                   Print values in file OUTPUT_HRM
    caladdfld                   Computes additional fields
        caltopdc                   height of top of dry convection
        calclmod                   modified cloud depth, modified cloud cover
        calhzero                   height of 0 C-level
        calverintegr               total content of qv, qc, qi, O3
        calceiling                 height of lowest significant cloud layer
        calconvin                  CAPE_ML, CIN_ML, CAPE_MU, CIN_MU
    pp_org                      Organizes the post-processing
    make_fn                     Create file name of Ready-file



1.11 Calling tree of subroutine physics
physics                         Parameterisation of diabatic processes
    pargsp_qi                   Grid-scale precipitation processes
    parclc                      Calculation of partial cloud cover, related
                                layer properties for radiative transfer scheme
    parrad                      Longwave and shortwave radiation
        calc_smu0               Calculation of solar zenith angle
        aerosol_opt             Parameters for optical depth of different
                                   aerosols and vertical distribution
         ozone                  Parameters of a T5 spectral distribution
                                   of ozone depending on the time of year
         legtri                 Legende function for triangular truncation
         fesft                  Organise the radiative transfer calculations
             opt_th             Optical properties of the non-gaseous
                                   constituents for one spectral interval in the
                                   thermal spectrum
              inv_th            Solve the linear system of equations for thermal
                                   fluxes
                   coe_th       Optical effects of atmospheric layers on thermal
                                   radiation based on basic optical properties
                                   of non-gaseous constituents and gaseous
                                   absorption coefficients
              opt_so            Optical properties of the non-gaseous
                                   constituents for one spectral interval in the
                                   solar spectrum
              inv_so            Solve the linear system of equations for solar
                                   fluxes
                   coe_so       Optical effects of atmospheric layers on solar
                                   radiation based on basic optical properties
                                   of non-gaseous constituents and gaseous
                                   absorption coefficients
        date_time
    partura                     Calculate atmospheric turbulent exchange
                                   coefficients
    parturs                     Calculate turbulent transfer coefficients in
                                   surface layer
    moist_c                     Calculate fields which are needed in the moist
                                   convection schemes (Tiedtke or Bechtold)
        grad2s                  Calculate gradient of scalar
        div2v                   Calculate the 2-d divergence of horizontal wind
    parcon                      Organize the Tiedtke convection scheme
        cu_ini                  Initialization of cumulus convection arrays
        cu_base                 Calculate cloud base parameters
        cu_asc                  Perform cloud ascent calculations
        cu_dlfs                 Calculate level of free sinking for downdrafts
        cu_ddraf                Perform cumulus downdraft calculations
        cu_asc                  Perform cloud ascent calculations
                                          - 25 -


         cu_flux               Calculate final values of convective fluxes
         cu_dtdq               Calculate surface precipitation and tendencies
                                  of T and qv due to convection
        cu_dudv                Calculate momentum tendencies due to convection
    parcon_2                   Organize the Bechtold convection scheme
        SUCST                  Initialize the constants of the model
        SU_YOETHF              Initialize some derived variables
        SUCUMF                 DEFINE DISPOSABLE PARAMETERS FOR MASSFLUX SCHEME
        SUPHLI                 SET-UP ROUTINE FOR COMMON BLOCK *YOEPHLI*
        SUVDF                  SET-UP ROUTINE FOR COMMON BLOCK *YOEVDF*
        SUVDFS                 INITIALIZE COMMON BLOCK *YOEVDFS*
        SUCLDP                 INITIALIZE COMMON YOECLD CONTROLLING *CLOUDSC*
        CUMASTRN               Master routine for the Bechtold convection scheme

    parsoil_a                  First part of the soil process parametrisation
    parsoil_b                  Second part of soil process parametrization
    parsso                     Organize the execution of the SSO scheme
    parsea_ice                 Organize the execution of the sea ice model



1.12 Calling tree for post-processing subroutine pp_org
pp_org                               Organize post-processing of GRIB code files
    pp_verint                        Vertical interpolation on p-levels
    pp_makegribs                     Pack fields in GRIB code and write them out
        pp_open_file                 Open output file
        pp_output                    Organize the output
            pp_output_init           Initializes output for namelist *namelist*
                 pp_timecode         Set unit of time and time range indicator
            pp_output_write          Do the output of GRIB code files
                 pp_makegds          Grid definition section of GRIB
                 pp_makepds          Product definition section of GRIB
                 grbex1              Pack data in GRIB format
                 cuegex              Write packed data to file
        cclose                       Close the GRIB output file
                                             - 26 -


2. NAMELIST Input of the HRM
2.1 Introduction
The runtime control parameters of the HRM are contained in an ASCII file called
INPUT_HRM in the form of NAMELIST variables.
These variables are split into nine different NAMELIST groups which have to appear in the fol-
lowing order in the file INPUT_HRM:

/ hrm_ctl /    -      general control variables and switches mostly contained in COMMON
                      block / comorg /
/ grid_ctl /   -      control variables defining the HRM grid, contained in COMMON
                      block / comgrid /
/ dyn_ctl /    -      control variables and switches for the dynamics of the model, contained
                      in COMMON block / comdyn /
/ ini_ctl /    -      control variables and switches for the initialization of the HRM, either
                      an implicit nonlinear normal mode initialization (INMI) or a digital
                      filtering initialization (DFI), contained in COMMON block / comini /
/ phy_ctl /    -      control variables and switches for the physics of the HRM, contained in
                      COMMON block / comphy /
/ dia_ctl /    -      control variables and switches for the diagnostics of the HRM,
                      contained in COMMON block / comdia /
/ met_ctl /    -      control variables and switches for the meteograph print-out, contained
                      in COMMON block / comgpmet /; for each grid point/station to be printed,
                      there must be one NAMELIST block / met_ctl /
/ gribin /     -      control variables and switches for the input of the initial data (analysis)
                      and the lateral boundary data (analyses, or forecasts of another model,
                      e.g. , the global model GME of the DWD), contained in COMMON
                      block / comio /
/ gribout /    -      control variables and switches for the post-processing of the HRM, i.e.
                      the creation of GRIB1 output files. The control variables and switches
                      are contained in COMMON block / comio /


The last NAMELIST group ( / gribout / ) can occur several times. Each group may define a dif-
ferent list of variables for GRIB output, different time steps and/or different (sub-) domains. If
no GRIB output is required, this group can be omitted altogether.
                                          - 27 -


2.2 Example of an INPUT_HRM file
An example of an INPUT_HRM file is given in Fig. 2.2.1

#
# Namelist Input Control file (INPUT_HRM)
#
 &hrm_ctl
  ydate_ini='2006011500', nproc1=1, nproc2=8, lmpi=.true.,
  lana_lb=.false., lana_qi=.true., llb_qi=.true., hinc_lb=3., hstop=72.,
 &end
 &grid_ctl
  ie=217, je=193, i3e=60, pollat=90.0, pollon=-180.0,
  startlat=-36.0000, startlon=12.000, endlat=-12.000, endlon=39.00,
 /end
 &dyn_ctl
  dt=90., lsits=.true., epsass=0.15, nsi=8, rkh4=0.8e12, ldry=.false.,
  rkh2=0.8e5,
 /end
 &ini_ctl
  ldfi_nstart=.true., dtbak=90., dtfwd=90.,
 /end
 &phy_ctl
  lphys=.true., lrad=.true., lgsp=.true., lsso=.true.,
  lcon=.true., ltur=.true., lsoil=.true.,
  hincrad_s=1., hincrad_t=1., ninctura=5, ninccon=5, nincsso=5,
 &end
 &dia_ctl
  hincdia=6.,
  hincctl=1.,
 &end
 &met_ctl
  ngpmet=2, hincmet=1., ymettype='short',
  zlongpmet=25.60, zlatgpmet=-21.42, ynamegp='Letlhakane',
 /end
 &met_ctl
  zlongpmet=22.50, zlatgpmet=-15.00., ynamegp='Met_Point_2'
 &end
 &gribin
  yanadir='/uhome/for3maj/hrm/',
  ylbdir= '/uhome/for3maj/hrm/',
 &end
 &gribout
  ngrib=0, lpp_ini=.true.,
  ysystem='file',
  ydir='/uhome/for3maj/hrm/',
  yvarpl='T','FI','U','V','PS','RELHUM','OMEGA',
  yvarml='FIS','FR_LAND',
 &end
 &gribout
  hcomb=3.,72.,3.,
  ysystem='file',
  ydir='/uhome/for3maj/hrm/',
  yvarml='T_G','T_2M','ASOB_S','ATHB_S','ASOB_T','ATHB_T',
         'RAIN_GSP','SNOW_GSP','RAIN_CON','SNOW_CON',
         'CLCH','CLCM','CLCL','CLCT',
         'ASHFL_S','ALHFL_S',
         'U_10M','V_10M','TD_2M',
         'U','V','T','HBAS_CON','HTOP_CON','HTOP_DC','PS',
  yvarpl='T','FI','U','V','PS','RELHUM','OMEGA',
 &end


Figure 2.2.1 Example of an input file of the HRM
                                           - 28 -


2.3 Explanation of the different control variables and switches
/ hrm_ctl / - General control variables and switches

 Parameter          Type               Meaning of Parameter                        Default
ydate_ini         CH*10   Initial date of the forecast in the form             ´          ´
                          yyyymmddgg, where
                          yyyy: year, e.g. 2004
                          mm: month, e.g. 06
                          dd:     day,     e.g. 25
                          gg:     time, e.g. 18 (UTC)
ydate_lb          CH*10   Initial date of the forecast providing the lateral   ´          ´
                          boundary values in the form
                          yyyymmddgg;
                          if omitted, ydate_lb = ydate_ini
nproc1            INT     Number of processors used in parallel in                   1
                          the west – east direction.
                          A 2D-domain decomposition is used.
nproc2            INT     Number of processors used in parallel in                   1
                          the south – north direction.
                          ATTENTION:
                          The total number of processors used:
                          nproc = nproc1 x nproc2 must not exceed 256.
nstart            INT     Number of first time step; if nstart > 0, HRM              0
                          is in restart mode, i.e. it requires data of two
                          consecutive time levels (t, t-t) to resume a
                          forecast.
                          Alternatively:
hstart            REAL    Same as nstart, but time in hours                          0.
nstop             INT     Number of time steps to be performed;                      0
                          Alternatively:
hstop             REAL    Same as nstop, but time in hours                           0.
nincmxm           INT     Interval in time steps for the validity of maxi-           0
                          mum and minimum values, i.e. every nincmxm
                          time steps the relevant arrays are reset to de-
                          fault values.
                          Alternatively:
hincmxm           REAL    Same as nincmxm, but time in hours                         6.
ninc_lb           INT     Interval in time steps between two sets of lat-            0
                          eral boundary values.
                          Alternatively:
hinc_lb           REAL    Same as ninc_lb, but time in hours                         6.
lana_lb           LOGICAL If .true., HRM uses analyses as lateral boun-            .false.
                          dary condition; if .false., the forecast of anoth-
                          er model, e.g. the GME of the DWD, is used
ldebug            LOGICAL Debug switch, if .true. HRM will print quite a           .false.
                          lot of debug information
lmpi              LOGICAL MPI switch, if .true., run HRM with commu-               .false.
                          nication based on MPI for distributed memory
                          computers
                                     - 29 -


 Parameter    Type               Meaning of Parameter                            Default
ninc_s_f     INT       Interval in time steps for a full update of all               0
                       prognostic/diagnostic fields in processor 0 for
                       the MPI-Version of the HRM
hinc_s_f     REAL      Same as ninc_s_f but time in hours                            1.
ytrans_in    CH*100    Name of directory for ´ready´ files. These files      ´            ´
                       are used to tell HRM during the run that a spe-
                       cific lateral boundary data file is ready. (if
                       empty, HRM will not wait for the files).
ytrans_out   CH*100    Name of directory for ´ready´ files. These files      ´            ´
                       are used to tell programs which use HRM data
                       during the run that a specific step has been
                       completely written and the corresponding
                       GRIB file is ready for usage.
lana_qi      LOGICAL   qi switch; if .true. take initial values of qi from   .true.
                       analysis file. If .false. qi is set to 0 initially.
llb_qi       LOGICAL   qi switch; if .true. take lateral boundary values     .true.
                       of qi from lateral boundary data file. If .false.
                       qi is copied from the interior of the HRM do-
                       main to the lateral boundaries.
lprog_o3     LOGICAL   Ozone (o3) switch; if .true. compute o3 as pas-       .false.
                       sive tracer, no ozone chemistry. Initial values
                       of o3 have to be provided in the initial data
                       file.
llb_o3       LOGICAL   o3 switch; if .true. take lateral boundary values     .false.
                       of o3 from lateral boundary data file. If .false.
                       o3 is copied from the interior of the HRM do-
                       main to the lateral boundaries.
qvmin        REAL      Minimum value of water vapour (security); if          1.e-12
                       qv < qvmin, qv is set to qvmin.
qcmin        REAL      Minimum value of cloud water (security); if qc        1.e-12
                       < qcmin, qc is set to 0.
qimin        REAL      Minimum value of cloud ice content (securi-           1.e-12
                       ty); if qi < qimin, qi is set to 0.
o3min        REAL      Minimum value of ozone mixing ratio (securi-          1.e-20
                       ty); if o3 < o3min, o3 is set to 0.
nrow         INT       Number of rows for south-north scan in some               6
                       subroutines; used to control the stack space
ltiming      LOGICAL   Timing switch; if .true. perform timing of            .true.
                       HRM and print results to the logfile
lmix_idlb    LOGICAL   Mixing switch: Mix initial data and lateral           .true.
                       boundary values if HRM is driven by forecasts
                       of another model
llb_pprun    LOGICAL   LBC-PP switch; if .true. run HRM to perform           .false.
                       the postprocessing of the LBC data only (i. e.
                       interpolation to pressure levels). No real HRM
                       forecast is performed!
lframe       LOGICAL   Frame switch; if .true. LBC data are frames           .false.
nframe       INT       Frame width in rows/columns                            10
                                                - 30 -


/ grid_ctl / - Grid definition

 Parameter             Type                 Meaning of Parameter                         Default
pollat               REAL         Latitude of rotated north pole; for regular lati-     90.
                                  tude/longitude grid set pollat = 90. (°)
pollon               REAL         longitude of rotated north pole; for regular         -180.
                                  latitude/longitude grid set pollon = -180. (°)
startlat             REAL         latitude of lower left corner of the HRM do-            0.
                                  main (in °)
startlon             REAL         longitude of lower left corner of the HRM do-           0.
                                  main (in °)
endlat               REAL         latitude of upper right corner of the HRM do-           0.
                                  main (in °)
endlon               REAL         longitude of upper right corner of the HRM              0.
                                  domain (in °)
ie                   INT          number of grid points in west-east direction            0
je                   INT          number of grid points in south-north direction          0
i3e                  INT          number of layers in the atmosphere                      0
i3e_soil             INT          number of soil layers                                   7


Attention:

The mesh sizes dlon, dlat are not controlled via NAMELIST but computed from ie, startlon,
endlon and je, startlat, endlat (see subroutine readnlst).


         Δλ = (endlon – startlon) / (ie – 1) and Δφ = (endlat – startlat) / (je – 1)

where Δλ = Δφ is the grid spacing in degree, e.g. Δλ = Δφ = 0.125°.


Because an FFT-solver is used in the west-east direction, ie-1 must be a power of 2, 3, and/or 5,
i.e.

         ie – 1 = 2l x 3m x 5n

where l, m, n are positive integers (l > 0);
e.g. ie = 101, 109, 121, 129, 145, 151, 161, 163, 181, 193, 201, 217, 241, 251, 257, 271, 289,
301, 321, 325, 361, 381, 385, 401, 433, 451, 481, 487, 501, 513, 541, 577, 601, 641, 649, 721,
751, 769, 801, 811, 865, 901, 961.

If the radiative upper boundary condition is used, i.e. for rubc = .true. in NAMELIST
/ dyn_ctl /, je-1 must be a power of 2, 3, and/or 5, too.
                                               - 31 -


/ dyn_ctl / - Control variables and switches for the dynamics of HRM

 Parameter            Type                 Meaning of Parameter                          Default
dt                  REAL         Time step (in seconds)                                200.
rkh4                REAL         Diffusion coefficient (linear fourth order, unit:       0.
                                 m4/s)
rkh2                REAL         Diffusion coefficient (linear second order,             0.
                                 unit: m2/s)
lsits               LOGICAL      Logical switch for semi-implicit (SI) time            .true.
                                 stepping; if .true., perform SI time stepping.
lray                LOGICAL      Logical switch for Rayleigh damping of wind           .true.
                                 field to reduce risk of CFL violation
i3e_ray             INT          Rayleigh damping is applied from layer index            4
                                 1 to layer index i3e_ray
ldry                LOGICAL      Logical switch for dry forecast: if .true., per-      .false.
                                 form dry forecast, i.e. set water vapour, cloud
                                 water and cloud ice contents to 0.
lrubc               LOGICAL      Logical switch for the radiative upper boun-          .false.
                                 dary condition: if .true., use the r.u.b.c.
                                 ATTENTION: Only for nproc1 = 1!
nsi                 INT          Number of modes used in the semi-implicit                5
                                 time stepping (split semi-implicit scheme)
epsass              REAL         Asselin time filter coefficient; the Asselin fil-       0.15
                                 ter ties the three time levels
                                 (t-t), t, (t+t) of the leap frog time integration
                                 scheme together.

Attention:
For the semi-implicit time stepping scheme (i.e. for lsits = .true.), the time step t must fulfill the
Courant-Friedrich-Levy Criterion t < xmin / (2 vmax), where xmin is the minimum mesh
size in west-east direction (see Section 1.8.2) and vmax is the maximum wind speed (around 110
m/s). For example, for xmin = 10 km and vmax = 110 m/s the maximum stable time step is t =
60s. The HRM will abort with an error message if the wind speed exceeds the CFL-Criterion.
The horizontal diffusion coefficients must not exceed certain critical values which depend on the
mesh size  and the time step t:
rkh4 < (2 )4 / (32 4 t)    and rkh2 < (2 )2 / (8 2 t)
The following table lists the time steps and maximum horizontal diffusion coefficients for some
mesh sizes

 (°)              0.250            0.200             0.125            0.100             0.0625
 (km)               28               22                14               11                  7
t (s)              150              100                75               60                 40
rkh2 (m2/s)      2.6 x 105        2.5 x 105         1.3 x 105        1.0 x 105          6.2 x 104
rkh4 (m4/s)      2.1 x 1013       1.2 x 1013        2.6 x 1012       1.3 x 1012         3.1 x 1011

Note:
As a rule of thumb, use 25% to 50% of the maximum horizontal diffusion coefficients.
                                             - 32 -


/ ini_ctl / - Control variables and switches for the initialization

 Parameter           Type                Meaning of Parameter                       Default
lnmi               LOGICAL Logical switch, if .true., perform the INMI          .false.
                           prior to the forecast.
nitnmi             INT     Number of iterations of the INMI                        3
nvm                INT     Number of vertical modes treated by the INMI            5
dtnmi              REAL    Time step during the INMI                              dt
ldfi_nstart        LOGICAL Logical switch, if .true., perform a DFI prior to    .false.
                           the forecast.
ldfi_nstop         LOGICAL Logical switch, if .true., perform a DFI at the      .false.
                           end of the forecast.
ldfi_incr          LOGICAL Logical switch, if .true., perform an IDFI prior     .false.
                           to the forecast.
ninc_fg            INT     Forecast length in time steps of first guess           0
hinc_fg            REAL    Same as ninc_fg but time in hours                      6.
ydate_fg          CH*10    Initial date of first guess (FG) in the form         ‗ ‗
                           yyyymmddgg (for IDFI)
ydate_lbfg        CH*10    Initial date of LBC data used to produce the         ‗        ‗
                           first guess (FG) for IDFI in the form
                           yyyymmddgg
luselbfg          LOGICAL Logical switch, if .true., run wth LBC data           .false.
                           used to produce first guess in IDFI or correct
                           LBC data in dfi_nstop
hstep_dfi         REAL     Forecast time (in hours) for production of DFI           0.
                           FG (for 3D-VAR FGAT)
nstep_dfi         INT      Forecast time (in time steps) for production of          0
                           DFI FG (for 3D-VAR FGAT)
nvmdfi            INT      Number of vertical modes which are being             i3e
                           filtered during the DFI
lhdiff             LOGICAL Logical switch, if .true., perform the DFI with      .true.
                           horizontal diffusion
lepsass            LOGICAL Logical switch, if .true., perform the DFI with      .true.
                           Asselin filtering
tspan              REAL    Time-span (in seconds) for the adiabatic and         1800.
                           diabatic stages of the initialization
taus               REAL    Cutoff period (in seconds) for the filter (taus is   1800.
                           the stop-band edge of the Dolph filter)
dtbak              REAL    Time step (s) for the hindcast filtering stage           dt
dtfwd              REAL    Time step (s) for the forecast filtering stage           dt
ndfi               INT     Indicator for the method of filtering:                   2
                           = 0: No filtering (ldfi = .false.)
                           = 1: Launching (forward stage only)
                           = 2: Full two-stage filtering (default)
nfilt              INT     indicator for method of filtering                        1
                           = 1: Dolph-Chebyshev filter (default)
                           (no other filter possible yet)
                                             - 33 -


Attention:

Two different initialization schemes are available for the HRM, namely the adiabatic implicit
nonlinear normal mode initalization (INMI) and the diabatic digital filtering initialization (DFI).
Usually the DFI will result in a better physical balance of the initial state of the HRM.

dtnmi must be around 1/7 of dt (see the stability criteria for dt and dtnmi in the scientific docu-
mentation).

NINT(tspan/dtbak) and NINT(tspan/dtfwd) must be even numbers!


/ phy_ctl / - Control variables and switches for the physical parameterizations

 Parameter           Type                Meaning of Parameter                       Default
lphys              LOGICAL Logical switch, if .true., run with physical pa-       .true.
                           rameterizations. If .false., no physical parame-
                           terizations will be computed, but grid-scale
                           condensation/evaporation is included assum-
                           ing 100% relative humidity over water inside
                           of clouds. To suppress this calculation, too, set
                           ldry = .true. in NAMELIST group / dyn_ctl /.
lrad               LOGICAL Logical switch, if .true., run with the radiation       .true.
                           scheme which computes heating rates in the
                           atmosphere (solar and thermal) and the energy
                           balance (solar and thermal) at the ground.
                           To save computing time, radiation will be
                           called at certain intervals defined by the fol-
                           lowing four control variables:
nincrad_s          INT     Interval (in time steps) between two complete            0
                           solar radiation computations;
                           Alternatively:
hincrad_s          REAL    Interval (in hours) between two complete solar           2.
                           radiation computations. An interval of one or
                           two hours yields sufficient accuracy.
nincrad_t          INT     Interval (in time steps) between two complete            0
                           thermal radiation computations;
                           Alternatively:
hincrad_t          REAL    Interval (in hours) between two complete                 2.
                           thermal radiation computations. An interval of
                           one or two hours yields sufficient accuracy.
lgsp               LOGICAL Logical switch, if .true., run with the grid-scale      .true.
                           precipitation scheme which computes the ef-
                           fect of grid-scale precipitation on temperature,
                           water vapour and cloud liquid water (and op-
                           tionally cloud ice) in the atmosphere as well as
                           the rates of grid-scale rain and snow fall at the
                           ground.
lcon               LOGICAL Logical switch, if .true., run with moist con-          .true.
                                        - 34 -


 Parameter        Type              Meaning of Parameter                       Default
                           vection (Tiedtke´s mass flux scheme) which
                           computes the effect of moist convection on
                           temperature, water vapour and winds in the
                           atmosphere as well as the rates of convective
                           rain and snow fall at the ground.
                           To save computing time, moist convection
                           may not be called each time step, but at certain
                           intervals defined by the following variable:
ntype_con        INT       Type of convection parameterization                 1
                           0: no convection
                           1: Tiedtke scheme
                           2: Bechtold scheme
ninccon          INT       Interval (in time steps) between two calls of        5
                           the moist convection parameterization scheme.
                           ninccon should be an odd number to avoid
                           using the same time family all the time. ninc-
                           con = 3 or 5 yields sufficient accuracy.
ltur             LOGICAL   Logical switch, if .true., run with the turbu-     .true.
                           lence (vertical diffusion) scheme which com-
                           putes exchange coefficients in the atmosphere
                           (different coefficients for temperature/water
                           substances and momentum) and the transfer
                           coefficients at the ground (Prandtl layer). Over
                           water, the roughness length z0 is computed,
                           too.
                           To save computing time, the exchange coeffi-
                           cients in the atmosphere may not be computed
                           each time step but at certain intervals defined
                           by the following control variable:
ninctura         INT       Interval (in time steps) between two calls of       5
                           the computation of vertical exchange coeffi-
                           cients. ninctura should be an odd number to
                           avoid using the same time family all the time.
                           ninctura = 3 or 5 yields sufficient accuracy.
lsoil            LOGICAL   Logical switch, if .true., run with the surface    .true.
                           (soil) parameterization scheme. Over water,
                           the sea surface temperature (SST) is kept con-
                           stant during the integration.
lmelt_soil       LOGICAL   Logical switch, if .true., include melt-           .true.
                           ing/freezing in soil model
lmelt_soil_var   LOGICAL   Logical switch, if .true., include temperature     .true.
                           dependency of melting
lforest          LOGICAL   Logical switch, if .true., run with forest data    .false.
                           (evergreen and deciduous)
llai             LOGICAL   Logical switch, if .true., use leaf area index     .false.
                           data
nsoil_moist      INT       Index of soil layer with artificial setting of      0
                           moisture content
ntype_trvg       INT       Type of vegetation transpiration                    2
                                     - 35 -


 Parameter     Type               Meaning of Parameter                    Default
ntype_evsl   INT     Type of parameterization of bare soil evapora-       2
                     tion (2: BATS scheme)
lai_min      REAL    Minimum value of leaf area index (LAI)                1.e-10
lsso         LOGICAL Logical switch, if .true., run with the SSO         .true.
                     scheme.
                     To save computing time, the SSO scheme may
                     not be executed each time step but at certain
                     intervals defined by the following control vari-
                     able:
nincsso      INT     Interval (in time steps) between two calls of        5
                     the SSO scheme. nincsso should be an odd
                     number to avoid using the same time family all
                     the time. nincsso = 3 or 5 yields sufficient ac-
                     curacy.
lsea_ice     LOGICAL Logical switch, if .true., run with the sea ice     .false.
                     model; in this case HRM expects to read
                     fr_ice, h_ice, t_ice in the initial data set and
                     h_ice, t_ice in the lateral boundary data sets.

condp_l      REAL        Minimum cloud depth (in Pa) over land for the   3.0e4
                         production of precipitation.


condp_w      REAL        Minimum cloud depth (in Pa) over water for      1.5e4
                         the production of precipitation.
                                            - 36 -


/ dia_ctl / - Control variables and switches for diagnostics

As of now, two types of diagnostics have been implemented in HRM, namely dia and ctl.

dia computes mean values of all prognostic variables (plus some derived quantities) for the total
model domain (land and water points) and for land and water separately. Multi-level fields are
given on each model level. The results are written to an ASCII file named DIAGNOSTICS.

ctl computes control variables, mostly mean values of quantities like surface pressure, surface
pressure tendency, kinetic energy, maximum wind speed, cloud cover or precipitation rates,
which allow a quick-look control of the model run. The results are written to an ASCII file
named OUTPUT_HRM.

 Parameter           Type                Meaning of Parameter                     Default
nstartdia          INT         Number of first diagnostic step;                   0
                               If nstartdia > nstop, no diagnostics will be
                               performed.
                               Alternatively:
hstartdia         REAL         Same as nstartdia, but in hours                    0.
nincdia           INT          Interval (in time steps) between two calls of      0
                               the diagnostic calculations; if nincdia = 0,
                               no diagnostics will be performed.
                               Alternatively:
hincdia           REAL         Interval (in hours) between two calls of the      24.
                               diagnostic calculations; if hincdia = 0.,
                               no diagnostics will be performed.
nstartctl          INT         Number of first control output step;               0
                               If nstartctl > nstop, no control output will be
                               computed.
                               Alternatively:
hstartctl         REAL         Same as nstartctl, but in hours                    0.
nincctl            INT         Interval (in time steps) between two calls of      0
                               the control calculations; if nincctl = 0,
                               no control output will be computed.
                               Alternatively:
hincctl            INT     Interval (in hours) between two calls of the           1.
                           control calculations; if hincctl = 0.,
                           no control output will be computed.
lpr_id             LOGICAL Logical switch, if .true., check the initial data,    .true.
                           i.e. print some information like maximum and
                           minimum values of each input field
lpr_lb             LOGICAL Logical switch, if .true., check the lateral          .false.
                           boundary data, i.e. print some information like
                           maximum and minimum values of each lateral
                           boundary field
                                           - 37 -


/ met_ctl / - Control variables for the meteograph print-out

For each meteograph point to be printed, a separate NAMELIST / met_ctl / has to appear in the
file INPUT_HRM; the number of separate points is given in the variable ngpmet which has to be
given in the first NAMELIST / met_ctl /, and must not exceed 200.

 Parameter          Type               Meaning of Parameter                       Default
ngpmet            INT         Number of meteograph points                         0
nstartmet         INT         First time step for meteograph prints;              0
                              Alternatively:
hstartmet         REAL        Same as nstartmet, but in hours                     0.
nincmet           INT         Interval (in time steps) between two calls of       0
                              the meteograph print-out;
                              Alternatively:
hincmet           REAL        Same as nincmet, but in hours                       1.
m1gpmet           INT         j1-index of meteograph point                        0
m2gpmet           INT         j2-index of meteograph point;                       0
                              Alternatively:
zlongpmet         REAL        Longitude (in degree) of meteograph point           0.
zlatgpmet         REAL        Latitude (in degree) of meteograph point            0.
ynamegp           CH*24       Name of meteograph point                           ´     ´
ymettype          CH*5        Type of meteograph print-out; either ´short´ or    ´short´
                              ´long´; the ´short´ type prints one row for each
                              time step to be printed, the ´long´ one more
                              than one page of information.
The ASCII Meteograph files are named MET_ynamegp where ynamegp is the name of the me-
teograph point. A PERL script is available (see section 8) to visualise the content of the me-
teograph files.
                                            - 38 -



/ gribin / - Control variables for the initial data (analysis) and the lateral boundary data

 Parameter          Type                Meaning of Parameter                          Default
ninc_wait         INT          Interval, in seconds, for hrm to perform a              0
                               check at the ´ready´ files (for boundary condi-
                               tion), in the directory especified in ytrans_in.
nmax_wait         INT          Maximum time, in seconds, for hrm to wait for           0.
                               the ´ready´ file, in the directory especified in
                               ytrans_in, before stopping the run.
yanadir           CH*80        Directory path of initial data file                ´         ´
ylbdir            CH*80        Directory path of lateral boundary data files      ´         ´
yfgdir            CH*80        Directory path of first guess (FG) data files      ´         ´
ylbfgdir          CH*80        Directory path of lateral boundary data used       ´         ´
                               for first guess (FG) data files
yhead_fg          CH*3         Header of FG file name                              ‗hfff‗
nversana          INT          Version number of initial data (analysis)            1
nverslb           INT          Version number of lateral boundary data              1


/ gribout / - Control variables and switches for the HRM post-processing

This NAMELIST group may appear several times to define the writing of GRIB1 data. Data may
be written out on the full model domain as well as on a sub-domain. Additionally, multi-level
fields may be interpolated from model to pressure levels. Finally, some derived quantities like
mean sea level pressure and relative humidity can be computed.

 Parameter          Type                Meaning of Parameter                          Default
ngrib             INT     Post-processing times in time steps; at most            -1
                          max_nl_steps = 300 may be defined here.
                          Alternatively:
hgrib             REAL    Same as ngrib, but in hours                              -1.
ncomb             INT     Post-processing times in time steps as triplets:         -1,-1,-1
                          start, end, increment; at most max_nl_comb =
                          10 triplets may be defined here.
                          Alternatively:
hcomb             REAL    Same as ncomb, but in hours                               -1.,-1.,-1.
lpp_ini           LOGICAL Logical switch, if.true. , write out the initia-        .false.
                          lized analysis or first guess fields
ysystem           CH*4    Type of data storage:                                   ´file´
                          ´daba´: ORACLE data base (not yet impl.)
                          ´file´ : file in directory ydir (see below)
ydir              CH*100  Directory of data base or file                          ´ ´
ydabapw           CH*10   Password of data base                                   ´ ´
ykind             CH*1    Kind of result GRIB1 files:                             ´f´
                          ´f´: forecast data
                          ´r ´: restart data (2 files: t, t+t)
yarea             CH*1    Area coding (for file name):                            ´f´
                          ´f´: full model domain;
                          any other letter: sub-domain
                                     - 39 -


 Parameter     Type               Meaning of Parameter                       Default
ytunit       CH*1    Time unit indicator (for file name):                ´f´
                     ´t´: time step in file name
                     ´f´: forecast mode (hstop < 99d)
                     ´c´: climate mode (hstop > 99d)
yext         CH*1    Extension parameter of file name;                   ´ ´
                     ´p´: Data interpolated to pressure levels
yvarml       CH*9    Name of model level fields (see Tab. 7.1, 7.2,      ´         ´
                     7.3, 7.5, 7.6); at most max_nl_varml = 150
                     different names may be given.
yvarpl       CH*9    Name of pressure level fields (see Tab. 7.4 and     ´             ´
                     7.7); at most max_nl_varpl = 50 different
                     names may be given.
plev         REAL    Pressure levels in Pa; at most                           0.
                     max_nl_plev = 50 may be given.
ncenter      INT     Centre identifer (WMO) in the PDS                       78
i1_start     INT     start index of (sub-) domain in j1-direction              1
i2_start     INT     start index of (sub-) domain in j2-direction              1
i1_end       INT     end index of (sub-) domain in j1-direction               ie
i2_end       INT     end index of (sub-) domain in j2-direction               je
nrbit        INT     Number of bits per value for GRIB packing;              16
                     usually 16 bits are enough, but for restart files
                     nrbit is set to 32 to increase accuracy.
nvers        INT     Version number of model run                            1
lfilter      LOGICAL Logical switch, if.true. , filter the fields on     .true.
                     pressure levels and mean sea level pressure to
                     remove small-scale noise
                                            - 40 -


3. Using the post-processing functions
To write out the initialized analysis or first guess, a separate NAMELIST group / gribout / is
needed where lpp_ini = .true.
During post-processing the HRM writes out (normally to disk) GRIB1 code fields on model le-
vels and interpolated to pressure levels, for the full model domain (and/or for sub-domains).
The post-processing is controlled by the NAMELIST parameters of group / gribout /.
If no pressure levels have been specified in / gribout / the HRM uses as a default 12 standard
pressure levels, namely 50, 100, 150, 200, 250, 300, 400, 500, 700, 850, 950, 1000 hPa. The ver-
tical interpolation from model to pressure levels is based on tension splines except for relative
humidity which is interpolated linearly.
The following derived quantities
 RELHUM (relative humidity),
 TD (dew point temperature),
 TQV (total water vapour content; precipitable water),
 TQC (total cloud water content, integral over column),
 TQI (total cloud ice content),
 TO3 (total ozone content),
 TOT_PREC (total precipitation, i. e. rain + snow, grid scale + convective),
 CEILING (the lowest significant cloud layer) and
 PSMSL (mean sea level pressure (yvarpl =´PS´))
 CAPE_MU (CAPE of most unstable parcel)
 CIN_MU (CIN of most unstable parcel)
 CAPE_ML (CAPE of mean surface layer parcel)
 CIN_ML (CIN of mean surface layer parcel)
 SHOW_IN (Showalter index; convection index)
 SUR_LI_IN (surface lifted index; convection index)
may be computed, too.
If GrADS is used for visualization, variables which are in different code tables but have the
same element number, like TOP_CON (tab=201, ee=73) and CLCL (tab=2, ee=73), have to be in
separate files because GrADS can handle only one code table. The NAMELIST variable yarea
(/ gribout /) may be used for this purpose; e. g. CLCL can be written in a file with yarea=´f´,
whereas TOP_CON is written in a file with yarea=´1´.
ncenter may be used to write in the PDS (product definition section) of the GRIB fields the
WMO centre identification number (e.g. 78 for the DWD).
                                             - 41 -


4. File name conventions
The file naming convention is closely related to the directory structure of the models since im-
portant information not contained in the file names has to be given in the names of the directo-
ries.

4.1 Naming convention for directories
The directory names should provide the following information

*        experiment name (or number), e.g. exp_13421
*        type of model run, e.g. assimilation, early run, main run
*        initial date of the forecast, e.g. in the form yyyymmddgg where
         yyyy: year, mm: month, dd: day, gg: time.

4.2 General form of file names of the HRM
Analysis files
The analysis file name has the following general form

yhead//yyyymmddgg//yext

where

yhead (3 Character): File-Header, e.g.

haf:    HRM analysis, full model domain


yyyymmddgg (10 Character): date in the form
yyyy: year
mm: month
dd:   day
gg:   time

yext (1 Character): Extension, e.g.
p:     data interpolated from model to pressure levels

Example:
haf1997091612:        HRM analysis valid at 12 UTC on 16th of September 1997

Lateral boundary data files
For the lateral boundary data of the HRM, two options exist.
If lana_lb = .true. (in NAMELIST / hrm_ctl /), the HRM is driven by analyses, i.e. the lateral
boundary data files have the form described above.
If lana_lb = .false., the HRM is driven by forecasts of another model, e.g. the GME of the DWD.
Then the lateral boundary data file name has the following general form

yhead//ytunit//'xxxxxxxx'//yext

where
                                              - 42 -




yhead (3 Character): File-Header,.
hbf: HRM boundary data, full model domain
ytunit (1 Character): Time unit of forecast range, e.g.
t:     forecast range given in time steps
f:     forecast mode: the forecast range is given in the form
       ddhhmmss where
       dd: day, hh: hour, mm: minute, ss: second
c:     climate mode: the forecast range is given in the form
       yyydddhh where
       yyy: year, ddd: Julian day, hh: hour

xxxxxxxx (8 Character): Forecast range in a form depending on ytunit.

yext (1 Character): Extension, e.g.
p:     data interpolated from model to pressure levels

Example:
hbff01210000:          HRM boundary data at day 1, 21 hours, i.e. a 45-h boundary data

Forecast files

The forecast file name has the following general form

yhead//ytunit//'xxxxxxxx'//yext

where

yhead (3 Character): File-Header, e.g.
hff:    HRM forecast, full model domain

ytunit (1 Character): Time unit of forecast range, e.g.
t:       forecast range given in time steps
f:       forecast mode: the forecast range is given in the form
         ddhhmmss where
         dd: day, hh: hour, mm: minute, ss: second
c:       climate mode: the forecast range is given in the form
         yyydddhh where
         yyy: year, ddd: Julian day, hh: hour

xxxxxxxx (8 Character): Forecast range in a form depending on ytunit.

yext (1 Character): Extension, e.g.
p:       data interpolated from model to pressure levels

Examples:
hfff01180000: HRM forecast at day 1, 18 hours, i.e. a 42-h forecast
hffc11223312p: HRM forecast, climate mode, year 112, day 233, hour 12, interpolated from mod-
               el to pressure levels
                                                - 43 -


Valid file headers (yhead)

haf:     HRM analysis (uninitialized), full model domain
hif:     HRM analysis (initialized), full model domain
hbf:     HRM boundary data, full model domain
hff:     HRM forecast, full model domain
hrf:     HRM restart, full model domain

Notes:
The initialized analysis is identical to the 0-h forecast of the corresponding model, e.g. with the file
name hiff00000000; to facilitate a separate storage of initialized analysis files for later use, e.g. for
diagnostic studies, the initialized analysis may be renamed to the analysis file form, i.e.
hifyyyymmddhh where hif is the header of the initialized analysis of the full model domain.

Sub-domains are denoted by a 3rd letter other than "f" which denotes the full domain.

The 3rd letter (called yarea in NAMELIST / gribout / ) may be also used to group a special set of
variables in one GRIB file. E. g. the input data for the interpolation program HMX2HMY (see
Section 11) are written in a file with yarea =’h’.
                                             - 44 -


5. Multitasking of the HRM
5.1 Introduction
The HRM code is prepared for the use of more than one processor (PE: processing element) of
shared memory computers like SGI Altix and distributed memory computers like a IBM p675, or
Linux PC clusters.
The multitasking approach chosen is mainly based on a 2D domain decomposition where the
horizontal model domain (i. e. ie x je grid points) is distributed as evenly as possible to all PEs
assigned to the HRM (Fig. 7). Up to 256 PEs may be used in parallel.

(1, je)                                                                                 (ie, je)


                                                                               Task nproc =
                                                                              nproc1 x nproc2


                                            Task jt
                                       with halo of two
                                        rows/columns


     Task 1                                                                     Task nproc1



(1, 1)                                                                                   (ie, 1)

Fig. 7 2D domain decomposition of the HRM with nproc = 5 x 3 tasks (PEs).

The number of PEs used in parallel is controlled by the Namelist variables nproc1 and nproc2 in
/ hrm_ctl /. nproc1 is the number of tasks (threads) in west – east direction, nproc2 is the number
of tasks in south – north direction.
The 2D domain decomposition splits the computational work between nproc1 x nproc2 tasks.
Each processor/task (jt) computes the forecast only for a 2D sub-domain defined by (mi1sc,
mi1ec) in west – east and (mi2sc, mi2ec) in south – north direction. These indices are computed
from nproc1 and nproc2 in the subroutine readnlst and stored in the INTEGER arrays:
mj1_start (jt) for mi1sc,      mj1_end (jt) for mi1ec,
mj2_start (jt) for mi2sc,      mj2_end (jt) for mi2ec.

The indices of the corners of the full model domain (_f) are defined as:
ig1s_f = 1, ig1e_f = ie, ig2s_f = 1 and ig2e_f = je.

To make best use of the inherent parallelism of the HRM code, the number of columns minus 2
(i. e. ie – 2) should be a multiple of nproc1, and the number of rows minus 2 (i. e. je - 2) should
be a multiple of nproc2. Additionally, cache effects like pre-fetching of data from memory into
the cache and cache re-use will have an important impact on the actual performance of the HRM
on a parallel computer system, too.
Thus different combinations of nproc1 and nproc2 should be tested before deciding on the op-
timal operational configuration.
                                             - 45 -


The Helmholtz equations related to the semi-implicit time stepping scheme are solved by a fast
direct method based on an FFT in west – east direction and Gaussian elimination in south – north
direction. The solution of the 2D Helmholtz equations is performed in parallel, and the nsi modes
of the split semi-implicit scheme are mapped onto the nproc2 processors via mj3_start and
mj3_end.


5.2 OpenMP-Version of the HRM for shared memory computers
The HRM code includes standard OpenMP directives in the main control routine progorg. Here
the compiler is told to call the large parallel regions (DO loops with a CALL to the subroutines
prog_s_01, prog_s_02, …, prog_s_08 and physics) in parallel for jt = 1, nproc1 x nproc2 threads
because there is no side effect. Additionally, the subroutines pp_verint and allocate_fields are
parallelized, too. Because of the shared memory, all threads have access to the data of the full
model domain, and the compiler automatically takes care of the necessary synchronization after
each parallel region.
At run-time the actual number of threads (processors) is defined by the Namelist variables
nproc1 and nproc2 in / hrm_ctl / and the total number of threads used nproc = nproc1 x nproc2
has to be defined in the environment variable OMP_NUM_THREADS
(e. g. export OMP_NUM_THREADS=15 for nproc1=3 and nproc2=5).

Table 5.1 shows the speed-up of the HRM (with 217 x 193 grid point, 40 layers, 14 km grid
spacing, time step 90 s, 24-h forecast) on one shared memory node of an IBM p575 with 8
Power5 processors per node. nproc1 was set to 1.

Table 5.1       Speed-up of HRM on one node of an IBM p575 for a 24-h forecast

nproc2              1        2         3          4        5         6          7         8
Wallclock (s)     7808     3963      2620       1983     1624      1387       1235      1099
Speed-up of       1.00     1.97      2.98       3.94     4.81      5.63       6.32      7.10
full model
Speed-up of       1.00     1.96      2.90       3.85      4.76      5.67      6.54      7.45
physics
Speed-up of       1.00     1.97      2.98       3.96      4.86      5.78      6.57      7.69
dynamics

For this model domain (217 x 193 grid points, 40 layers), running HRM on up to eight proces-
sors yields a good speed-up because je-2 is close to a multiple of nproc2.


5.3 MPI-Version of the HRM for distributed memory computers
The MPI-Version of the HRM which can use distributed memory computers like Linux PC clus-
ters has been developed by Dr. Pham Hong Quang (Vietnam, e-mail: quang@cadprovn.com) and
integrated into the HRM code by D. Majewski (DWD). In February 2006 the MPI communica-
tion was optimised by T. C. Babu (India).

Only processor 0 (myid = 0) is holding all columns (west – east: from 1 to ie) and rows (south to
north: from 1 to je), all the other processors (from 1 to nproc-1) hold only several columns (from
ig1s to ig1e) and rows (from ig2s to ig2e, which have different values in each processor!). ig1s,
ig1e, ig2s and ig2e are defined including a ―halo‖ of at most two columns and rows around the
computation domain in the subroutine readnlst.
                                                - 46 -


The start and end indices of the computation domain are stored in mj1_start, mj1_end, mj2_start
and mj2_end, and computed depending on the total number of columns ie and rows je and the
number of processors in west –east direction (nproc1) and in south – north direction (nproc2) in
readnlst. The total domain boundaries are stored in the variables ig1s_f ( = 1), ig1e_f ( = ie) and
ig2s_f ( = 1), ig2e_f ( = je). For the semi-implicit time stepping, the solution of the 2D-
Helmholtz equations is also done in parallel and the nsi modes are mapped onto the nproc1 x
nproc2 processors via mj3_start and mj3_end.

Several communication routines take care of the necessary exchange of data between the proces-
sors, namely:
recv_full:              called by progorg; it sends all prognostic/diagnostic fields after section 1
of progorg (i.e. after initialization) for the start time step from processor 0 to all the other ones. It
is also called by df_ini before the backward and forward stepping.

recv_lb:               called by hrmorg; it sends all lateral boundary fields from processor 0 to
all the other ones.

send_recv_physics: called by progorg; it exchanges all physics-related fields (which are used
in progexp with a halo of one column/row) between the processors.

send_recv_prog_03: called by progorg; it exchanges the prognostic fields (ps, v, t) computed in
progexp and needed by the semi-implicit scheme. two columns/rows for ps and v, one col-
umn/row for t.

send_recv_prog_04: called by progorg; it exchanges the fields computed in sicor1 (rhs_si,
exp_si) and needed by the semi-implicit scheme.

send_recv_prog_05: called by progorg; it exchanges the field computed in sicor2 (dttdiv) and
needed by the semi-implicit scheme.

send_recv_prog_08: called by progorg; it exchanges some prognostic/diagnostic fields (ps, u, v,
t, qv, qc, qi, o3, fih, fif) after the computation of Asselin filtering, condensation and geopotential;
2 rows, time levels nt3p and nt3c.

send_full:              called by progorg; it sends all prognostic/diagnostic fields from processors
1 to nproc-1 to processor 0 every ninc_s_f time steps. This is necessary for output (like diagnos-
tics and meteographs) and post-processing. All the time levels (nt3p, nt3c and nt3m, nt2m and
nt2c) are being sent. It is also called by df_ini after the calculation of the filtered fields.

To run the MPI-Version of the HRM the variable lmpi must be .true. in Namelist /hrm_ctl/
and the hrm binary must be linked with the native MPI library and compiled with the na-
tive mpif.h file of the computer!

Of course, the efficiency of the MPI-Version of the HRM depends to a large extent on the com-
munication speed (latency, transfer rates) of the network!
                                               - 47 -


6. Computer resources needed by the HRM
6.1 Number of grid points per layer
Without recompilation, the HRM may use different numbers of grid points per layer, determined
by the NAMELIST parameters ie and je in / grid_ctl /. Because an FFT-solver is used in the
west - east direction, ie-1 must be a power of 2, 3, and/or 5, i.e.
       ie – 1 = 2l x 3m x 5n
where l, m, n are positive integers (l > 0);
e.g. ie = 101, 109, 121, 129, 145, 151, 161, 163, 181, 193, 201, 217, 241, 251, 271, 289, 301,
321, 325, 361, 385, 401, 433, 451, 481, 487, 501, 541.

If the radiative upper boundary condition is used, i.e. for rubc = .true. in NAMELIST
/ dyn_ctl /, je-1 must be a power of 2, 3, and/or 5, too.

The GRIB1 code cannot encode mesh sizes like 0.0625° properly because it packs such informa-
tion in 1/1000 °, only. Therefore, the mesh sizes (, ) are derived in the HRM from ie and je
and the coordinates of the lower left corner and upper right corner of the model domain in the
subroutine readnlst.


6.2 Vertical resolution
The vertical resolution of the HRM is determined by the NAMELIST parameter i3e in group
/ grid_ctl /.


6.3 Total memory requirements
The permanent memory space required by the HRM is allocated at the beginning of the forecast
in subroutine allocate_fields depending on the horizontal resolution parameters ie, je and num-
ber of layers i3e. Here, all prognostic, diagnostic and constant fields are allocated as 2-, 3-, or 4-
dimensional arrays. Additionally, local memory space is needed for dynamic arrays in many sub-
routines, especially in physics, fefst, and progexp.

All in all, about
                      75 x ie x je x i3e + 120 x ie x je + 6 x nplev x ie x je
words are allocated.
nplev is the number of pressure levels in the post-processing. The HRM code itself needs about 8
MByte.

The HRM can be used with 32- or 64-Bit precision, i.e. with 4 or 8 Bytes per value.

Table 6.1      HWM (High Water Mark in MByte) of memory for the HRM at different
               horizontal resolutions ie x je and the fixed number of layers i3e = 25, and
               for 32-Bit precision.

 ie x je      51 x 51     73 x 73    101 x 101 121 x 121 145 x 145 151 x 151 181 x 181
 HWM            21          44          84        121       174       189       271
(MByte)
                                               - 48 -


6.4 Total CPU requirement of the HRM
Computing a full radiation step every forecast hour, an HRM time step including explicit dynam-
ics, semi-implicit correction and all physical parameterizations costs about

       3100 Flop/ (grid point, layer, time step).

The semi-implicit time step t depends on the horizontal mesh size (, ) according to Tab.
6.2.

Table 6.2      Semi-implicit time step t (s) depending on the horizontal mesh size (, ).

(, )      1.00°        0.75°       0.50°            0.25°     0.125°       0.10°     0.05°
  t (s)       600          450         300              150        75          60        30


Thus the total cost (CP(24-h)) of a 24-h forecast is given by

       CP(24-h) = 3100 * ie * je * i3e * 24 * 3600 /  t        (Flop).

Tab. 6.3 shows the total cost of a 24-h forecast at different mesh sizes and number of grid points;
the number of layers i3e is set to 40.


Table 6.3      Total cost (CP(24-h) ) of a 24-h forecast depending on the horizontal mesh size
               and the number of grid points; the number of layers i3e is set to 40.

, ie x je   0.50°, 101 x 101      0.25°, 101 x 101         0.25°, 151 x 151   0.125°, 121 x 121
CP(24-h)            0.37                 0.74                     1.63               2.10
(TFlop)


For operational applications at DWD, a 24-h forecast should be completed in less than 30 mi-
nutes (including I/O of GRIB fields). Thus the sustained speed of the computer on which HRM
is running operationally must exceed CP(24-h) / 1800s (Flops = Flop/s).


Table 6.4      Sustained speed (GFlops) required to run a 24-h forecast operationally
               (in less than 30 minutes) depending on the horizontal mesh size and the
               number of grid points; the number of layers i3e is set to 40.

, ie x je   0.50°, 101 x 101      0.25°, 101 x 101         0.25°, 151 x 151   0.125°, 121 x 121
 Speed)            0.21                  0.40                     0.91               1.17
(GFlops)
                                            - 49 -


6.5 Timing examples
OpenMP Version of the HRM for shared memory computers
The following timing example has been taken from a 78-h forecast of HRM with 145x137 grid
points, 40 layers, 0.25° (~ 28 km) mesh size and a time step of 150 s with nproc1 = 1 and
nproc2 = 8, i.e. on 8 CPUs (one node of the IBM p575 with 8 Power5 processors). The post-
processing of GRIB forecast files has been performed every three hours.
   Timing of HRM with    8 threads on IBM p575 (Power5)
   ie: 145 je: 137 i3e: 40 dx: 28 km dt: 150 s
   Number of time steps:        1872

   Time for start up of HRM:                            0.95   s
   Time for reading the l.b.c. data:                    6.08   s
   Time for copying the l.b.c. data:                   66.20   s
   Time for the diabatic processes:                   634.69   s
   Time for the explicit forecast:                    280.69   s
   Time for r.h.s. of Helmholtz eq.:                   29.53   s
   Time for solution of Helmholtz eq.:                 18.40   s
   Time for addition of SI corrections:                52.18   s
   Time for Asselin filtering:                         77.46   s
   Time for geopotential calculation:                  67.28   s
   Time for diagnostics/meteographs:                   28.61   s
   Time for post-processing GRIB files:                65.10   s
   Total wallclock time for HRM run:                 1327.17   s
  Forecast completed for + 078 h

Another timing example for the same domain but a grid spacing of 0.125° (~ 14 km). The model
runs almost 6.2 times longer (8168.64 s vs. 1327.17 s) than the 28-km version because there are
four times more grid points and the time step has to be reduced from 150 s to 75 s because of the
CFL stability criterion.
   Timing of HRM with    8 threads on IBM p575 (Power5)
   ie: 289 je: 273 i3e: 40 dx: 14 km dt: 75 s
   Number of time steps:        3744

   Time for start up of HRM:                            3.24   s
   Time for reading the l.b.c. data:                   13.92   s
   Time for copying the l.b.c. data:                  460.12   s
   Time for the diabatic processes:                  3589.94   s
   Time for the explicit forecast:                   2138.19   s
   Time for r.h.s. of Helmholtz eq.:                  143.64   s
   Time for solution of Helmholtz eq.:                 71.63   s
   Time for addition of SI corrections:               396.22   s
   Time for Asselin filtering:                        547.56   s
   Time for geopotential calculation:                 418.30   s
   Time for diagnostics/meteographs:                  136.05   s
   Time for post-processing GRIB files:               249.84   s
   Total wallclock time for HRM run:                 8168.64   s
  Forecast completed for + 078 h
                                         - 50 -


MPI Version of the HRM for distributed memory computers
The following timing example has been taken from a 24-h forecast of HRM with 217x193 grid
points, 40 layers, 0.125° (~ 14 km) mesh size and a time step of 90 s with nproc1 = 1 and
nproc2 = 8, or nproc2 = 16 on one/two node(s) of the IBM p575 with 8 Power5 processors per
node. The post-processing of GRIB forecast files has been performed every three hours.

Timing of HRM with 1x8 = 8 tasks
   ie: 217 je: 193 i3e:        40   14 km grid spacing      90 s time step
   Number of time steps:             960

   Time for start up of HRM:                         3.50   s
   Time for communication of start data:             0.41   s
   Time for reading the l.b.c. data:                 3.63   s
   Time for communication of lbc data:               0.56   s
   Time for copying the l.b.c. data:                40.46   s
   Time for the diabatic processes:                542.19   s
   Time for communication of diab. proc:             0.33   s
   Time for the explicit forecast:                 255.14   s
   Time for communication of expl. for.:             0.46   s
   Time for r.h.s. of Helmholtz eq.:                11.60   s
   Time for communication of r.h.s. Hel:             0.57   s
   Time for solution of Helmholtz eq.:               2.82   s
   Time for communication of solut. Hel:             0.50   s
   Time for addition of SI corrections:             59.83   s
   Time for Asselin filtering:                      63.18   s
   Time for geopotential calculation:               55.58   s
   Time for communication of cond/evap:              3.22   s
   Time for diagnostics/meteographs:                17.91   s
   Time for post-processing GRIB files:             47.85   s
   Time for communication of post-proc.:             4.89   s
   Total communication time for HRM run:            10.94   s
   Total wallclock time for HRM run:              1114.63   s
  Forecast completed for + 024 h

Timing of HRM with 1x16 = 16 tasks
   Time for start up of HRM:                        3.14    s
   Time for communication of start data:            0.59    s
   Time for reading the l.b.c. data:                4.40    s
   Time for communication of lbc data:              0.51    s
   Time for copying the l.b.c. data:               20.13    s
   Time for the diabatic processes:               275.03    s
   Time for communication of diab. proc:            0.37    s
   Time for the explicit forecast:                130.22    s
   Time for communication of expl. for.:            0.45    s
   Time for r.h.s. of Helmholtz eq.:                5.15    s
   Time for communication of r.h.s. Hel:            0.74    s
   Time for solution of Helmholtz eq.:              2.88    s
   Time for communication of solut. Hel:            0.37    s
   Time for addition of SI corrections:            30.77    s
   Time for Asselin filtering:                     29.53    s
   Time for geopotential calculation:              28.93    s
   Time for communication of cond/evap:             3.06    s
   Time for diagnostics/meteographs:               18.39    s
   Time for post-processing GRIB files:            55.62    s
   Time for communication of post-proc.:            6.93    s
   Total communication time for HRM run:           13.02    s
   Total wallclock time for HRM run:              617.21    s
Forecast completed for + 024 h
                                             - 51 -


Some timing of the HRM (MPI-Version) on a Linux PC Cluster with
10 CPUs (Xeon, 3.06 Ghz) and Myrinet Interconnect;
Timing of HRM 0.15°x0.15°, t=90s with nproc1=1 and nproc2=10
   ie: 201 je: 201 i3e: 31
   Number of time steps:       1920
   Time for start up of HRM:                  4.05 s
   Time for reading the l.b.c. data:         10.09 s
   Time for copying the l.b.c. data:        129.05 s
   Time for the diabatic processes:         377.62 s
   Time for the explicit forecast:          307.63 s
   Time for r.h.s. of Helmholtz eq.:        364.14 s
   Time for solution of Helmholtz eq.:       24.18 s
   Time for addition of SI corrections:      70.41 s
   Time for Asselin filtering:               70.36 s
   Time for geopotential calculation:        51.10 s
   Time for diagnostics/meteographs:         15.52 s
   Time for post-processing GRIB files:     372.57 s
   Total communication time for HRM run:    367.49 s
   Total wallclock time for HRM run:       2164.21 s
  Forecast completed for + 048 h

And finally a timing of HRM 0.11°x0.11°, t=60s with nproc1=4 and nproc2=20 on a Linux PC
Cluster with 16 Dual Quadcore processors and Infini-Band Interconnect; hourly GRIB output.
  ie: 601 je: 401 i3e: 40
   Number of time steps:       4320
   Time for start up of HRM:                            57.63   s
   Time for communication of start data:                 6.09   s
   Time for reading the l.b.c. data:                    82.09   s
   Time for communication of lbc data:                  13.16   s
   Time for copying the l.b.c. data:                    85.98   s
   Time for the diabatic processes:                    361.31   s
   Time for communication of diab. proc:                 5.87   s
   Time for the explicit forecast:                     581.25   s
   Time for communication of expl. for.:                 8.25   s
   Time for r.h.s. of Helmholtz eq.:                    92.09   s
   Time for communication of r.h.s. Hel:               104.14   s
   Time for solution of Helmholtz eq.:                 139.12   s
   Time for communication of solut. Hel:                88.36   s
   Time for addition of SI corrections:                 84.37   s
   Time for Asselin filtering:                         191.56   s
   Time for geo potential calculation:                  42.46   s
   Time for communication of cond/evap:                 64.77   s
   Time for diagnostics/meteographs:                    55.59   s
   Time for post-processing GRIB files:               1066.10   s
   Time for communication of post-proc.:               207.09   s
   Total communication time for HRM run:               497.72   s
   Total wallclock time for HRM run:                  3337.26   s
  Forecast completed for + 072 h

Some of the tasks in the timing examples above are performed on one CPU only, namely
 Start up of HRM,
 Reading the l.b.c. data,
 Diagnostics and meteographs,
 Post-processing of GRIB files (except the smoothing of fields on pressure levels which is
    done in parallel).
All the other tasks are performed in parallel, i.e. the wallclock time for them will reduce if more
CPUs (see Section 5) are being used.
                                                   - 52 -


7. Overview of the GRIB1 variables
7.1 Introduction
The GRIdded Binary Code (GRIB, version1) is the standard input and output format of HRM. If
a field is defined in the standard WMO table (tab=2), HRM uses this definition. Additionally, the
national tables (tab=201, 202 and 203) are used. Thus a GRIB field is identified by an element
number (ee) and a table number (tab).

7.2 Tables of the GRIB variables
The following Tables 7.1 to 7.7 list the name, element-nr (ee), table-nr (tab), level-typ (lvtyp),
level-top (lvt), level-bottom (lv) and physical unit of all GRIB fields known to HRM.

                            Table 7.1:        Constant fields (d=00000000)

                                                                                      HRM (GRIB 1)




                                                                                                            Physical Unit
                                                            Element-Nr.


                                                                          Table-Nr.


                                                                                       (LVTYP)
                                                                                        Lev-Typ
       Name                    Element




                                                                                                  Levtop

                                                                                                  Levbot
                                                                                                  (LVT)

                                                                                                   (LV)
                                                               (EE)




 FIS             orography * g                                   6        WMO             1        -   -   m2/s2

 Z0              roughness length (only land points!)          83         WMO             1        -   -       m

 FR_LAND         land fraction of surface                      81         WMO             1        -   -         1
                 soil type of surface (key from 1 to
                 10);
                 1: ice, 2: rock, 3: sand, 4: sandy
 SOILTYP                                                       57         202             1        -   -         1
                 loam,
                 5: loam, 6: clay and loam, 7: clay,
                 8: peat, 9: sea water, 10: sea ice
 RLAT            geographical latitude                        114         202             1        -   -   Deg. N

 RLON            geographical longitude                       115         202             1        -   -   Deg. E
                 root depth of vegetation (vegetation
 ROOT                                                          62         202             1        -   -       m
                 period)
                 plant cover of surface (vegetation
 PLCOV_V                                                       67         202             1        -   -         1
                 period)
                 ground fraction covered by ever-
 FOR_E                                                         75         202             1        -   -         1
                 green forest
                 ground fraction covered by decidu-
 FOR_D                                                         76         202             1        -   -         1
                 ous forest
 LAI             leaf area index                               61         202             1        -   -         1

 LAI_MX          leaf area index (vegetation period)           69         202             1        -   -         1
                 standard deviation of subgrid scale
 SSO_STDH                                                      46         202             1        -   -       m
                 orography
                 anisotropy of subgrid scale orogra-
 SSO_GAMMA                                                     47         202             1        -   -         1
                 phy
                                                    - 53 -



                                                                                         HRM (GRIB 1)




                                                                                                              Physical Unit
                                                             Element-Nr.


                                                                            Table-Nr.


                                                                                          (LVTYP)
                                                                                           Lev-Typ
      Name                       Element




                                                                                                     Levtop

                                                                                                     Levbot
                                                                                                     (LVT)

                                                                                                      (LV)
                                                                (EE)
                  angle between principal axis of
SSO_THETA         subgrid scale orography and global            48          202              1        -   -   rad
                  east
                  mean slope of subgrid scale orogra-
SSO_SIGMA                                                       49          202              1        -   -        1
                  phy




                      Table 7.2:          Multi-level fields of uninitialised analyses

                                                                                         HRM (GRIB 1)




                                                                                                                Physical Unit
                                                              Element-Nr.


                                                                             Table-Nr.

                                                                                          (LVTYP)
                                                                                           Lev-Typ
     Name                    Element




                                                                                                     Levtop

                                                                                                     Levbot
                                                                                                     (LVT)

                                                                                                      (LV)
                                                                 (EE)




U            zonal wind                                          33         WMO             110       -   -    m/s

V            meridional wind                                     34         WMO             110       -   -    m/s

FI           geopotential                                          6        WMO             109       -   -   m2/s2

T            temperature                                         11         WMO             110       -   -         K

QV           specific humidity                                   51         WMO             110       -   -   kg/kg

QC           specific cloud liquid water content                 31         201             110       -   -   kg/kg

QI           specific cloud ice content                          33         201             110       -   -   kg/kg

O3           ozone mixing ratio (optional)                      180         202             110       -   -   kg/kg
                                                    - 54 -


                   Table 7.3:        Single-level fields of uninitialised analyses

                                                                                       HRM (GRIB 1)




                                                                                                            Physical Unit
                                                             Element-Nr.

                                                                           Table-Nr.

                                                                                       (LVTYP)
                                                                                        Lev-Typ
     Name                   Element




                                                                                                  Levtop

                                                                                                  Levbot
                                                                                                  (LVT)

                                                                                                   (LV)
                                                                (EE)
PS          surface pressure on model orography                   1        WMO            1        -   -     Pa
            temperature at the top of snow or sur-
T_SNOW                                                         203         201            1        -   -       K
            face temperature
            temperature at the bottom of snow or
T_S                                                             85         WMO          111        -   0       K
            surface temperature (if no snow)
T_G         surface temperature (weighted from T_S              11         WMO            1        -   -       K
            and T_SNOW); if no snow: T_G = T_S
            = T_SNOW. Over water, T_G is kept
            constant during the model run.
            If T_G < - 1.8°C over water, sea ice is
            assumed.
QV_S        specific humidity at the surface; over              51         WMO            1        -   -   kg/kg
            water, this corresponds to 100% relative
            humidity.
RHO_SNOW    snow density                                       133         201            1        -   -   kg/m3

W_SNOW      water content of snow                               65         WMO            1        -   -   mmH2O
            weighting function indicating ‗fresh-
FRESHSNW    ness‘of snow in upper few centimeters              129         201            1        -   -         1
            of snow cover (for albedo)
W_I         water content of interception storage              200         201            1        -   -   mmH2O
            ozone content, vertically integrated
VMO3                                                            65         202            1        -   -   Pa(O3)
            (climatological)
            height of ozone maximum
HMO3                                                            64         202            1        -   -     Pa
            (climatological)
PCLOV       plant cover of surface (climatological)             87         WMO            1        -   -      %

LAI         Leaf area index (climatological)                    61         202            1        -   -         1

ROOT        root depth of plants (climatological)               62         202            1        -   -      M

FR_ICE      ice fraction for ocean or lake surfaces            91          WMO            1        -   -         1

H_ICE       sea ice thickness                                   92         WMO            1        -   -       m
            ice surface temperature or water surface
T_ICE                                                          215         201            1        -   -       K
            temperature
                                                       - 55 -


             Table 7.4:        Fields of the multi layer soil model of uninitialised analyses

                                                                                           HRM (GRIB 1)




                                                                                                                           Physical Unit
                                                                 Element-Nr.

                                                                               Table-Nr.

                                                                                           (LVTYP)
                                                                                            Lev-Typ
      Name                         Element




                                                                                                       Levtop


                                                                                                                 Levbot
                                                                                                       (LVT)


                                                                                                                  (LV)
                                                                    (EE)
    T_SO        soil temperature                                    197        201          111          -       depth        K
                water content of soil layer
    W_SO                                                            198        201          111          -         in     mmH O
                (liquid and frozen)
    W_SO_ICE    ice content of soil layer                           199        201          111          -        cm      mmH O


The soil temperatures are defined for the nine levels 0, 0.5, 2, 6, 18, 54, 162, 486 and 1458 cm in
soil. The temperature at the interface soil – atmosphere is identical to the temperature in the
depth of 0.5 cm. The temperature at level 1458 cm is set to the climate average of the 2m-
temperature.

WARNING: As the depth could only be coded in whole centimetres in GRIB1 Code, the
depth of 0.5 cm will be coded as 1 cm.
Water and ice content are computed for the six soil layers 0 – 1, 1 – 3, 3 – 9, 9 – 27, 27 – 81, 81
– 243; the layers 243 – 729 and 729 – 2187 are kept constant in time.


      Table 7.5:       Fields of uninitialised analyses interpolated to pressure levels or to MSL


                                                                                           HRM (GRIB 1)
                                                                                                                           Physical Unit
                                                                Element-Nr.

                                                                               Table-Nr.

                                                                                            (LVTYP)
                                                                                             Lev-Typ




     Name                      Element
                                                                                                       Levtop

                                                                                                       Levbot
                                                                                                       (LVT)

                                                                                                        (LV)
                                                                   (EE)




U              zonal wind                                          33          WMO            100            -    -        m/s

V              meridional wind                                     34          WMO            100            -    -        m/s

FI             geopotential                                          6         WMO            100            -    -        m2/s2

T              temperature                                         11          WMO            100            -    -            K

RELHUM         relative humidity                                   52          WMO            100            -    -            %

QI             cloud ice content                                   33          201            100            -    -       kg/kg

O3             ozone mixing ratio (optional)                      180          202            100            -    -       kg/kg
               surface pressure, reduced to mean sea
PS                                                                   2         WMO            102            -    -          Pa
               level
                                                    - 56 -


     Table 7.6:      Hybrid multi-level fields of forecasts (VV>0) and initialised analyses (VV=0)

                                                                                       HRM (GRIB 1)




                                                                                                            Physical Unit
                                                             Element-Nr.

                                                                           Table-Nr.

                                                                                       (LVTYP)
                                                                                        Lev-Typ
     Name                     Element




                                                                                                  Levtop

                                                                                                  Levbot
                                                                                                  (LVT)

                                                                                                   (LV)
                                                                (EE)
U             zonal wind                                         33        WMO          110        -   -   m/s

V             meridional wind                                    34        WMO          110        -   -   m/s

FI            geopotential                                        6        WMO          109        -   -   m2/s2

T             temperature                                        11        WMO          110        -   -       K

QV            specific humidity                                  51        WMO          110        -   -   kg/kg

QC            specific cloud liquid water content                31        201          110        -   -   kg/kg

QI            specific cloud ice content                         33        201          110        -   -   kg/kg

CLC           cloud cover                                        29        201          110        -   -       %

O3            ozone mixing ratio (optional)                    180         202          110        -   -   kg/kg

OMEGA         vertical velocity  = dp/dt                        39        WMO          110        -   -   Pa/s

TD            Dew point temperature                              17        WMO          110        -   -       K

        Table 7.7:      Single-level fields of forecasts (VV>0) and initialised analyses (VV=0)

                                                                                       HRM (GRIB 1)         Physical Unit
                                                             Element-Nr.


                                                                           Table-Nr.

                                                                                       (LVTYP)
                                                                                        Lev-Typ




     Name                     Element
                                                                                                  Levtop

                                                                                                  Levbot
                                                                                                  (LVT)

                                                                                                   (LV)
                                                                (EE)




PS            surface pressure on model orography                 1        WMO            1        -   -      Pa
              temperature at the top of snow or surface
T_SNOW                                                         203         201            1        -   -       K
              temperature
              temperature at the bottom of snow or
T_S                                                             85         WMO          111        -   0       K
              surface temperature (if no snow)
T_G           surface temperature (weighted from T_S            11         WMO            1        -   -       K
              and T_SNOW); if no snow: T_G = T_S =
              T_SNOW. Over water, T_G is kept con-
              stant during the model run.
QV_S          specific humidity at the surface; over            51         WMO            1        -   -   kg/kg
              water, this corresponds to 100% relative
              humidity.
RHO_SNOW snow density                                          133         201            1        -   -   kg/m3
                                                        - 57 -


 Continuation of Table 7.7(2):               Single-level fields of forecasts (VV>0) and initialised ana-
                                                 lyses (VV=0)

                                                                                           HRM (GRIB 1)




                                                                                                                Physical Unit
                                                                 Element-Nr.


                                                                               Table-Nr.

                                                                                           (LVTYP)
                                                                                            Lev-Typ
   Name                       Element




                                                                                                      Levtop

                                                                                                      Levbot
                                                                                                      (LVT)

                                                                                                       (LV)
                                                                    (EE)
W_SNOW        water content of snow                                 65         WMO            1        -   -   mmH2O
              weighting function indicating ‗fresh-
FRESHSNW      ness‘of snow in upper few centimeters of             129         201            1        -   -         1
              snow cover (for albedo)
H_SNOW        Schneehöhe (= W_SNOW/RHO_SNOW)                        66         WMO            1        -   -       m

W_I           water content of interception storage                200         201            1        -   -   mmH2O
              turbulent transfer coefficient for momen-
TCM                                                                170         201            1        -   -         1
              tum at the surface
              turbulent transfer coefficient for heat and
TCH                                                                171         201            1        -   -         1
              moisture at the surface
              solar radiation balance at the surface;
ASOB_S                                                             111         WMO            1        -   -   W/m2
              mean over forecast period
              thermal radiation balance at the surface;
ATHB_S                                                             112         WMO            1        -   -   W/m2
              mean over forecast period
              photosynthetic active radiation balance at
APAB_S                                                                5        201            1        -   -   W/m2
              the surface; mean over forecast period
ALB_RAD       (solar) shortwave albedo at the surface               84         WMO            1        -   -      %
         solar radiation balance at the top of the
ASOB_T                                                             113         WMO            8        -   -   W/m2
         atmosphere ; mean over forecast period
         thermal radiation balance at the top of the
ATHB_T                                                             114         WMO            8        -   -   W/m2
         atmosphere ; mean over forecast period
         rain     (grid-scale precipitation),
RAIN_GSP                                                           102         201            1        -   -   kg/m2
         accumulated since start of the forecast
         snow (grid-scale precipitation),
SNOW_GSP                                                            79         WMO            1        -   -   kg/m2
         accumulated since start of the forecast
         rain     (convective precipitation),
RAIN_CON                                                           113         201            1        -   -   kg/m2
         accumulated since start of the forecast
         snow (convective precipitation),
SNOW_CON                                                            78         WMO            1        -   -   kg/m2
         accumulated since start of the forecast
CAPE_MU       CAPE of most unstable parcel                         143         201            1        -   -    J/kg

CIN_MU        CIN of most unstable parcel                          144         201            1        -   -    J/kg

CAPE_ML       CAPE of mean surface layer parcel                    145         201            1        -   -    J/kg

CIN_ML        CIN of mean surface layer parcel                     146         201            1        -   -    J/kg

SHOW_IN       Showalter Index (Convection)                         149         203            1        -   -       K

SUR_LI_IN     surface lifted index (Convection)                    147         203            1        -   -       K
                                                       - 58 -


 Continuation of Table 7.7(3):              Single-level fields of forecasts (VV>0) and initialised ana-
                                                lyses (VV=0)

                                                                                          HRM (GRIB 1)




                                                                                                                  Physical Unit
                                                                Element-Nr.

                                                                              Table-Nr.

                                                                                          (LVTYP)
                                                                                           Lev-Typ
     Name                   Element




                                                                                                     Levtop

                                                                                                     Levbot
                                                                                                     (LVT)

                                                                                                      (LV)
                                                                   (EE)
            surface water run-off;                                 90         WMO          112        0   10     kg/m2
RUNOFF_S
            accumulated since start of the forecast.
            ground water run-off;                                  90         WMO          112       10   100    kg/m2
RUNOFF_G
            accumulated since start of the forecast.
U_10M       zonal wind          10m above ground                   33         WMO          105        -   10      m/s

V_10M       merdional wind 10m above ground                        34         WMO          105        -   10      m/s

T_2M        temperature         2m above ground                    11         WMO          105        -    2         K
            dew point temperature   2m above
TD_2M                                                              17         WMO          105        -    2         K
            ground
            minimum of temperature 2m above
TMIN_2M                                                            16         WMO          105        -    2         K
            ground
            maximum of temperature 2m above
TMAX_2M                                                            15         WMO          105        -    2         K
            ground
            maximum wind speed     10m above
VMAX_10M                                                          187         201          105        -   10      m/s
            ground
Z0          roughness length (land and water points)               83         WMO            1        -    -         m

CLCT        total cloud cover                                      71         WMO            1        -    -        %

CLCH        high cloud cover (0 – 400 hPa)                         75         WMO            1        -    -        %

CLCM        medium cloud cover (400 – 800 hPa)                     74         WMO            1        -    -        %

CLCL        low cloud cover (800 hPa – surface)                    73         WMO            1        -    -        %
BAS_CON     base index of main convective cloud
                                                                   72         201            1        -    -           1
            (index of vertical level)
            top index of main convective cloud
TOP_CON                                                            73         201            1        -    -           1
            (index of vertical level)
HBAS_CON    height of cloud base above msl                         68         201            2        -    -         m

HTOP_CON    height of cloud top above msl                          69         201            3        -    -         m

HTOP_DC     height of top of dry convection above msl              82         201            1        -    -         m
            average u-momentum flux at the surface;
AUMFL_S                                                           124         WMO            1        -    -    kg/(ms2)
            averaged since start of forecast
            average v-momentum flux at the surface;
AVMFL_S                                                           125         WMO            1        -    -    kg/(ms2)
            averaged since start of forecast
            average sensible heat flux at the surface;
ASHFL_S                                                           122         WMO            1        -    -     W/m2
            averaged since start of forecast
            average latent heat flux at the surface;
ALHFL_S                                                           121         WMO            1        -    -     W/m2
            averaged since start of forecast
                                                      - 59 -


  Continuation of Table 7.7(4):            Single-level fields of forecasts (VV>0) and initialised ana-
                                               lyses (VV=0)

                                                                                         HRM (GRIB 1)




                                                                                                                 Physical Unit
                                                               Element-Nr.

                                                                             Table-Nr.

                                                                                         (LVTYP)
                                                                                          Lev-Typ
   Name                      Element




                                                                                                    Levtop

                                                                                                    Levbot
                                                                                                    (LVT)

                                                                                                     (LV)
                                                                  (EE)
            modified cloud depth for media (e. g. TV)
CLDEPTH                                                          203         203            1        -    -            -
            representation of clouds
CLCT_MOD    modified cloud cover for media (e. g. TV)            204         203            1        -    -            -
            representation of clouds
HZEROCL     height of 0°C level above mean sea level               84        201            4        -    -         m

TQV         total water vapour content (integral over              54        WMO            1        -    -     kg/m2
            column)
TQC         total cloud water content (integral over               76        WMO            1        -    -     kg/m2
            column)
TQI         total cloud ice content (integral over col-            58        WMO            1        -    -     kg/m2
            umn)
TO3         total ozone content (integral over column)             10        WMO            1        -    -     Dobson

TOT_PREC    Total precipitation, i.e. the sum of                   61        WMO            1        -    -     kg/m2
            RAIN_GSP + RAIN_CON +
            SNOW_GSP + SNOW_CON
CEILING     height (above mean sea level) of the low-            157         203            1        -    -         M
            est significant cloud layer
FR_ICE      ice fraction for ocean or lake surfaces                91        WMO            1        -    -           1

H_ICE       sea ice thickness                                      92        WMO            1        -    -         m
            ice surface temperature or water surface
T_ICE                                                            215         201            1        -    -          K
            temperature



           Table 7.8:        Fields of the multi layer soil model of forecasts (VV>0) and
                                      initialised analyses (VV=0)

                                                                                         HRM (GRIB 1)
                                                                                                                 Physical Unit
                                                               Element-Nr.

                                                                             Table-Nr.

                                                                                         (LVTYP)
                                                                                          Lev-Typ




   Name                      Element
                                                                                                    Levtop

                                                                                                    Levbot
                                                                                                    (LVT)

                                                                                                     (LV)
                                                                  (EE)




T_SO        soil temperature                                     197         201          111        -   dept        K
                                                                                                          h
            water content of soil layer
W_SO                                                             198         201          111        -          mmH O
            (liquid and frozen)                                                                           in
W_SO_ICE    ice content of soil layer                            199         201          111        -   cm     mmH O
                                                         - 60 -


     The soil temperatures are defined for the nine levels 0, 0.5, 2, 6, 18, 54, 162, 486 and 1458 cm in
     soil. The temperature at the interface soil – atmosphere is identical to the temperature in the
     depth of 0.5 cm. The temperature at level 1458 cm is set to the climate average of the 2m-
     temperature.

     WARNING: As the depth could only be coded in whole centimetres in GRIB1 Code, the
     depth of 0.5 cm will be coded as 1 cm.
     Water and ice content are computed for the six soil layers 0 – 1, 1 – 3, 3 – 9, 9 – 27, 27 – 81, 81
     – 243; the layers 243 – 729 and 729 – 2187 are kept constant in time.

         Table 7.9:      Fields of initialised analyses (VV=0) and forecasts (VV>0), interpolated to
                                            pressure levels or to MSL

                                                                                            HRM (GRIB 1)




                                                                                                                 Physical Unit
                                                                  Element-Nr.

                                                                                Table-Nr.

                                                                                            (LVTYP)
                                                                                             Lev-Typ
      Name                         Element




                                                                                                       Levtop

                                                                                                       Levbot
                                                                                                       (LVT)

                                                                                                        (LV)
                                                                     (EE)




U                zonal wind                                          33         WMO          100        -   -   m/s

V                meridional wind                                     34         WMO          100        -   -   m/s

FI               geopotential                                          6        WMO          100        -   -   m2/s2

T                temperature                                         11         WMO          100        -   -       K

RELHUM           relative humidity                                   52         WMO          100        -   -      %

TD               dew point temperature                               17         WMO          100        -   -       K

QI               specific cloud ice content                          33         201          100        -   -   kg/kg

O3               ozone mixing ratio (optional)                      180         202          100        -   -   kg/kg

OMEGA            vertical velocity  = dp/dt                         39         WMO          100        -   -   Pa/s
                 surface pressure, reduced to mean sea
PS                                                                     2        WMO          102        -   -     Pa
                 level

     Attention
     For variables which are averaged over the forecast range, the value at t=0, i.e.  (0) , is the mean
     value over the first time step of HRM.

     For these variables, the mean value over a time interval, e.g. from t1 to t2 is calculated by:
                                                   - 61 -




                                      1 t2                1 t 2           t1
                                                                                  
                                             
                                   t 2 - t 1 t1
                                                 dt =          
                                                       t 2 - t1  0
                                                                     dt -   dt  =
                                                                                  
                                                                          0      

                                              1
                                                                            
                                                    t 2  ( t 2 ) - t1 ( t1 ) .
                                           t 2 - t1


The global solar radiation GLOB at the surface can be calculated approximately (neglecting back-
scattering from clouds) from the albedo (ALB_RAD, unit: %) and the solar radiation balance at the
surface (ASOB_S).
                               GLOB = ASOB_S/(1 – ALB_RAD*0.01)

The albedo depends on the soil type, soil moisture, plant cover and snow cover.


7.3 Product Definition Section PDS and Grid Description Section GDS of GRIB1 fields
To use GRIB1 fields of the HRM some basic knowledge is required of the content of the Product
Definition Section PDS (which identifies the field, e.g. element and table numbers, level, refer-
ence date, forecast time, etc.) and of the Grid Description Section GDS (which defines the hori-
zontal and vertical grid). The program grbin1 of the libgrib1.a returns, besides the unpacked
data, the PDS and GDS in the INTEGER arrays ipds and igds.

Table 7.10     The Product Definition Section PDS of the HRM

 Index       Octet         Content                                       Explanation
    1          1-3             54            length of the PDS (in Byte/Octets)
    2           4              2             table number tab, WMO-Table: tab = 2,
                                             additional national tables tab = 201 and 202
    3           5              78            data producer; e.g. for the DWD: 78; use the Namelist
                                             variable ncenter in group /gribout/ to encode another
                                             WMO centre identifier
    4           6             201            type of model (above 200 free choice)
    5           7             255            catalog-number of GRIB
    6           8             128            block-flag, indicates, if other sections like GDS follow
                                             (see WMO-GRIB Documentation Tab. 1)
    7           9              2             element number ee, see Tab. 7.1 to 7.7 in this text; atten-
                                             tion: Take care of table number tab (see above), too!
    8          10             102            level type lvtyp, see Tab. 7.9 (see below)
  9-10        11-12            0             level, depends on lvtyp, the level is identified by lv (level)
                                             or lvt (level top).
   11           13             98            year (for 2000: ipds(11) = 100)             reference date
   12           14             7             month                                       of GRIB1-field:
   13           15             20            day                                         initial date of the
   14           16             0             hour                                        forecast or
   15           17             0             minute                                      analysis
   16           18             1             time unit (tui) for P1/P2, see Tab. 7.10 (see below)
   17           19             12            forecast period 1 (P1, VV); depends on ipds (19)
   18           20             0             forecast period 2 (P2); depends on ipds (19)
   19           21             0             time flag (tflag), see Tab. 7.11 (see below)
                                                 - 62 -


 Index          Octet         Content                              Explanation
   20           22-23             0         number of mean or accumulated cases
   21            24               0         number of missing cases for sums or averages
   22            25               20        century
   23            26              255        ‚sub-center‗, national use
   24           27-28             1         scale factor D (decimal): 10D
  25-36         29-40             0         reserved for national use (from Octet 41 to end)
   37            41              254        free
   38            42              255        free
   39           43-45            200        time step (in s) of the model
   40            46              255        free
   41            47              255        free
   42            48               98        year                                      Creation date of
   43            49               07        month                                     the GRIB1-field
   44            50               20        day
   45            51               4         hour
   46            52               14        minute
   47           53-54            999        version number of model


Table 7.11       Level-types lvtyp of the HRM

lvtyp =                        Explanation                            ipds (9)          ipds (10)
ipds (8)
   1        surface field                                                 0                 0
   2        cloud base                                                    0                 0
   3        cloud top                                                     0                 0
   4        height of 0°C level                                           0                 0
   8        top of the model                                              0                 0
  100       pressure level (vertically interpolated)                      0          pressure in hPa
  102       reduced to mean sea level (msl)                               0                 0
  105       height above ground                                           0            height in m
  109       hybrid level (layer interface); for variables at              0                j3
            layer interfaces (half levels); level is defined by
            the index „j3―.
  110       hybrid layer; for variables at layers (main levels);          j3              j3 + 1
            layer is defined by „ j3― and „j3 + 1―.                     (top)           (bottom)
  111       level in the soil                                              0          depth z in cm
  112       layer in the soil                                          zo (cm)           zu (cm)
                                                                        (top)           (bottom)


Table 7.12       Unit of time of the HRM

ipds (16)                 Explanation
    1        hour
    2        day
   10        3 hours
   11        6 hours
   12        12 hours
                                             - 63 -


Table 7.13    Time flag (tflag) of the HRM

ipds (19)               Explanation
    0     forecast product valid at reference
          time + P1 (if P1 > 0)
          or
          uninitialised analysis at reference
          time (if P1 = 0)
    1     Initialised analysis (P1 = 0)
    2     product valid at time range: reference
          time + P1 to reference time + P2
    3     mean over time range: reference
          time + P1 to reference time + P2
    4     accumulated over time range:
          reference time + P1 to
          reference time + P2, valid at
          reference time + P2

Table 7.14    The Grid Description Section GDS of the HRM

 Index       Octet         Content                            Explanation
   1          1-3           298         length of the GDS (in Byte/Octets), including vertical
                                        coordinate parameters (ak, bk) for 31 layers;
                              42        without vertical coordinate parameters
   2            4             64        number of vertical coordinate parameters
                                        (i3e + 1)*2; for a 31-layer version
   3            5             43        start address (Byte-Nr.) of first vertical coordinate
                                        parameter in the GDS
   4            6             10        ´Data representation type´ in WMO GRIB –Tab. 6;
                                        ´10´: rotated latitude/longitude grid
   5           7-8           151        number of grid points in west-east direction (ie)
   6          9-10           144        number of grid points in south-north direction (je)
   7         11-13         -20000       rotated latitude of 1st grid point (lower left corner of
                                        model domain) in 10-3 degree;  s = 20.0°S
   8         14-16          10000       rotated longitude of 1st grid point (lower left corner
                                        of model domain) in 10-3 degree;  s = 10.0°E
   9           17             0         flag indicating of mesh sizes in longitudinal and lati-
                                        tudinal direction are given (WMO GRIB-Tab. 7); ´0´:
                                        mesh sizes not given.
   10        18-20          51500       rotated latitude of last grid point (upper right corner
                                        of model domain) in 10-3 degree;  e = 51.5°N
   11        21-23          85000       rotated longitude of last grid point (upper right corner
                                        of model domain) in 10-3 degree;  e = 85.0°E
   12        24-25             0        mesh size in south-north direction; not given
   13        26-27             0        mesh size in west-east direction; not given
   14         28              64        flag indicating the order of grid point storage
                                        ´scanning mode´; WMO GRIB-Tab. 8
 15-19       29-32            0         reserved
  20         33-35         -32500       geographical latitude of the rotated south pole in 10-3
                                        degree;  sp = 32.5°S
                                             - 64 -


 Index        Octet        Content                              Explanation
   21         36-38         10000       geographical longitude of the rotated south pole in
                                        10-3 degree;  sp = 10.0°E
  22          39-42             0       rotation angle
 26-89       43-298          .......    packed vertical coordinate parameters (64 elements
                                        for 31 layers, i.e. 2 x 32 coordinate parameters),
                                        first ak (j3), j3 = 1,i3e + 1, then bk(j3), j3 = 1,i3e + 1

The WMO-GRIB-Tables are part of the official GRIB1-description of the WMO.

Attention:
The mesh sizes are not given in the HRM GRIB1 data because the coding allows only to store
mesh sizes to an accuracy of 10-3 Grad; e.g. it is impossible to store 0.1875° exactly. Therefore,
the mesh size has to be derived from the other information given in the GDS (lower left and up-
per right corners, number of grid points in west-east and south-north direction).
If a regular latitude/longitude grid is used for HRM ( i. e. POLLAT = 90. and POLLON = -180.
in NAMELIST /grid_ctl/), the GDS (4) is set to "0" and the pole coordinates (GDS(20),
GDS(21)) are not given.
                                            - 65 -


8. Visualisation based on the GrADS package
The public domain graphic package GrADS (Grid Analysis and Display System)
is a good tool to visualise the forecasts of the HRM.
GrADS can be downloaded from the ftp-server http://grads.iges.org/grads/downloads.html.
Information about GrADS is available from: http://grads.iges.org/grads/head.html.

Two shortcomings of the GrADS package have to be taken into account:

   GrADS does not know anything about rotated latitude/longitude grids but treats them as reg-
    ular geographical ones,

   GrADS does not use the table number (―tab‖) of the PDS of the GRIB1 data but only the
    element number (―ee‖) to identify the fields in GRIB1 files.

To overcome the first shortcoming, switch off the automatic drawing of the land contours (i.e.
with the GrADS command set mpdraw off ), and draw the land fraction (fr_land) of the HRM
instead. Thus insert the following sequence of GrADS commands in the plot sequence after
drawing the field in question:

set gxout contour                 ! switch on contour lines
set clab off                      ! do not label contour lines
set clevs 0.5                     ! plot only line 0.5 (land is fr_land > = 0.5 )
d fr_land                         ! draw land fraction as coast lines

To overcome the second shortcoming, use different files for those HRM variables which create
problems to GrADS because they have the same element number. Currently, five elements have
to be treated that way, namely

   VMO3 (ee= 65, tab=202)          because of       W_SNOW (ee= 65, tab=2),
   RLAT (ee=114, tab=201)          because of       ATHB_T       (ee=114, tab=2),
   TOP_CON (ee=73, tab=201)        because of       CLCL        (ee= 73, tab=2),
   T_SNOW (ee=203, tab=201)        because of       CLDEPTH (ee= 203, tab=203),
   QI (ee=33, tab=201)             because of       U (ee=33, tab=2).


Dr. Helmut Frank (e-mail: Helmut.Frank@dwd.de) provided a PERL script (gme2ctl.pl) to automat-
ically generate the GrADS control (ctl) files for given HRM GRIB code analysis and forecast
files (see section 12). This script, based on wgrib, makes use of the HRM GRIB tables which are
not known to wgrib otherwise.

Moreover, there is another PERL script which allows to visualise the ASCII Meteograph files
(see section 12, too).
                                              - 66 -


9. Visualisation based on the VIS5D package
The public domain graphic package VIS5D is another good tool to visualise the forecasts of the
HRM. Contrary to GrADS, VIS5D can be used to draw 3-dimensional iso-surfaces (e.g. of the
cloud liquid water content or the kinetic energy) and animate 3-d forecast results. Information
about VIS5D can be found at http://www.ssec.wisc.edu/~billh/vis5d.html .

Because VIS5D cannot read GRIB1 code files directly but has its own internal data format, an
interface program named read_hrm_fields has been provided. It reads HRM files on model and
pressure levels for one or more forecast steps and writes selected fields to a single file in VIS5D
format (with the extension .v5d).

The runtime control parameters of the program read_hrm_fields are contained in an ASCII file
called INPUT_READ_HRM_FIELDS in the form of NAMELIST variables.

The following table explains the different control variables and switches in the NAMELIST
group /org_ctl/.

     Parameter        Type                  Meaning of Parameter                        Default
ydate_ini          CH*10        Initial date of the forecast in the form           ´     ´
                                yyyymmddgg, where
                                yyyy: year, e.g. 1998
                                mm: month, e.g. 06
                                dd: day, e.g. 25
                                gg:     time, e.g. 18 (UTC)
ydir_hrm           CH*80        Directory of the HRM data                          ´    ´
yarea              CH*1         Area indicator of GRIB1 file (e.g. ´w´)            w
ytunit             CH*1         Time unit indicator of GRIB1 file (e.g. ´f´)       f
ymode_out          CH*5         Type of output file (ASCII, GRIB1, VIS5D);         ´VIS5D´
                                currently, only VIS5D and ASCII have been
                                implemented
yvarml (:)         CH*9         Name of model level fields                         ´     ´
yvarpl (:)         CH*9         Name of pressure level fields                      ´     ´
nstart             INT          Number of first time step                          0
hstart             REAL         Same as nstart, but time in hours                  0.
nstop              INT          Number of last time step to be read                0
hstop              REAL         Same as nstop, but time in hours                   0.
ninc               INT          Interval in time steps between two sets of HRM     0
                                data
hinc               REAL         Same as ninc, but time in hours                    6.
lanalysis          LOGICAL      If .true., HRM data are analyses                   .true.
ldebug             LOGICAL      Debug switch, if .true. the program will print     .false.
                                quite a lot of debug information
ie                 INT          Number of HRM grid points in west – east di-       0
                                rection
je                 INT          Number of HRM grid points in south – north         0
                                direction
nlev_p             INT          Number of pressure levels                          0
iplev (:)          INT          Pressure levels in unit hPa (up to 50 levels can   0
                                be specified)
                                                - 67 -


10. Operational Use of the HRM Based on GME Data
10.1 Introduction
The global model GME (Majewski et al., 2002) of the DWD covers the globe with a triangular
grid of about 30 km mesh size, and it uses 60 layers. One layer consists of 655362 grid points
which is about 1.4 MByte of data in GRIB1 code. Thus it is impossible to get full global fields
via the Internet to use them as initial or lateral boundary data of the HRM. Therefore, a program
has been written which cuts out only those GME data which cover a given HRM region and
creates a bitmap which tells the de-Griber which GME grid points are defined and which are not
defined. By this method the data amount can be reduced drastically, e.g. from 655362 to 12000
grid points per field. The resulting files are small enough (2 to 30 MByte after compression with
bzip2) for efficient transfer via the Internet.
The bitmap is provided as an ASCII file. It is created at the DWD depending on the size of the
given HRM region to be covered by the GME data. To reduce the amount of GME to be trans-
ferred even farther, the lateral boundary data can be given also for a ―frame‖ of width of about
10 HRM rows/columns.


10.2 GME data needed for the HRM
To provide initial and lateral boundary data for the HRM one needs the following GME data:
Multi level fields (on all i3e_gme=60 GME layers):
t               temperature                                  unit: K
qv              water vapour content                         unit: kg/kg
qc              cloud water content                          unit: kg/kg
qi              cloud ice content                            unit: kg/kg
o3              ozone mixing ratio (optionally)              unit: kg/kg
u               zonal wind component                         unit: m/s
v               meridional wind component                    unit: m/s
Fields of the multi level soil model (0: i3e_soil + 1 for t_so; 1: i3e_soil + 1 for w_so, w_so_ice)
t_so           soil temperature                               unit: K
w_so           total (liquid and frozen) water content        unit: mm H2O
w_so_ice       ice content in the soil                        unit: mm H2O
Single level fields:
ps              surface pressure on model orography          unit: Pa
fi_control      geopotential at GME level 15                 unit: m2/s2
t_s             surface temperature                          unit: K
t_snow          snow or surface temperature                  unit: K
qv_s            specific humidity at the surface             unit: kg/kg
w_i             water content of interception storage        unit: mm H2O
w_snow          water content of snow                        unit: mm H2O
rho_snow        snow density                                 unit: kg/m3
freshsnw        snow freshness factor                        unit: -
fr_ice          sea ice fraction (either 0 or 1); optional   unit: -
h_ice           sea ice thickness; optional                  unit: m
t_ice           sea ice top temperature; optional            unit: K

Thus a total of
6 (7)*i3e_gme + 3*(i3e_soil+1)+1 + 9+ 3 GME fields are needed.
                                            - 68 -


For usual HRM domain sizes of about 3000 x 3000 km2, i.e. 145 x 145 grid points at a mesh size
of 0.20°, about 11500 GME grid points are needed. Of course, the topographical fields of the
GME (they are contained in a GRIB file named invar_i256a.ndvi for GME with the 30 km grid
spacing) have to be provided, too, but this has to be done only once.
Thus the size of the GME file needed to produce an initial / lateral boundary data file of the
HRM is approximately:
(6* 60 + 3*8+1 + 9 + 3) * 23000 Byte = 9.1 MByte
For a 48-h forecast of the HRM with lateral boundary data at 3-hourly intervals one needs to
transfer
(48/3 + 1) * 9.1 MByte = 155 MByte.
If these data are compressed with bzip2 the amount to be transfered from DWD to the HRM user
will be reduced by 30 to 40%. At a line capacity of 32 kByte/s (= 256 kBit/s) the transfer will
take less than 32 minutes.
For the ―frame‖ version of lateral boundary data where GME data are given for a ―frame‖ of
width nframe HRM grid points (nframe = 10) around the HRM domain the reduction of the
amount of GME data will be up to another 50% or more depending on the HRM domain size and
shape.
The GME files for each country using the HRM are produced on a computer at the DWD and
sent via the Internet directly to the computer of the country running the HRM. GME data are
distributed four a day at 3-hourly intervals (based on 00 and 12 UTC analyses up to 78h; the data
distribution starts at 2:45 and 14:45 UTC; based on 06 and 18 UTC analyses up to 48h; the data
distribution starts at 8:45 and 20:45 UTC).
 To get your GME sub-domain, please contact M. Gertz (e-mail: michael.gertz@dwd.de) to spe-
cify your HRM domain.

10.3 NAMELIST Input of the program GME2HRM
A program is available which interpolates the GME data from the triangular grid to the rotated
latitude/longitude grid of the HRM. The runtime control parameters of the program GME2HRM
are contained in an ASCII file called INPUT_GME2HRM in the form of NAMELIST va-
riables.
These variables are split into six different NAMELIST groups which have to appear in the fol-
lowing order in the file INPUT_GME2HRM:
/ org_ctl /           -      general control variables and switches mostly contained in
                             COMMON block / comorg /
/ grid_gme_ctl /      -      control variables defining the GME grid, contained in
                             COMMON blocks /param_gme/ and / comorg /
/ grid_hrm_ctl /      -      control variables defining the HRM grid, contained in
                             COMMON blocks /param_hrm/ and / comgrid_hrm /
/ dia_ctl /           -      control variables and switches for the diagnostics of the
                             program, contained in COMMON block / comdia /
/ gribin /            -      control variables and switches for the input data, contained in
                             COMMON blocks / comio / and /comsoil/
/ gribout /           -      control variables and switches for the GRIB1 output files. The
                             control variables and switches are contained in COMMON
                             blocks / comio / and /comsoil/
                                        - 69 -


10.4 Example of an INPUT_GME2HRM file
An example of an INPUT_GME2HRM file is given in Fig. 10.4.1

#
# Namelist Input Control file (INPUT_GME2HRM)
#
 &org_ctl
  ydate_ini='2005012500',
  lana_lb=.false., hinc_lb=3., hstart=12, hstop=60., lprog_qi=.true.,
 &end
 &grid_gme_ctl
  ni=256, i3e_gme=60, i3_con_fi=15,
 &end
 &grid_hrm_ctl
  pollat=90.0, pollon=-180.0, startlat=-20.0, startlon=-10.0,
  endlat=51.5, endlon=65.0, ie=151, je=144, i3e_hrm=60,
 &end
 &dia_ctl
  lpr_gp=.true., lpr_id=.true., lr_od=.true.,
  i1_print=20, 30, 40,
  i2_print=25, 35, 45,
 &end
 &gribin
  ydir_gme='/uhome/for3maj/gme/',
  ytopodir_gme='/uhome/for3maj/gme_topo/',
  ytopolfn_gme='invar.i256a',
  ytopodir_hrm='/uhome/for3maj/hrm_topo/',
  ytopolfn_hrm='hrm_310x300',
  ie_topo_hrm=310,
  je_topo_hrm=300,
  ybitmapdir = ´/uhome/for3maj/gme/const/´,
  ybitmaplfn = ´bitmap915´,
 &end
 &gribout
  ydir_hrm='/uhome/for3maj/hrm/',
  &end


Figure 10.4.1 Example of an input file of the program GME2HRM
                                           - 70 -


10.5 Explanation of the different control variables and switches
/ org_ctl / - General control variables and switches

 Parameter          Type                Meaning of Parameter                       Default
ydate_ini         CH*10       Initial date of the forecast in the form          ´      ´
                              yyyymmddgg, where
                              yyyy: year, e.g. 2004
                              mm: month, e.g. 06
                              dd:     day,     e.g. 25
                              gg:     time, e.g. 18 (UTC)
nproc             INT         Number of processors used in parallel on pa-          1
                              rallel shared memory machines (e.g. SGI Ori-
                              gin, HP, SUN, DEC); not yet implemented.
nstart            INT         Number of first time step;                            0
                              Alternatively:
hstart            REAL        Same as nstart, but time in hours                     0.
nstop             INT         Number of time steps to be interpolated;              0
                              Alternatively:
hstop             REAL        Same as nstop, but time in hours                      0.
ninc_lb           INT         Interval in time steps between two sets of lat-       0
                              eral boundary values.
                              Alternatively:
hinc_lb           REAL        Same as hinc_lb, but time in hours                    6.
ytrans_in         CH*100      Name of directory for ´ready´ files. These files ´         ´
                              are used to tell GME2HRM during the run that
                              a specific step has been completely transferred
                              and the corresponding GRIB file is ready for
                              interpolation (if empty, GME2HRM will not
                              wait for the files).
ytrans_out        CH*100      Name of directory for ´ready´ files. These files ´         ´
                              are used to tell HRM during the run that a spe-
                              cific step has been completely interpolated by
                              GME2HRM and the corresponding GRIB file
                              is ready for usage.
ninc_wait         INT         Interval, in seconds, for gme2hrm to perform a        0
                              check at the ´ready´ files (for boundary condi-
                              tion), in the directory especified in ytrans_in.
nmax_wait         INT         Maximum time, in seconds, for gme2hrm to              0.
                              wait for the ´ready´ file, in the directory espe-
                              cified in ytrans_in, before stopping the run.
lana_lb           LOGICAL     If .true., analyses are used as lateral boundary    .false.
                              condition; if .false., the GME data are fore-
                              casts
linitial          LOGICAL     If .true., produce initial data for the HRM, oth-   .false.
                              erwise produce lateral boundary data
ldebug            LOGICAL     Debug switch, if .true. GME2HRM will print          .false.
                              quite a lot of debug information
lprog_qi          LOGICAL     Cloud ice (qi) switch; if .true. interpolate qi     .false.
                              from GME to HRM grid.
                                           - 71 -


 Parameter         Type                Meaning of Parameter                        Default
lprog_o3         LOGICAL Ozone (o3) switch; if .true. interpolate o3 from         .false.
                         GME to HRM grid.
qvmin            REAL    Minimum value of water vapour (security); if             1.e-12
                         qv < qvmin, qv is set to qvmin.
qcmin            REAL    Minimum value of cloud water (security); if qc           1.e-12
                         < qcmin, qc is set to 0.
qimin            REAL    Minimum value of cloud ice content (securi-              1.e-12
                         ty); if qi < qimin, qi is set to 0.
o3min            REAL    Minimum value of ozone mixing ratio (securi-             1.e-20
                         ty); if o3 < o3min, o3 is set to 0.
lfilter          LOGICAL Topography filter switch; if .true. filter HRM           .false.
                         topography, i.e. remove the smallest wave
                         lengths
eps_filter       REAL    Filter coefficient of topographic filtering              0.1
luvgeo           LOGICAL Wind rotation switch; if .true. do not trasform          .false.
                         the wind components u, v to to rotated lati-
                         tude/longitude coordinate system
ldatchk          LOGICAL Date check switch; if .true. check the date/time         .true.
                         of the GME GRIB files
lframe           LOGICAL Frame switch; if .true., LBC data are defined            .false.
                         for a frame of width ―nframe‖ rows and col-
                         umns
nframe           INT     Width of frame (number of rows/columns)                    10
lsea_ice         LOGICAL Sea ice model switch; if .true. provide sea ice          .false.
                         model fields (fr_ice, h_ice, t_ice)

/ grid_gme_ctl / - Grid definition of the GME

 Parameter         Type                Meaning of Parameter                        Default
ni               INT         resolution of the GME                                128
i3e_gme          INT         number of layers of the GME                           31
i3_con_fi        INT         index of control level for geopotential               15

/ grid_hrm_ctl / - Grid definition of the HRM

 Parameter         Type                Meaning of Parameter                        Default
pollat           REAL        latitude of rotated north pole; for regular lati-    90.
                             tude/longitude grid set pollat = 90. (°)
pollon           REAL        longitude of rotated north pole; for regular        -180.
                             latitude/longitude grid set pollon = -180. (°)
startlat         REAL        latitude of lower left corner of the HRM do-           0.
                             main (in °)
startlon         REAL        longitude of lower left corner of the HRM do-          0.
                             main (in °)
endlat           REAL        latitude of upper right corner of the HRM do-          0.
                             main (in °)
endlon           REAL        longitude of upper right corner of the HRM             0.
                                             - 72 -


 Parameter           Type                Meaning of Parameter                       Default
                               domain (in °)
ie                INT          number of grid points in west-east direction         0
je                INT          number of grid points in south-north direction       0
i3e_hrm           INT          number of layers                                     0
zak (51)          REAL         vertical coordinate parameter (pressure part)        0.
zbk (51)          REAL         vertical coordinate parameter (sigma part)           0.
i3e_soil          INT          number of layers of the multi-layer soil model;      7
                               must be the same in GME and HRM!
Attention:
The mesh sizes dlon, dlat are not controlled via NAMELIST but computed from ie, startlon,
endlon and je, startlat, endlat.
If the vertical coordinate parameters (zak, zbk) are not given but the number of layers in the
HRM (i3e_hrm) is equal to the one of the GME (i3e_gme) HRM will use the same vertical coor-
dinate parameters as the GME.
Note: the parameters (zak, zbk) must be given for the half layers.
GME and HRM must have the same number and placement of layers for the multi-layer soil
model!

/ dia_ctl / - Control variables and switches for diagnostics
As of now, three types of diagnostics have been implemented in the program GME2HRM,
namely
a)     Print of mean, max, min of each input field (lpr_id= .true.),
       print of mean, max, min of each output field (lpr_od = .true.).
b)     Print of full fields at two levels (i3_print1, i3_print2) and switched on by the variables
       lpr_ps, lpr_t, ..., lpr_dpdt.
c)     Print of values at selected grid points (up to 20) switched on by lpr_gp = .true. and for
       the selection of grid point indices i1_print (20), i2_print (20). The results are written to
       an ASCII file named DIAGNOSTICS_GME2HRM.

 Parameter           Type                Meaning of Parameter                       Default
i3_print1         INT          first level of printing of multi level fields        1
i3_print2         INT          second level of printing of multi level fields      15
i1_print (20)     INT          j1-index of grid points to be printed                0
i2_print (20)     INT          j2-index of grid points to be printed                0
lpr_ps            LOGICAL      if .true., print ps, fis                           .false.
lpr_t             LOGICAL      if .true., print t at two layers                   .false.
lpr_u             LOGICAL      if .true., print u at two layers                   .false.
lpr_v             LOGICAL      if .true., print v at two layers                   .false.
lpr_grh           LOGICAL      if .true., print grh at two layers                 .false.
lpr_qv            LOGICAL      if .true., print qv at two layers                  .false.
lpr_qc            LOGICAL      if .true., print qc at two layers                  .false.
lpr_qi            LOGICAL      if .true., print qi at two layers                  .false.
lpr_o3            LOGICAL      if .true., print o3 at two layers                  .false.
lpr_ud            LOGICAL      if .true., print divergent wind correction ud      .false.
lpr_vd            LOGICAL      if .true., print divergent wind correction vd      .false.
lpr_dpdt          LOGICAL      if .true., print surface pressure tendency         .false.
                                             - 73 -


 Parameter           Type                 Meaning of Parameter                         Default
lpr_gp             LOGICAL if .true., print grid point diagnostics                 .true.
lpr_id             LOGICAL Logical switch, if .true., check the initial data,      .true.
                           i.e. print some information like maximum and
                           minimum values of each input field
lpr_od             LOGICAL Logical switch, if .true., check the output data,       .true.
                           i.e. print some information like maximum and
                           minimum values of each output field

/ gribin / - Control variables and switches of the input

 Parameter           Type                 Meaning of Parameter                         Default
ytopodir_gme       CH*80        Directory path of the topographical data of the    ´        ´
                                GME
ytopodir_hrm       CH*80        Directory path of the topographical data of the    ´        ´
                                HRM
ytopolfn_gme       CH*14        File name of the topographical data of the         ´        ´
                                GME
ytopolfn_hrm       CH*14        File name of the topographical data of the         ´        ´
                                HRM
ydir_gme           CH*80        Directory path of the data file(s) of the GME      ´        ´
ybitmapdir         CH*80        Directory path of the bitmap file of the GME       ´        ´
ybitmaplfn         CH*14        File name of the bitmap of the GME                 ´        ´
nvers_gme          INT          Version number of the GME data                       1
ie_topo_hrm        INT          Number of grid points in west-east direction of    400
                                the topographical data file of the HRM
je_topo_hrm        INT          Number of grid points in south-north direction     400
                                of the topographical data file of the HRM

/ gribout / - Control variables and switches of the output

Parameter         Type          Meaning of Parameter                               Default
ydir_hrm           CH*80        Directory path of the output data file(s) of the   ´        ´
                                HRM
nvers_hrm          INT          Version number of the HRM data                       1
itypnr_hrm         INT          Type number (for GRIB code) of the HRM             201
                                data
ATTENTION
Initial data of GME are always given for the full HRM domain because they are used to create
initial data of the HRM. In the ―frame‖ version of lateral boundary data (lframe = .true.) a second
bitmap is needed for these GME data (forecast step > 0h). Thus the first lateral boundary data
set of HRM (forcast step: 0h) has to be derived by a gme2hrm run using the GME initial data file
(with the corresponding bitmap), while all other lateral boundary data sets of HRM (forecast
steps > 0h) have to be derived with the second bitmap.
                                              - 74 -


10.6 Memory requirements of the program GME2HRM
The program GME2HRM has to allocate data of the GME (dimensions given by ni and i3e_gme)
and of the HRM (dimensions given by ie, je and i3e_hrm). All in all, the program allocates the
following data

Permanent storage:

3 x (ni + 1) x (ni + 1) x 10 +
(36 + 27) x (ie + 2) x (je + 2) + 6 x (ie + 2) x (je + 2) x MAX (i3e_gme, i3e_hrm)

Local storage:

2 x (ni + 1) x (ni + 1) x 10 x i3e_gme + 6 x (ni + 1) x (ni + 1) x 10 +
2 x (ie + 2) x (je + 2)

Thus the total memory needed by the program for typical values like

ni = 256, i3e_gme = 60, ie = 151, je = 150, i3e_hrm = 60

is about 800 Mbyte on 32-Bit workstations.


10.7 Operational scheduler
M. Gertz (DWD) developed an operational scheduler based on Unix scripts which allows an
automatic production of HRM forecasts. This scheduler is in operational use in Bulgaria, Kenya,
Oman, United Arab Emirates and Vietnam.

The scheduler consists of three main parts

   Part I is running on the machine where the GME data files sent by the DWD arrive. This
    scripts checks for the start of the GME data transmission. It is initialised via Cron and has to
    run all the time.

   Part II is running on the machine where the GME data files have to be archived. This script is
    initialised via Cron and has to run all the time.

   Part III is running on each machine where a HRM has to run. This script is initialised via
    Cron and has to run all the time; it will initialise a complete HRM run including gme2hrm,
    hrm, and some post-processing (e.g. graphics).

For more information, please contact M. Gertz (e-mail: michael.gertz@dwd.de).
                                                - 75 -


11. Interpolation HRM-X to HRM-Y
11.1 Introduction
The program HMX2HMY allows to use HRM forecasts of a coarser grid HRM (HRM-X) as
initial/lateral boundary conditions for a high-resolution HRM (HRM-Y). Both HRMs must use
the same rotated pole coordinates and the same number and placement of soil layers, but the
number and placement of the atmospheric layers may be totally different.
The domain of HRM-X must be at least two rows/columns larger than the HRM-Y domain be-
cause the horizontal interpolation of atmospheric fields is based on a bicubic spline function. For
surface fields, a bilinear interpolation is used and the land/sea masks of both HRMs are usually
taken into account.


11.2 HRM-X data needed for the HRM-Y
To provide initial and lateral boundary data for HRM-Y one needs the following HRM-X data:

Multi level fields (on all i3e_hmx HRM-X layers):
t               temperature                                  unit: K
qv              water vapour content                         unit: kg/kg
qc              cloud water content                          unit: kg/kg
qi              cloud ice content                            unit: kg/kg
o3              ozone mixing ratio (optionally)              unit: kg/kg
u               zonal wind component                         unit: m/s
v               meridional wind component                    unit: m/s

Fields of the multi level soil model (0: i3e_soil + 1 for t_so; 1: i3e_soil + 1 for w_so, w_so_ice)
t_so           soil temperature                               unit: K
w_so           total (liquid and frozen) water content        unit: mm H2O
w_so_ice       ice content in the soil                        unit: mm H2O

Single level fields:
ps              surface pressure on model orography          unit: Pa
fi_control      geopotential at HRM-X level 15               unit: m2/s2
t_s             surface temperature                          unit: K
t_snow          snow or surface temperature                  unit: K
qv_s            specific humidity at the surface             unit: kg/kg
w_i             water content of interception storage        unit: mm H2O
w_snow          water content of snow                        unit: mm H2O
rho_snow        snow density                                 unit: kg/m3
freshsnw        snow freshness factor                        unit: -
fr_ice          sea ice fraction (either 0 or 1); optional   unit: -
h_ice           sea ice thickness; optional                  unit: m
t_ice           sea ice top temperature; optional            unit: K

Thus a total of
6 (7)*i3e_hmx + 3*( i3e_soil + 1) + 1 + 9 + 3 HRM-X fields are needed

Usually, these data are written in HRM-X in a separate GRIB1 file which uses the yarea-
parameter ‗h‘ (in HRM-NAMELIST / gribout / ).
                                           - 76 -


Thus two more NAMELIST / gribout / have to be added to the INPUT_HRM of HRM-X of the
form:

 &gribout
    ngrib=0, lpp_ini=.true.,
    ydir= '${hrm_fctdir}/',
    ysystem='file', ytunit='f',               yarea='h',
    yvarml='T'       ,'QV'                    ,'QC'      ,'QI'      ,'U'     ,
           'V'       ,'FI'                    ,'PS'      ,'T_S'     ,'T_SNOW',
           'W_SNOW' ,'QV_S'                   ,'W_I'   ,'FRESHSNW','RHO_SNOW',
           'T_SO'    ,'W_SO'                  ,'W_SO_ICE','PLCOV' ,'FR_ICE' ,
           'T_ICE'   ,'H_ICE',
 /end
&gribout
    hcomb= 1.,78.,1.,
    ydir= '${hrm_fctdir}/',
    ysystem='file', ytunit='f',               yarea='h',
    yvarml='T'       ,'QV'                    ,'QC'      ,'QI'      ,'U'     ,
           'V'       ,'FI'                    ,'PS'      ,'T_S'     ,'T_SNOW',
           'W_SNOW' ,'QV_S'                   ,'W_I'   ,'FRESHSNW','RHO_SNOW',
           'T_SO'    ,'W_SO'                  ,'W_SO_ICE',
           'T_ICE'   ,'H_ICE',
 /end


11.3 NAMELIST Input of the program HMX2HMY
The runtime control parameters of the program HMX2HMY are contained in an ASCII file
called INPUT_HMX2HMY in the form of NAMELIST variables.
These variables are split into six different NAMELIST groups which have to appear in the fol-
lowing order in the file INPUT_HMX2HMY:

/ org_ctl /          -      general control variables and switches mostly contained in
                            COMMON block / comorg /
/ grid_hmx_ctl /     -      control variables defining the HRM-X grid, contained in
                            COMMON blocks / param_hmx / and / comgrid_hmx /
/ grid_hmy_ctl /     -      control variables defining the HRM-Y grid, contained in
                            COMMON blocks / param_hmy / and / comgrid_hmy /
/ dia_ctl /          -      control variables and switches for the diagnostics of the
                            program, contained in COMMON block / comdia /
/ gribin /           -      control variables and switches for the input data, contained in
                            COMMON blocks / comio / and /comsoil/
/ gribout /          -      control variables and switches for the GRIB1 output files. The
                            control variables and switches are contained in COMMON
                            blocks / comio / and /comsoil/
                                           - 77 -


11.4 Example of an INPUT_HMX2HMY file
An example of an INPUT_HMX2HMY file is given in Fig. 11.4.1

#
# Namelist Input Control file (INPUT_HMX2HMY)
#
 &org_ctl
  ydate_ini='1998120100',
  lana_lb=.false., hinc_lb=3., hstart=12, hstop=60., lprog_qi=.true.,
 &end
 &grid_hmx_ctl
  pollat_x=90.0, pollon_x=-180.0, startlat_x=-20.0, startlon_x=-10.0,
  endlat_x=51.5, endlon_x=65.0, iex=151, jex=144, i3e_hmx=60,
 &end
 &grid_hmy_ctl
  pollat_y=90.0, pollon_y=-180.0, startlat_y=-10.0, startlon_y=0.0,
  endlat_y=41.5, endlon_y=50.0, iey=201, jey=207, i3e_hmy=60,
 &end
 &dia_ctl
  lpr_gp=.true., lpr_id=.true., lpr_od=.true.,
  i1_print=20, 30, 40,
  i2_print=25, 35, 45,
 &end
 &gribin
  ydir_hmx='/uhome/for3maj/hmx/',
  ytopodir_hmx='/uhome/for3maj/hmx_topo/',
  ytopolfn_hmx='hmx_0.5',
  ytopodir_hmy='/uhome/for3maj/hmy_topo/',
  ytopolfn_hmy='hmy_0.25',
  ie_topo_hmx=310,
  je_topo_hmx=300,
  ie_topo_hmy=440,
  je_topo_hmy=380,
 &end
 &gribout
  ydir_hmy='/uhome/for3maj/hmy/',
&end


Figure 11.4.1 Example of an input file of the program HMX2HMY

11.5 Explanation of the different control variables and switches
/ org_ctl / - General control variables and switches

 Parameter          Type                Meaning of Parameter                     Default
ydate_ini         CH*10       Initial date of the forecast in the form       ´         ´
                              yyyymmddgg, where
                              yyyy: year, e.g. 2004
                              mm: month, e.g. 06
                              dd:     day,     e.g. 25
                              gg:     time, e.g. 18 (UTC)
nproc             INT         Number of processors used in parallel on pa-        1
                              rallel shared memory machines (e.g. SGI Ori-
                              gin, HP, SUN, DEC); not yet implemented.
nstart            INT         Number of first time step;                          0
                              Alternatively:
hstart            REAL        Same as nstart, but time in hours                   0.
nstop             INT         Number of time steps to be interpolated;            0
                              Alternatively:
                                    - 78 -


 Parameter    Type              Meaning of Parameter                          Default
hstop        REAL      Same as nstop, but time in hours                        0.
ninc_lb      INT       Interval in time steps between two sets of lat-         0
                       eral boundary values.
                       Alternatively:
hinc_lb      REAL      Same as hinc_lb, but time in hours                      6.
ytrans_in    CH*100    Name of directory for ´ready´ files. These files   ´         ´
                       are used to tell HMX2HMY during the run
                       that a specific step has been completely trans-
                       ferred and the corresponding GRIB file is
                       ready for interpolation.
ytrans_out   CH*100    Name of directory for ´ready´ files. These files   ´         ´
                       are used to tell HRM-Y during the run that a
                       specific step has been completely interpolated
                       by HMX2HMY and the corresponding GRIB
                       file is ready for usage.
ninc_wait    INT       Seconds to wait for the next lateral boundary           0
                       data file:
nmax_wait    INT       Maximum seconds to wait for the next lateral            0.
                       boundary data file
lana_lb      LOGICAL   If .true., analyses are used as lateral boundary   .false.
                       condition; if .false., the HRM-Y data are fore-
                       casts
linitial     LOGICAL   If .true., produce initial data for HRM-Y, oth-    .false.
                       erwise produce lateral boundary data
ldebug       LOGICAL   Debug switch, if .true. HMX2HMY will print         .false.
                       quite a lot of debug information
lprog_qi     LOGICAL   Cloud ice (qi) switch; if .true. interpolate qi    .true.
                       from HRM-X to HRM-Y grid.
lprog_o3     LOGICAL   Ozone (o3) switch; if .true. interpolate o3 from   .false.
                       HRM-X to HRM-Y grid.
qvmin        REAL      Minimum value of water vapour (security); if       1.e-12
                       qv < qvmin, qv is set to qvmin.
qcmin        REAL      Minimum value of cloud water (security); if qc     1.e-12
                       < qcmin, qc is set to 0.
qimin        REAL      Minimum value of cloud ice content (securi-        1.e-12
                       ty); if qi < qimin, qi is set to 0.
o3min        REAL      Minimum value of ozone mixing ratio (securi-       1.e-20
                       ty); if o3 < o3min, o3 is set to 0.
lfilter      LOGICAL   Topography filter switch; if .true. filter HRM-    .false.
                       Y topography, i.e. remove the smallest wave
                       lengths
eps_filter   REAL      Filter coefficient of topographic filtering        0.1
lsea_ice     LOGICAL   Sea ice model switch; if .true. provide sea ice    .false.
                       model fields (fr_ice, h_ice, t_ice)
                                          - 79 -


/ grid_hmx_ctl / - Grid definition of HRM-X

 Parameter         Type              Meaning of Parameter                     Default
pollat_x         REAL        Latitude of rotated north pole; for regular    90.
                             latitude/longitude grid set pollat = 90. (°)
pollon_x         REAL        Longitude of rotated north pole; for regular -180.
                             latitude/longitude grid set pollon = -180. (°)
startlat_x       REAL        Latitude of lower left corner of HRM-X           0.
                             domain (in °)
startlon_x       REAL        Longitude of lower left corner of HRM-X          0.
                             domain (in °)
endlat_x         REAL        Latitude of upper right corner of HRM-X          0.
                             domain (in °)
endlon_x         REAL        Longitude of upper right corner of HRM-X         0.
                             domain (in °)
iex              INT         Number of grid points in west-east direction     0
jex              INT         Number of grid points in south-north direc-      0
                             tion
i3e_hmx          INT         Number of atmospheric layers                     0
i3e_soil         INT         Number of soil layers; must be the same for      7
                             HRM-X and HRM-y
i3_con_fi        INT         Control level of HRM-X geopotential             15

Attention:
The pole coordinates (pollat_x, pollon_x) of HRM-X must be the same as the ones of HRM-Y!


/ grid_hmy_ctl / - Grid definition of HRM-Y

 Parameter         Type              Meaning of Parameter                     Default
pollat_y         REAL        Latitude of rotated north pole; for regular    90.
                             latitude/longitude grid set pollat = 90. (°)
pollon_y         REAL        Longitude of rotated north pole; for regular -180.
                             latitude/longitude grid set pollon = -180. (°)
startlat_y       REAL        Latitude of lower left corner of HRM-Y          0.
                             domain (in °)
startlon_y       REAL        Longitude of lower left corner of HRM-Y         0.
                             domain (in °)
endlat_y         REAL        Latitude of upper right corner of HRM-Y         0.
                             domain (in °)
endlon_y         REAL        Longitude of upper right corner of HRM-Y        0.
                             domain (in °)
iey              INT         Number of grid points in west-east direction    0
jey              INT         Number of grid points in south-north direc-     0
                             tion
i3e_hmy          INT         Number of atmospheric layers                    0
zak (51)         REAL        Vertical coordinate parameter (pressure         0.
                             part)
zbk (51)         REAL        Vertical coordinate parameter (sigma part)      0.
                                               - 80 -


Attention:
The mesh sizes dlon_y, dlat_y are not controlled via NAMELIST but computed from iey, star-
tlon_y, endlon_y and jey, startlat_y, endlat_y.
If the vertical coordinate parameters (zak, zbk) are not given but the number of layers in HRM-Y
(i3e_hmy) is equal to the one of HRM-X (i3e_hmx) HRM-Y will use the same vertical coordi-
nate parameters as the HRM-X.
The number and placement of soil layers (i3e_soil) must be the same for HRM-X and HRM-Y.


/ dia_ctl / - Control variables and switches for diagnostics

As of now, three types of diagnostics have been implemented in the program HMX2HMY,
namely

a)       Print of mean, max, min of each input field (lpr_id= .true.),
         print of mean, max, min of each output field (lpr_od = .true.).
b)       Print of full fields at two levels (i3_print1, i3_print2) and switched on by the variables
         lpr_ps, lpr_t, ..., lpr_dpdt.
c)       Print of values at selected grid points (up to 20) switched on by lpr_gp = .true. and for
         the selection of grid point indices i1_print (20), i2_print (20). The results are written to
         an ASCII file named DIAGNOSTICS_HMX2HMY.

 Parameter             Type               Meaning of Parameter                      Default
i3_print1           INT     first level of printing of multi level fields           1
i3_print2           INT     second level of printing of multi level fields        15
i1_print (20)       INT     j1-index of grid points to be printed                   0
i2_print (20)       INT     j2-index of grid points to be printed                   0
lpr_ps              LOGICAL if .true., print ps, fis                             .false.
lpr_t               LOGICAL if .true., print t at two layers                     .false.
lpr_u               LOGICAL if .true., print u at two layers                     .false.
lpr_v               LOGICAL if .true., print v at two layers                     .false.
lpr_grh             LOGICAL if .true., print grh at two layers                   .false.
lpr_qv              LOGICAL if .true., print qv at two layers                    .false.
lpr_qc              LOGICAL if .true., print qc at two layers                    .false.
lpr_qi              LOGICAL if .true., print qi at two layers                    .false.
lpr_o3              LOGICAL if .true., print o3 at two layers                    .false.
lpr_ud              LOGICAL if .true., print divergent wind correction ud        .false.
lpr_vd              LOGICAL if .true., print divergent wind correction vd        .false.
lpr_dpdt            LOGICAL if .true., print surface pressure tendency           .false.
lpr_gp              LOGICAL if .true., print grid point diagnostics              .true.
lpr_id              LOGICAL Logical switch, if .true., check the initial         .true.
                            data, i.e. print some information like maxi-
                            mum and minimum values of each input
                            field
lpr_od              LOGICAL Logical switch, if .true., check the output          .true.
                            data, i.e. print some information like maxi-
                            mum and minimum values of each output
                            field
                                            - 81 -


/ gribin / - Control variables and switches of the input

 Parameter          Type               Meaning of Parameter                         Default
ytopodir_hmx      CH*80        Directory path of the topographical data of    ´      ´
                               HRM-X
ytopodir_hmy      CH*80        Directory path of the topographical data of    ´      ´
                               HRM-Y
ytopolfn_hmx      CH*14        File name of the topographical data of         ´      ´
                               HRM-X
ytopolfn_hmy      CH*14        File name of the topographical data of         ´      ´
                               HRM-Y
ydir_hmx          CH*80        Directory path of the data file(s) of HRM-X    ´      ´
nvers_hmx         INT          Version number of HRM-X data                     1
ie_topo_hmx       INT          Number of grid points in west-east direction   400
                               of the topographical data file of HRM-X
je_topo_hmx       INT          Number of grid points in south-north direc-    400
                               tion of topographical data file of HRM-X
ie_topo_hmy       INT          Number of grid points in west-east direction   400
                               of the topographical data file of HRM-Y
je_topo_hmy       INT          Number of grid points in south-north direc-    400
                               tion of topographical data file of HRM-Y
yarea             CH*1         Area character for HRM-X data files            ‗h‘

/ gribout / - Control variables and switches of the output

Parameter         Type         Meaning of Parameter                           Default
ydir_hmy          CH*80        Directory path of the output data file(s) of   ´      ´
                               HRM-Y
nvers_hmy         INT          Version number of HRM-Y data                     1
itypnr_hmy        INT          Type number (for GRIB code) of HRM-Y           201
                               data
                                             - 82 -


12. Implementation of the HRM System
12.1 Introduction
So far, the HRM has been tested successfully on the following computer systems:
Cray X1, Cray XD1, SGI Origin, SGI Altix, IBM Power 3, IBM Power 4, IBM Power 5, DEC,
Sun and HP workstations, as well as Linux PC and Linux Cluster systems.

The following system requirements are needed:
 Unix (or Linux) operating system,
 Korn shell,
 make,
 Fortran 90 compiler (for multi-processor shared memory systems, the compiler should be
   able to translate Open-MP directives; for distributed memory systems like PC-Clusters, the
   MPI library must be implemented.),
 C compiler including preprocessor to handle IF DEF.

See also section 6 of this User´s Guide for the memory requirements.

At the DWD the HRM package is usually prepared with a test data set covering your domain of
interest, and reference outputs are included, too.


12.2 Download HRM package from DWD’s ftp-server
If you are a new user, start from the step below. If you already have your system working and
need to update some routine, replace the old one and follow the instructions ahead for the respec-
tive routine.

  Create a master directory (e.g. hrm) and change into it.

⊲ mkdir hrm
⊲ cd hrm

  From our ftp-server (ftp-incoming.dwd.de) with user: feu2 password: xxxxxxx (ask
   for the password via e-mail) get the "gzipped" tar-file of the HRM System set up and
   tested for you at the DWD.

⊲ ftp ftp-incoming.dwd.de
⊲ cd download
⊲ binary

⊲ get hrm_country_name.tar.gz

  "gunzip" and "de-tar" (tar -xvf ....tar) the tar-file

e.g.   ⊲ gunzip hrm_country_name.tar.gz
       ⊲ tar -xvf hrm_country_name.tar
                                            - 83 -



⇛ Now there are the following sub-directories:

bin                   directory of the binaries (gme2hrm, hmx2hmy and hrm)
gme                   directory of GME data (constant and actual fields)
gme2hrm               directory of INPUT and job files for "gme2hrm"
grads_ctl             directory of .ctl-files of graphic program "GrADS"
hmx2hmy               directory of INPUT and job files for "hmx2hmy"
hrm                   directory of HRM data (constant fields) and INPUT file for ―hrm‖
include               directory of some header files
lib                   directory of the libraries
obj                   directory of some object files
reference_output:     directory of some reference output (ASCII)
src                   directory of the source code with the sub-directories
gme2hrm_2.x           GME ⇨ HRM interpolation; version 2.x
grib1                 GRIB1-library
hmx2hmy_2.x           HRM⇨ HRM interpolation; version 2.x
hrm_2.x               HRM; version 2.x
math_dwd_1.x          Mathematical-library (Version 1.x, with version number ―x‖ )
newgrib1              GRIB1-library for 64-bit computers (like Cray)
read_hrm_grib         GRIB1 post-processing, e. g. GRIB ⇨ ASCII; GRIB ⇨ VIS5D
scheduler             Operational scheduler based on ksh-script
supplement            Supplementary programs
support               Additional programs (like dummy_mpi)
wgrib_dwd             GRIB1 reader; preparation of ctl- and idx-files for GrADS


12.3 Creation of the libraries which are needed by HRM and GME2HRM
The HRM system contains compiler calls and flags for xlf (IBM), ifc (INTEL), PGI and SUN
compilers. To use the proper compilers/flags remove the "#" sign in front of the corresponding
line in the "Makefile" or "Options" and "LinkLibs" files.

Go to the source code directory src

⊲ cd src

  GRIB1-library

HRM can run with either 32 bit (4 byte) INTEGERS/REALS or with 64 bit (8 byte) INTE-
GERS/REALS. The only difference is the GRIB code library which you have to link.
For 32-bit computers please use the source code in the directory grib1.
For 64-bit computers you may use the source code in the directory newgrib1.

But even on a 64-bit computer you can still use 32-bit INTEGERS/REALS because of the mem-
ory (RAM). You can store twice as much data in 32 bit compared to 64 bit. Since the model ac-
curacy is not affected by the length of INTEGERS/REALS you can use 32-bit even on 64-bit
computers.

 grib1 source code
                                                   - 84 -


    This is the most complicated task, because the GRIB1-library contains programs written in For-
    tran77 and C; the Fortran-C interface is not standard but depends on the computer. This is taken
    care of by using IF DEFs in the C-routines. Please have a look at these routines (cback.c,
    cclose.c, cinquire.c, copen.c, crewind.c, cuegex.c, cuegin.c, fsleep.c) and check, which of the
    DEFINEs fit your computer. If necessary, introduce your own DEFINEs, otherwise you have to
    set the proper name in the Makefile (CCFLAGS =             -02    -D . . .).
    For Intel-type PCs (and Linux PC Clusters) set -DDEC!!

    ⊲ cd grib1

The makefile "Makefile" contains the proper settings for xlf, ifc and pg compiler. Just remove
the "#" in front of the corresponding lines.

    -> vi Makefile

    Now "make" the library

    -> make

    In the directory "../../lib" you will find the library "libgrib1.a".

newgrib1 source code (for 64-bit computer systems)
The structure of the library is the following: By detaring newgrib1.tar a directory "newgrib1" is
created with 2 subdirectories:

1) include: contains 4 include-files:

     fortran_c.h: defines the environment of the Fortran-C interface. Depending on the compiler-
      flag -Dxxx, the following variables are defined:
                 FORTRAN_UNDERLINE and FORTRAN_UPPERCASE

              "xxx" can be one of the following:
               __sgi
               __uxp__
               __linux__
               _CRAY
               _AIX
               _WIN32

     libdwd_c.h: contains some internal bookkeeping and the definitions of platform dependent
      functions for input/output with C-routines.

              Especially the SGI/Cray "FFIO" can be chosen by
              setting the compiler flag -DFFIO

     machconsts.h: contains values for machine dependent settings, especially the value for the
      number of bits per default integer.

              With -DCRAY the values for 64-bit machines (like the ones
              from CRAY) can be chosen.
                                                - 85 -


     undef.h:   contains some values for undefined numbers

2) source: contains the source code of the grib-library together with a Makefile. In the Makefile,
the compiler options and the path of the library have to be set.

⊲ cd newgrib1/source

Choose the compiler flags, especially the appropriate setting for the Fortran-C interface and the
machine constants (see above). Set the Path, where the library should be written and the Path of
the include-files.

⊲ vi Makefile

Now "make" the library

⊲ make

In the directory "../../lib" you will find the library "libgrib1.a".

For running the GME2HRM using the ASCII bitmap-files, the following environment variables
have to be set to:

    LIBDWD_BITMAP_TYPE=ASCII
    LIBDWD_BITMAP_PATH= (write the path to the bitmap file)

(because this library can also deal with binary bitmap files, which is the default).


     Mathematical library

⊲ cd math_dwd_1.x

The file "Options" contains the proper settings for xlf, ifc and pg compiler. Just remove the "#"
in front of the corresponding lines.

⊲ vi Options

Now "make" the library

⊲ make

In the directory "../../lib" you will find the library "libmath_dwd.a".


     Supplement library

⊲ cd supplement

The makefile "Makefile" contains the proper settings for xlf, ifc and pg compiler. Just remove
the "#" in front of the corresponding lines.
                                               - 86 -




⊲ vi Makefile

Now "make" the library

⊲ make

In the directory "../../lib" you will find the library "libsupplement.a".


  Some object files and support stuff (in the include directory)

Attention:
The files "mpif.h" and "dummy_mpi.o" are necessary for linking the OpenMP-Version of
HRM for shared memory computer systems.
On distributed memory systems (like Linux PC-Clusters), these files must not be used during the
linking of HRM but the MPI-library installed on this system must be included during linking.

⊲ cd support

The makefile "Makefile" contains the proper settings for xlf, ifc and pg compiler. Just remove
the "#" in front of the corresponding lines.
⊲ vi Makefile

Now "make" the object files

⊲ make

In the directory "../../obj" you will find three files "dummy_mpi.o", "dummy_mpi_gme.o",
"hpm.o", and in the directory "../../include" three files "gribtab.h", "mpif.h", "pr_gribvar.h".
ATTENTION: On distributed memory computers do not use these dummy MPI stuff but the
native MPI library and the correct "mpif.h" file!


12.4 Creation and test of the binaries (HRM, GME2HRM and HMX2HMY)

  Binary "gme2hrm"

⊲ cd gme2hrm_2.x

The files "Options" and "LinkLibs" contain the proper settings for xlf, ifc and pg compiler. Just
remove the "#" in front of the corresponding lines.

Now "make" the object files, library and the binary

⊲ make

In the directory "../../lib" you will find the library "libgme2hrm.a".
In the directory "../../bin" you will find the binary "gme2hrm".
                                                - 87 -




ATTENTION: On Cray X1 (or other 64-bit integer machine): the variable intgribf must be set
to 8 in the param_hrm.h file.


  Binary "hrm"

⊲ cd hrm_2.x

Differentiate between shared memory (e.g. SGI Altix) and distributed memory systems (e.g.
Linux PC Clusters).
For shared memory systems, use the OpenMP compiler option and link with the dummy MPI
objects (from ../../obj) and the dummy "mpif.h" file (from ../../include).
For distrubuted memory systems, link with the native MPI library (e.g. mpich) and "mpif.h" file.

The files "Options" and "LinkLibs" contain the proper settings for xlf, ifc and pg compiler. Just
remove the "#" in front of the corresponding lines.

Now "make" the object files and the binary

⊲ make

In the directory "../../lib" you will find the library "libhrm.a".
In the directory "../../bin" you will find the binary "hrm".

The HRM can use more than one CPU; the multi-processor version is based on standard
OpenMP directives for shared memory computer systems or MPI parallelization with explicit
message passing for distributed memory computers.

  Test "gme2hrm" and compare the results with your reference files

In the directory "../gme/const" you will find the GME topography file; in the directory
"../gme/d00" you will find the GME data (initial and forecast data as well as the bitmap).
ATTENTION: The GME data are gezipped! Thus: -> gunzip g*.z first!
In the directory "../hrm/const" you will find the HRM topography file.

⊲ cd gme2hrm

Modify the input file INPUT_GME2HRM of "gme2hrm" according to your system
e.g. ytopodir_gme, ytopodir_hrm, ydir_gme, ybitmapdir, ydir_hrm;
set the proper directory path names

For a test, you can run "gme2hrm" interactively

⊲ ../bin/gme2hrm

Because the program uses quite a lot of stack space, you have to set the stack space to "unlim-
ited" (ulimit –s unlimited; or ask your system administrator!); otherwise the program will abort
with an error.
                                               - 88 -


Compare your ASCII control output files DIAGNOSTICS_GME2HRM and
OUTPUT_GME2HRM with the ones given in the directory "reference_output".
Small differences may be due to different computers, but larger differences require some cross
checking!

After creation of the initial data for the HRM (file "haf........" in the directory hrm/d00) you have
to create the lateral boundary data files for the HRM for 0h to 24h in 3-hourly intervals. Modify
the input file INPUT_GME2HRM; set "hstop" to 24.0 and "linitial" to .false.

Run "gme2hrm" again. Now you have created the lateral boundary data files (named "hbf........")
in the directory hrm/d00. The files "haf........" and "hbf........." are in GRIB1 format.


  Test "hrm" and compare the results with your reference files

⊲ cd hrm

Modify the input file INPUT_HRM of "hrm" according to your system e.g. yanadir, ylbdir, ydir;
set the proper directory path names.
Set the number of CPUs used (variables nproc1 and nproc2 in NAMELIST / hrm_ctl / ) to "1"
for the first run. Later on you may try to use more CPUs if your system has more than one CPU
available.
ATTENTION: For distributed memory systems set lmpi = .true. in Namelist / hrm_ctl /

For a test, you can run "hrm" interactively.

⊲ ../bin/hrm

Because the program uses quite a lot of stack space, you have to set the stack space to "unlim-
ited" (ask your system administrator!); otherwise the program will abort with an error.

Compare your ASCII control output files DIAGNOSTICS, OUTPUT_HRM and M_****** with
the ones given in the directory "reference_output". Small differences may be due to different
computers, but larger differences require some cross checking!

The forecast files of the HRM in GRIB1 format are named "hif......" for the initialized analysis
and "hff........." for the forecast steps. These files are written into the directory hrm/d00, too.

For multiprocessing on shared memory systems, i.e. for running HRM on more than one proc-
essor, the environment variable OMP_NUM_THREADS has to be set, e.g.
⊲ export OMP_NUM_THREADS=12
to use 12 processors in parallel. Of course, the product of nproc1 x nproc2 must be set to the
same number.

On distributed memory systems (communication is done via calls of MPI routines) the syntax
to use more than one processor may depend on the local installation; contact the system adminis-
trator! And do not forget to set lmpi = .true. in Namelist / hrm_ctl /


  Visualisation of the HRM results based on GrADS
                                               - 89 -


Dr. Helmut Frank of DWD (e-mail: Helmut.Frank@dwd.de) wrote a PERL program (gme2ctl.pl) to
create GrADS ctl files based on the HRM GRIB output files.

To install the wgrib binary do the following steps

⊲ cd wgrib_dwd

Compile and link the binary wgrib (which can handle DWD‘s local GRIB tables (201, 202, 203
and 204))
⊲ cc –o wgrib *.c

Now look for the original wgrib file which was given when you installed GrADS software. It
may be under /usr/local/bin. Now replace it by the one linked in the steps above.

⊲ cp wgrib /usr/local/bin/wgrib

Create the ctl files corresponding to the HRM GRIB output files (fields on model and pressure
levels will have separate ctl files!)

⊲ gme2ctl.pl -v $PATH/hff*0 $PATH/hif*0

⊲ gme2ctl.pl -v $PATH/hff*p $PATH/hif*p

where $PATH ist the directory path of the GRIB files of the HRM (forecast plus initialised
analysis).

Now you can start the GrADS session.

You will find the idx- and ctl-files in the current working directory. All forecast steps of the
model (e.g. at hourly or three-hourly intervals) may be displayed by setting the time index in the
GrADS session properly, e.g. set t 1: initialised analysis; set t 2: 1-hour (or 3-hour) forecast, etc.
  Binary “hmx2hmy”
If you want to nest a higher resolution HRM area in a coarser domain, you also need to install
the hmx2hmy binary.
⊲ cd hmx2hmy_2.x

The files "Options" and "LinkLibs" contain the proper settings for xlf, ifc and pg compiler. Just
remove the "#" in front of the corresponding lines.

⊲ vi Options (and LinkLibs)

Now "make" the object files and the binary

⊲ make

In the directory "../../lib" you will find the library "libhmx2hmy.a".
In the directory "../../bin" you will find the binary "hmx2hmy".
                                             - 90 -


12.5 HRM on GNU/Linux Clusters
By Jerremeo Rainier GABÁS {tagayakal@gmail.com} and
Raquel FRANCISCO, Ph.D. {raquelfrancisco2001@yahoo.com}

This section details the procedures used to compile and run HRM on a GNU/Linux cluster, the
possible problems that you may encounter, and the steps that you need to take when you encoun-
ter these problems.

Introduction
The Philippine Atmospheric, Geophysical and Astronomical Services Administration (PA-
GASA) runs the HRM operationally on two clusters. The old cluster is located in the Weather
and Flood Forecasting Center of PAGASA. The new one is at the Advanced Science and Tech-
nology Institute, a sister-agency of PAGASA, that is located some three kilometers from the
PAGASA campus.

Operating System
The older cluster is based on Debian Sarge, while the newer cluster is based Rocks 5 (based on
CentOS 5). Using Debian results in a fast and responsive cluster, but takes much time to install
and maintain because every node is maintained separately. Rocks is a GNU/Linux distribution
that is made to make cluster management easy, so that computing nodes can be removed and
added with minimal overhead, and is ideal for production cluster use. We recommend Rocks for
new computing cluster deployments.

Software Requirements
Other software that needed are:
   - Intel Fortran Compiler (www.intel.com/cd/software/products/asmo-na/eng/282048.htm)
       or
       PGI Fortran Compiler (www.pgroup.com/products/workindex.htm)
   - MPICH2 (www.mcs.anl.gov/mpi/mpich)
   - compat-libstdc++ (installed using yum or apt-get)

For Debian-based systems, you must install some additional software using apt-get:
   - ia32-libs
   - alien

Compilation
Before compiling an MPI-enabled HRM, an MPI library must be present in the system. There
are commercially-available MPI libraries as well as free and open-source versions. The Argonne
National Laboratory distributes one of the most popular MPI library called MPICH, and its
predecessor MPICH2. Whichever MPI library version is used, it must be compatible with the
Fortran compiler that we will use to compile other HRM binaries and libraries.

PAGASA uses th MPICH2 library that is compiled using the same Fortran compiler that will be
used to compile the other HRM binaries and libraries.

Compiling HRM libraries and binaries follow the same procedures as the normal HRM installa-
tion, except for the actual hrm binary (found in folders hrm_2.x). Note that 32-bit and 64-bit use
different GRIB1 libraries (grib1 and newgrib1, respectively). Please adjust accordingly accord-
ing to your machine architecture.
                                           - 91 -




Before we compile hrm (hrm_2.x), we need to edit two files — Options and LinkLibs — to refer
to the MPI-enabled compiler and libraries.

Options should have settings:

  FTNCALL = mpif90 -I$(SRCDIR) -tpp7 -xW
  FTNOPTS = -O3 -80 -cm -w
  FTNOPTS2 = -O3 -80 -cm -w
  FTNOPTS3 = -O2 -80 -cm -w
  FTNOPTS4 =
  LINKOPTS =

LinkLibs should have these lines:

  LIBS = -lmath_dwd \
     -lgrib1 \
     -lsvml \
     -lmpich

  LIBPATHS = \
       -L. \
       -L../../lib \
       -L/usr/lib64

  EXTOBJ =

These settings apply for MPICH2 that has been compiled by Intel Fortran Compiler for a 64-bit
AMD64-based computer.

Best Practices
To avoid confusion between single-processor and MPI-enabled versions, put the binaries and
configurations in separate directories.

Frequently-Asked Questions
The multiprocessor run gives a “large Courant number” error, but when used with a single
processor mode, the error disappears. How do I get rid of this error?

The problem appears when using IFC 10 with MPICH2 due to the wrong compiler optimization
for gen_vert.f subroutine which computes the wrong eigenvalues for the SI scheme.
The solution is to compile the gen_vert.f subroutine with no optimization (-O0). First, we need
to modify two files in hrm_2.x directory. The first file will be the Options file.
FTNOPTS4 = -O0
Next, we modify the ODependencies file. Replace the compiler option for gen_vert.f (initially
set to FTNOPTS) with FTNOPTS4. Remove the gen_vert.o file and start "make" again to create
a new binary.
      $(OBJDIR)/gen_vert.o: $(SRCDIR)/gen_vert.f \
      $(SRCDIR)/comconst.h $(SRCDIR)/comdyn.h \
      $(SRCDIR)/comorg.h $(SRCDIR)/units.h
      cd $(OBJDIR) && $(FTN) $(FTNOPTS4) (SRCDIR)/gen_vert.f
                                              - 92 -


I have compiled HRM using mpif90, but when I run it using mpiexec (or mpdrun) but it
gives me an error “You must run mpi with option -np ...”.

Please open your LinkLibs file, and check that your EXTOBJ is set to blank. The dummy_mpi.o
is only used when compiling using the native Fortran compiler (ifort, pgf90) for single processor
use.

I encountered an error regarding ld.so.conf, how do I solve it?

You need to add the MPI and Fortran libraries to your /etc/ld.so.conf file.

   echo ―/opt/intel/fce/9.0/lib‖ >> /etc/ld.so.conf
   echo ―/opt/mpich2/intel/9.0/lib‖ >> /etc/ld.so.conf
   ldconfig

Compilation encounters an ―IPO Error: unresolved: vmlsExp4‖ followed by other IPO errors.

   LIBPATHS = -L../../lib

   LIBS = -lsupplement \
      -lmath_dwd \
      -lgrib1 \
      -lsvml

I encounter segmentation faults when running GME2HRM, HRM and HMX2HMY. How
do I solve this?

Your PC memory was exhausted. Try to set it using ulimit.

ulimit -s unlimited

This command is necessary for running the HRM binaries, and it is better to run a bash file that
invokes both ulimit and the binary. For example:

   #!/bin/bash
   ulimit -s unlimited
   ../../bin/hrm

Note: As documented in the HRM manual, this command is dangerous as it could easily lead to
memory leaks. The maximum amount is not really ―unlimited‖ but is limited by both swap and
memory available; increase swap space or memory if needed. If problems persists, there could
be an error in your namelist file, or the input data itself. Please check if they are valid.

I have copied the HRM binary to all nodes, but it still gives me an error. What do I do?

You need to mount the HRM directory on all nodes using NFS (Network File System), and then
run the mpiexec command from there. If you are using Rocks, the user directory is
automatically mounted on the other nodes.

Updates to this document can be found online. A step-by-step installation for a Rocks-based
cluster is also available.
                                            - 93 -


13. Current Users of the HRM
The following list contains all active users of the HRM system. The users are strongly encour-
aged to get into contact via e-mail, and exchange ideas and software as freely as possible.
A WEB site is available, too, to facilitate the exchange of information:
http://www.met.gov.om/hrm/index.php.
Please help especially new users to make the best out of the HRM system.


Armenia Met. Service

Operational NWP user based on GME data.


Responsible scientist:
Izabella Mkrtchyan            mkrtchyan_mi@yahoo.com


Other group members:


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =        24.0
 endlon       =        56.0
 startlat     =        30.0
 endlat       =        52.0
 ie           =       257
 je           =       177
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       233



Characteristics of computer for the HRM:
Linux PC

Additional software for HRM available to other users:
not yet.

Research interest:
-

Web-page:
                                            - 94 -


Bosnia-Herzegovina Met. Service

Operational NWP user based on GME data.


Responsible scientist:
Kemal Sehbajraktarevic        kemo_seh@yahoo.com


Other group members:
Ibrahim Hadzismailovic        zenica3@gmail.com


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =         8.0
 endlon       =        28.0
 startlat     =        35.0
 endlat       =        55.0
 ie           =       161
 je           =       161
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       221



Characteristics of computer for the HRM:
Linux PC

Additional software for HRM available to other users:
not yet.

Research interest:
-

Web-page:
www.fhmzbih.gov.ba
                                           - 95 -



Botswana Met. Service

Operational NWP user based on GME data.


Responsible scientist:
 Mr. Galebonwe Ramaphane           gramaphane@gov.bw               Senior Meteorologist

Other group members:
 Mr. Oliver Thusang Moses           omoses@gov.bw                  Meteorologist I
 Mr. Charles Masule Molongwane      cmolongwane@gov.bw             Meteorologist I
 Ms. Kgololesego Sally Ndlovu        ksndlovu@gov.bw               Meteorologist II


Model domain (Mesh size: 0.11° ~ 12 km):
 startlon     =       -10.0
 endlon       =        56.0
 startlat     =       -44.0
 endlat       =         0.0
 ie           =       601
 je           =       401
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       226



Characteristics of computer for the HRM:
Linux-PC, 16 Dual Quad Core processors, Infini-band interconnect

Additional software for HRM available to other users:
not yet.

Research interest:
-

Web-page:
                                             - 96 -



Brasilian INMET
Operational NWP user based on GME data.
Responsible scientist:
Reinaldo Bomfim da Silveira     reinaldo.silveira@inmet.gov.br
Other group members:
Alaor Dall´Antonia Jr                alaor.dallantonia@inmet.gov.br
Fabricio Harter                      fabricio.harter@inmet.gov.br
Francisco Alves do Nascimento        francisco.alves@inmet.gov.br
Francisco Quixaba Filho              francisco.quixaba@inmet.gov.br
Gilberto Ricardo Bonatti             gilberto.bonatti@inmet.gov.br
Jose Mauricio Franco Guedes          zemauricio.guedes@inmet.gov.br
Juliana Maria Duarte Mol             juliana.mol@inmet.gov.br
Marcelo Almeida de Amorim            marcelo.amorim@inmet.gov.br
Ricardo Raposo dos Santos            ricardo.raposo@inmet.gov.br
Tatiane Felinto Barbosa              tatiane.felinto@inmet.gov.br

Model Domains:
First model domain for South America (mesh size 0.25 degrees ~28 km):
 startlon     =    -95.0
 endlon       =    -20.0
 startlat     =    -60.0
 endlat       =     15.0
 ie           =    301
 je           =    301
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    205

Second domain for Brazil (Mesh size: 0.0625° ~ 7 km):
 startlon     =    -84.0
 endlon       =    -30.0
 startlat     =    -40.0
 endlat       =     14.0
 ie           =    865
 je           =    865
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    205

Characteristics of computer for the HRM:
HRM is running operationally on 2 SGI - ALTIX3700BX2/ALTIX4700, Intel Itanium,
64/128CPUs, 64bits, LINUX SUSE.

Additional software for HRM to other users:
Scripts for visualization on GrADS software including station data, for meteographs and point
verification.
Scripts in Perl for automation of the operational forecasting;
MOS for selected stations; LAPS interface to HRM;
Research interest:
Verification; data assimilation improving analysis by merging local data source; statistical fore-
casting based on MOS and downscaling; QPF and applications to agriculture.
Web-page: http://www.inmet.gov.br http://meteoweb.inmet.gov.br/ (Meteographs)
http://mbarweb.inmet.gov.br/ (28km) http://mbarweb.inmet.gov.br:8080/hrm7/ (7km)
                                            - 97 -



Brazilian NAVY

 Operational NWP user based on GME data.

Responsible scientist:
Rodrigo Obino                       obino@smm.mil.br

Other group members:
Luiz Claudio                        lclaudio@smm.mil.br
Giovana Araujo                      giovana@smm.mil.br
Nilza Barros                        nilza@smm.mil.br

First model domain (Mesh size: 0.18° ~ 20 km):
 startlon     =    -72.0
 endlon       =    -18.0
 startlat     =    -49.9
 endlat       =     14.9
 ie           =    301
 je           =    361
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    301

Second domain for Antarctica (Mesh size: 0.15° ~ 16 km):
 startlon     =     -22.5
 endlon       =      22.5
 startlat     =     -18.75
 endlat       =      18.75
 ie           =     301
 je           =     251
 pollon       =     104.0
 pollat       =     157.0
 nrbmap       =     309

Nested model domain for the first area (Mesh size: 0.09° ~ 10 km):
 startlon     =    -55.8
 endlon       =    -34.2
 startlat     =    -35.5
 endlat       =     15.7
 ie           =    241
 je           =    221
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    301

Characteristics of computer for the HRM:
SGI Origin 350 (24X600 MHz) R16000, 14 Gbyte main memory.

Additional software for HRM available to other users:
not yet.

Research interest:
Ocean and wave modelling, air-sea interaction and post-processing treatment of HRM output.

Web-page:     http://www.mar.mil.br/dhn/chm/meteo/indexing.htm
                                            - 98 -



Bulgarian Met. Service

Operational NWP user based on GME data.


Responsible scientist:
Dimiter Syrakov             dimiter.syrakov@meteo.bg


Other group members:
Vasko Galabov               vasko.galabov@meteo.bg
Maria Prodanova             maria.prodanova@meteo.bg
Latin Latinov               latin.latinov@meteo.bg


Model domain (Mesh size: 0.0625° ~ 7 km):
 startlon     =     19.0
 endlon       =     31.0
 startlat     =     38.0
 endlat       =     47.0
 ie           =    193
 je           =    145
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    313



Characteristics of computer for the HRM:
SUN workstation and Linux-PC.


Additional software for HRM available to other users:
not yet.

Research interest:
Air pollution and ocean wave modelling.


Web-page:     http://weather.bg/hrm/hrm-en.html
                                             - 99 -


FU-Berlin

Research NWP user based on GME data in delayed mode.


Responsible scientist:
Eberhard Reimer        reimer@zedat.fu-berlin.de

Other group members:
Sahar Sodoudi      sodoudi@zedat.fu-berlin.de


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =     30.0
 endlon       =     80.0
 startlat     =      7.0
 endlat       =     52.0
 ie           =    401
 je           =    361
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    231


Characteristics of computer for the HRM:
IBM p575, Linux cluster.


Additional software for HRM available to other users:
not yet.


Research interest:
Air quality for mega cities, here: Tehran.


Web-page:
                                           - 100 -



Georgia

Operational NWP user based on GME data.


Responsible scientist:
Marine Arabidze        marabidze@environment.ge

Other group members:
Tamar Gobejishvili t.gobejishvili@environment.ge


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =        32.0
 endlon       =        56.0
 startlat     =        33.0
 endlat       =        48.0
 ie           =       193
 je           =       121
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       228


Characteristics of computer for the HRM:
Linux PC.


Additional software for HRM available to other users:
not yet.


Research interest:


Web-page:
                                           - 101 -


India, Space Physics Laboratory

Research NWP user based on GME data.


Responsible scientist:
Radhika Ramachandran:        radhikaspl@gmail.com
D. Bala Subrahamanyam:       db_subramanyam@vssc.gov.in



Other group members:
S. Indira Rani               rani_irs@yahoo.com



Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =     65.0
 endlon       =     95.0
 startlat     =      0.0
 endlat       =     30.0
 ie           =    241
 je           =    241
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    214


Characteristics of computer for the HRM:
Linux cluster.


Additional software for HRM available to other users:
not yet.

Research interest:
Air sea interaction, atmospheric boundary layer, mesoscale modelling, data assimilation
numerical weather prediction, air pollution.

Web-page:
                                           - 102 -


Indonesia Met. Service

Operational NWP user based on GME data.


Responsible scientist:
wido hanggoro [wido_hanggoro@yahoo.com]

Other group members:


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =        91.25
 endlon       =       158.75
 startlat     =       -13.0
 endlat       =        12.0
 ie           =       541
 je           =       201
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       235


Characteristics of computer for the HRM:
Linux cluster.


Additional software for HRM available to other users:
not yet.

Research interest:


Web-page:
                                           - 103 -



Iran (Arak University)

Research NWP user based on GME data in delayed mode.


Responsible scientist:
Saeed Moghimi          moghimis@gmail.com

Other group members:
Omid Alemi         omid.alemi@gmail.com


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =     30.0
 endlon       =     80.0
 startlat     =      7.0
 endlat       =     52.0
 ie           =    401
 je           =    361
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    231


Characteristics of computer for the HRM:
Linux cluster.


Additional software for HRM available to other users:
not yet.


Research interest:
Wave and surge modelling


Web-page: http://80.191.68.232/g4w/outs.html
                                             - 104 -


Israel IMS

Operational NWP user based on GME data.


Responsible scientist:
A. Spectorman          alexs@ims.gov.il



Model domain (Mesh size: 0.125° ~ 14 km)
 startlon      =      2.0
 endlon        =     32.0
 startlat      =    -32.0
 endlat        =     -7.0
 ie            =    241
 je            =    201
 pollon        =   -170.0
 pollat        =     32.5
 nrbmap        =    202



Characteristics of computer for the HRM:
SGI Origin300: 8 processors R12000 (600MHz), main memory size: 2 GBytes.


Additional software for HRM available to other users:
As part of the HRM package, programs for visualising the HRM output for GrADS and
Vis5D.


Research interest:
Air pollution control, rainfall enhancement, local forecasting.


Web-page:      http://www.ims.gov.il/en2.htm
                                           - 105 -



Italian Met. Service

Operational NWP user, based on ECMWF lateral boundary conditions.


Responsible scientist:
Massimo Ferri          m.ferri@ecmwf.int


Other group members:
Lucio Torrisi      l.torrisi@meteoam.it


Model domain (EuroHRM; Mesh size: 0.25° ~ 28 km):
 startlon     =    -56.5
 endlon       =     39.5
 startlat     =    -34.0
 endlat       =     30.0
 ie           =    385
 je           =    257
 pollon       =   -170.0
 pollat       =     32.5
 nrbmap       =   None

Additionally two temporary configurations (relocatable HRM) to support military forces.

Characteristics of computer for the HRM:
Compaq-Alpha GS60, 4 Processors, IBM p675 at ECMWF (Reading, UK)


Additional software for HRM available to other users:
 IFS2HRM (to obtain initial/lateral boundary data fields from the ECMWF model).


Research interest:
Quantitative precipitation forecasting.
3D Variational data assimilation scheme for the HRM.

Web-page:     http://www.meteoam.it/
                                           - 106 -



Jordan Met. Service

Operational NWP user based on GME data.


Responsible scientist:
Malkawi Hisham                hisham712001@yahoo.com

Other group members:


Model domain: (Mesh size: 0.100° ~ 11 km):
 startlon     =        27.0
 endlon       =        43.0
 startlat     =        26.0
 endlat       =        38.0
 ie           =       161
 je           =       121
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =      224



Characteristics of computer for the HRM:
Linux PC.

Additional software for HRM available to other users:
not yet.

Research interest:


Web-page:
                                           - 107 -



Kenya Meteorological Service

Operational NWP user based on GME data.


Responsible scientist:
J. G. Mungai           jmungai@meteo.go.ke



Other group members:
Gathura Gichuki    jggathura@yahoo.com
Vincent N. Sakwa sakwa_v@yahoo.co.uk


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =        26.0
 endlon       =        51.0
 startlat     =       -12.0
 endlat       =        12.0
 ie           =       201
 je           =       193
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       316



Characteristics of computer for the HRM:
Linux PC server.


Additional software for HRM available to other users:
not yet.

Research interest:

Web-page:     http://www.meteo.go.ke/nwp/
                                           - 108 -



Libya Meteorological Service

Operational NWP user based on GME data.


Responsible scientist:
Khaled Ghanpour               khaledelkames@yahoo.com


Other group members:



Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =         5.0
 endlon       =        30.0
 startlat     =        15.0
 endlat       =        40.0
 ie           =       201
 je           =       201
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       222



Characteristics of computer for the HRM:
Linux PC server.


Additional software for HRM available to other users:
not yet.

Research interest:

Web-page:
                                           - 109 -



Madagascar Meteorological Service

Operational NWP user based on GME data.


Responsible scientist:
Luc Yannick Randriamarolaza         doc_luc@yahoo.fr


Other group members:


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =        35.0
 endlon       =        60.0
 startlat     =       -32.5
 endlat       =        -7.5
 ie           =       201
 je           =       201
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       432 (Frame: 032)



Characteristics of computer for the HRM:
Linux PC server.


Additional software for HRM available to other users:
not yet.

Research interest:

Web-page:
                                           - 110 -



Malaysia Met. Service

Operational NWP user based on GME data


Responsible scientist:
Wan Maisarah Wan Ibadullah          wmaisarah@met.gov.my

Other group members:
Ling Leong Kwok                     llk@met.gov.my


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =        85.0
 endlon       =       135.0
 startlat     =       -19.0
 endlat       =        26.0
 ie           =       401
 je           =       361
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       234



Characteristics of computer for the HRM:
Linux-Cluster

Additional software for HRM available to other users:
not yet.

Research interest:


Web-page:
                                            - 111 -



Mauritius Met. Service

Research NWP user based on GME data


Responsible scientist:
Ganessen Virasami                   vganessen@yahoo.com

Other group members:


Model domain (Mesh size: 0.0625° ~ 7 km):
 startlon     =     40.0
 endlon       =     80.0
 startlat     =     40.0
 endlat       =      0.0
 ie           =    641
 je           =    641
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =      -



Characteristics of computer for the HRM:
Linux-Cluster

Additional software for HRM available to other users:
not yet.

Research interest:
Tropical cyclones; land/sea breeze circulation


Web-page:
                                            - 112 -



Mosambique Met. Service

Operational NWP user based on GME data.


Responsible scientist:
Genito Amos Maure (Eduardo Mondlane University in Maputo)     genito.maure@uem.mz


Other group members:
Sérgio Buque                   sergio_b@inam.gov.mz
Antonio Beleza                 antonio.beleza@gmail.com
Queiroz Alberto                queiroz_a@inam.gov.mz

Model domain (Mesh size: 0.11° ~ 12 km):
 startlon     =        15.96
 endlon       =        59.96
 startlat     =       -44.98
 endlat       =        -5.60
 ie           =       401
 je           =       359
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       320



Characteristics of computer for the HRM:
Dell PowerEdge 2950 QuadCore Xeon Processor Server (Eduardo Mondlane University)
8 Dual-Core AMD Opteron node Linux Networx cluster (Met Services)

Additional software for HRM available to other users:
not yet.

Research interest:

Web-page:
                                            - 113 -



Oman DGCAM

Operational NWP user based on GME data.


Responsible scientist:
Sultan Al-Yahyai       s.alyahyai@met.gov.om


Other group members:
Fauzi Al-Busaidi   f.albusaidi@met.gov.om
Khalid Al-Jahwari k.aljahwari@met.gov.om


Model domain (Mesh size 0.0625° ~ 7 km):
 startlon     =     30.0
 endlon       =     78.0
 startlat     =      7.0
 endlat       =     35.25
 ie           =    769
 je           =    453
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    204



Characteristics of computer for the HRM:
- Linux PC Cluster (Opteron AMD 576 cores with InfiniBand Interconnection)


Additional software for HRM available to other users:
- Portable HRM Station Value Verification Package "ORMVERIF"
- Operations scripts (ksh & grads) to create dynamic images for the website from HRM output.


Research interest:
Land-sea breeze circulation, precipitation forecasting.

Web-page:     http://www.met.gov.om/mod.php3
                                           - 114 -



Pakistan Met. Service

Operational NWP user based on GME data.


Responsible scientist:
Ghulam Rasul                grmet@yahoo.com


Other group members:
Afzaal Karori               afzaalkarori@yahoo.com
Jehangir Ashraf Awan        jehangir_awan@hotmail.com


Model domain (Mesh size 0.20° ~ 22 km):
 startlon     =     30.0
 endlon       =    100.0
 startlat     =      5.0
 endlat       =     60.0
 ie           =    351
 je           =    276
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    323




Characteristics of computer for the HRM:
HP Proliant DL380 Rack Mount Servers with
9 Nodes having 3.4 Ghz dual Intel Xeon processros with 4 Gb RAM each.



Additional software for HRM available to other users:
not yet.


Research interest:
Verification, monsoon studies.

Web-page:     http://www.pakmet.com.pk/
                                           - 115 -



The National Meteorological Administration of Romania (NMA Romania)

Operational NWP user based on GME data.


Responsible scientist:
Cosmin Barbu                cosmin.barbu@meteoromania.ro


Other group members:
Rodica Dumitrache           rodica.dumitrache@meteoromania.ro
Mihaela Doicin              mihaela.doicin@meteoromania.ro


Model domain (Mesh size: 0.125° ~ 14 km):
 startlon     =     -2.0
 endlon       =     25.0
 startlat     =    -26.0
 endlat       =      2.5
 ie           =    217
 je           =    229
 pollon       =   -170.0
 pollat       =     32.5
 nrbmap       =    310


Characteristics of computer for the HRM:
 Linux IBM Cluster; 14 Server blades with 2 processors Intel Xeon Quad 3.00 GHz on each
blade; Operating System: Red Hat Enterprise Linux 5; Connectivity: 10Gb Ethernet; Compilers:
PGI, Intel (Fortran, C, C++); Additional software: MPI, GPS; Storage Unit: 4.2 TB

observations: for the HRM integration are used only 44 cores.

Additional software for HRM available to other users:
GrADS package and GrADS scripts.


Research interest:
Regional application of the HRM with grid mesh of 7 km,
Case studies of heavy precipitation,
Driving model for INPUFF model for estimation of the diffusion and long range transport of
pollutants,
Coupling with hydrological models,
Coupling with wave models.

Web-page:     http://www.meteoromania.ro/
              http://www.meteoromania.ro/index.php?id=0&lang=en (English version)
                                            - 116 -



Spanish Meteorological Service (INM)

Operational NWP user based on GME, GFS (NCEP, USA), UM (UKMO, UK) and ECMWF
(UK) data (part of a multi-model, multi-analyses, multi-lateral- boundary-conditions ensemble).
HRM is using lateral boundary conditions from GME, GFS, UM and ECMWF models.

Responsible scientist
José A. García-Moya          png@inm.es


Other group members:
Pau Escriba                  pescriba@inm.es
Juan Simarro                 simarro@inm.es

Model domain (Mesh size: 0.25° ~ 28 km):
 startlon     =    -48.00
 endlon       =     48.00
 startlat     =    -35.50
 endlat       =     32.25
 ie           =    385
 je           =    272
 pollon       =    165.0
 pollat       =     35.0
 nrbmap       =    411



Characteristics of computer for the HRM:
Cray X1.


Additional software for HRM available to other users:
avn2hrm.pack.tar.gz and ecmwf.pack.tar.gz, available at ftp-incoming.dwd.de /download

Research interest:
Regional ensemble prediction.

Web-page:     http://www.aemet.es
                                           - 117 -


Tansania Met. Service

Operational NWP user based on GME data.


Responsible scientist:
Agnes Kijazi                  akijazi2000@yahoo.co.uk


Other group members:
Peter Nicky Mlonganile        pmlonganile@yahoo.co.uk

Model domain (Mesh size 0.25° ~ 28 km):
 startlon     =        20.0
 endlon       =        60.0
 startlat     =       -40.0
 endlat       =        20.0
 ie           =       161
 je           =       241
 pollon       =      -180.0
 pollat       =        90.0
 nrbmap       =       227



Characteristics of computer for the HRM:
Linux PC.


Additional software for HRM available to other users:
not yet.


Research interest:


Web-page:
                                            - 118 -



United Arab Emirates, National Meteorological Service

Operational NWP user based on GME data.


Responsible scientist:
Taha El-Hosary         taha_nagy@excite.com


Other group members:
Abdeltawab Shalaby AShalaby@ncms.ae
Majed Naser        majed@uaemet.gov.ae

Model domain I (Mesh size 0.25° ~ 28 km):
 startlon     =     15.0
 endlon       =     75.0
 startlat     =      6.0
 endlat       =     42.0
 ie           =    241
 je           =    145
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    317

Model domain II (Mesh size 0.0625° ~ 7 km):
 startlon     =     48.5
 endlon       =     63.5
 startlat     =     14.0
 endlat       =     29.0
 ie           =    241
 je           =    241
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    317



Characteristics of computer for the HRM:
Linux Networx Xeon-Cluster, (10+1) processors.


Additional software for HRM available to other users:
not yet.


Research interest:
Land-sea breeze circulation, precipitation forecasting.
                                            - 119 -



Vietnam, National Hydrometeorological Service and University of Hanoi

Operational NWP user based on GME data.

Responsible scientists
DoLe Thuy              met_research@fpt.vn
Kieu thi Xin           xinkt@vnuh.edu.vn



Other group members:
Phan Van Tan       tanpv@vnuh.edu.vn
Le Duc             leducvn@yahoo.com


Model domain I (Mesh size: 0.25° ~ 28 km):
 startlon     =     80.25
 endlon       =    130.25
 startlat     =     -5.0
 endlat       =     35.0
 ie           =    201
 je           =    161
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    215

Model domain II (Mesh size: 0.125° ~ 14 km):
 startlon     =     97.25
 endlon       =    127.25
 startlat     =      7.125
 endlat       =     27.125
 ie           =    241
 je           =    161
 pollon       =   -180.0
 pollat       =     90.0
 nrbmap       =    215

Characteristics of computer for the HRM:
Linux PC Cluster with 4x2 processors at National Hydrometeorological Service.
SUN 4-Processor workstation and Liunx PCs at University of Hanoi.


Additional software for HRM available to other users:
not yet.


Research interest:
Data assimilation (3D-Var), typhoon forecasting, quantitative precipitation forecasting.
                                           - 120 -



ETH Zürich, Klimaforschung und Atmosphärenphysik (University)

Research in regional climate modelling


Responsible scientist:
Christoph Schaer                    schaer@geo.umnw.ethz.ch

Other group members:
vidale@geo.umnw.ethz.ch             PierLuigi Vidale
luethi@atmos.umnw.ethz.ch           Daniel Luethi
kleinn@geo.umnw.ethz.ch             Jan Kleinn
heck@geo.umnw.ethz.ch               Pamela Heck
walser@geo.umnw.ethz.ch             Andre Walser
eneviratne@geo.umnw.ethz.ch         Sonia Eneviratne
wernli@atmos.umnw.ethz.ch           Heini Wernli
sbuss@atmos.umnw.ethz.ch            Sandro Buss
sdirren@atmos.umnw.ethz.ch          Sebastien Dirren

Also see: http://www.geo.umnw.ethz.ch/staff/homepages/vidale/hrm/HRM.html

Model domain 1 (Δ~ 56 km):
 startlon     =    -19.5
 endlon       =     20.5
 startlat     =    -25.
 endlat       =     20.
 ie           =     81
 je           =     91
 pollon       =   -170
 pollat       =     32.5

Model domain 2 (Δ~ 14 km, nested):
 startlon     =     -9.75
 endlon       =      3.75
 startlat     =    -15.5
 endlat       =     -3.0
 ie           =    109
 je           =    101
 pollon       =   -170.
 pollat       =     32.5


Characteristics of computer for the HRM:
8-16 processor CRAYs (SV1-Bs), Sun Ultra 30-60s.

Additional software for HRM available to other users:
not yet.

Research interests:
 Multi-year Regional Climate Modelling for the European region, with special emphasis on the
 study of the inter-annual variability in the hydrological cycle. Links with hydrological model-
 ling and land surface processes.
                                                - 121 -


14. References
Bechtold, P.; Bazile, E.; Guichard, F.; Mascart, P. and E. Richard, 2001: A mass-flux convection
       scheme for regional and global models. Q. R. J. Meteorol. Soc., Vol. 127, 869-886.

Burridge, D. M., 1975: A split semi-implicit reformulation of the Bushby-Timpson 10-Level model.
       Quart. J. Roy. Meteor. Soc. 101, 430, 777-792.

Davies, H. C., 1976: A lateral boundary formulation for multi-level prediction models.
        Quart. J. Roy. Meteor. Soc. 102, 432, 405-418.

Doms, G. and U. Schättler, 2003: The nonhydrostatic limited-area model LM of DWD. Part 1: Scientific
      documentation. Deutscher Wetterdienst, Offenbach, Germany.

Heise, H and R. Schrodin, 2002: Aspects of snow and soil modelling in the operational short range
       weather prediction models of the German Weather Service. Journal of Computational Technolo-
       gies, Vol. 7, Special Issue: Proceedings of the International Conference on Modelling, Databases
       and Information Systems for Atmospheric Science (MODAS), Irkutsk, Russia, June 25-29, 2001,
       121-140.

Herzog, H., 1995: Testing a radiative upper boundary condition in a nonlinear model with hybrid
       vertical coordinate. Meteorology and atmospheric physics, 55, 185-204.

Lott, F. and M. Miller, 1997: A new sub-grid scale orographic drag parameterization: its formulation and
          testing. Quart. J. Roy. Meteor. Soc., 123, 101-128.

Louis, J.-F., 1979: A parametric model of vertical eddy fluxes in the atmosphere.
        Boundary layer Meteor., 17, 187-202.

Lynch, P., 1997: The Dolph-Chebyshev window: A simple optimal filter. Mon. Wea. Rev., 125, 655-660.

Majewski, D., D. Liermann, P. Prohl, B. Ritter, M. Buchhold, T. Hanisch, G. Paul, W. Wergen and
      J. Baumgardner, 2002: The operational global icosahedral-hexagonal gridpoint model GME: De-
      scription and high-resolution tests. Mon. Wea. Rev., 130, 319-338.

Mellor, G. L. and T. Yamada, 1974: A hierarchy of turbulent closure models for planetary boundary
        layers. J. Atmos. Sci., 31, 1791-1806.

Mironov, D. and B. Ritter, 2003: A first version of the ice model for the global NWP system GME of the
      German Weather Service. WGNE Blue Book.

Mironov, D., and B. Ritter, 2004: A New Sea Ice Model for GME. Technical Note, Deutscher Wetter-
      dienst, Offenbach am Main, Germany, 12 pp.
      http://nwpi.krc.karelia.ru/flake/papers/MR_SeaIce_GME_DWDintern.ps

Ritter, B. and J. F. Geleyn, 1992: A comprehensive radiation scheme for numerical weather prediction
         models with potential applications in climate simulations. Mon. Wea. Rev., 120, 303-325.

Simmons, A. J. and D. M. Burridge, 1981: An energy and angular-momentum conserving vertical
      finite-difference scheme and hybrid vertical coordinates. Mon. Wea. Rev., 109, 758-766.

Temperton, C. and M. Roch, 1991: Implicit normal mode initialization for an operational regional model.
      Mon. Wea. Rev., 119, 667-677.

Tiedtke, M., 1989: A comprehensive mass flux scheme for cumulus parameterization in large scale
        models. Mon. Wea. Rev., 117, 1779-1800.

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:53
posted:11/16/2011
language:English
pages:121