ESMF 3rd Community Meeting, Boulder, July 15, 2004
PRISM
An Infrastructure Project for Climate
Research in Europe
by Nils Wedi @ ECMWF
Contributions by A. Caubel, P. Constanza, D. Declat, J.
Latour, V. Gayler, E. Guilyardi, C. Larsson, S. Legutke, R.
Redler, H. Ritzdorf, T. Schoenemeyer, S. Valcke, R.
Vogelsang and many others in the PRISM community ...
Overview
• What is PRISM?
• European Interest, partners
• PRISM Objectives
• PRISM Model Components
• PRISM Approach
• Architecture and User Interface
• Current Status and Perspective
What is PRISM?
• PRogram for Integrated Earth System Modelling
• A European project for climate modelling
involving 22 partners, 12/2001 – 12/2004
• Funded by the European Commission (4.8 M€)
• Involves state-of-the-art atmosphere, ocean, sea-
ice, atmospheric chemistry, land-surface and
ocean-biogeochemistry models
PRISM partners
• MPI-M, Germany (Guy • CSCS/ETH, Switzerland
Brasseur, coordinator) • INGV, Italy
• KNMI, The Netherlands • MPI-BGC, Germany
(Gerbrand Komen, co-
coordinator) • PIK, Germany
• MPI-M&D, Germany • ECMWF
• MetOffice, United Kingdom • UCL-ASTR, Belgium
• UREADMY, United Kingdom • NEC-ESS, Germany
• IPSL, France • FECIT/Fujitsu, France
• Météo-France, France • SGI, Germany
• CERFACS, France • SUN, Germany
• DMI, Denmark
• NEC-CCRLE, Germany
• SHMI, Sweden
• NERSC, Norway
Help scientists to spend more time
on science
Provide software infrastructure to
• easily assemble earth system coupled models based on
existing state-of-art components models
• launch/monitor complex/ensembles earth system
simulations
• access, analyse and share results across wide
community
Define and promote technical and scientific standards for
Earth System modelling
Technical and scientific standards
• Scientific:
Global parameters
Physical interfaces
• Technical:
Coupler and I/O
Data format and grids
Architecture and User Interface
Diagnostics and visualization
Coding and quality
PRISM model components
Atmosphere:
Météo-France (ARPEGE), MPG-IMET Atmospheric Chemistry:
(ECHAM), IPSL (LMDZ), MetOffice MPG-IMET, UREADMY, IPSL,
(Unified Model), UREADMY, INGV MetOffice, Météo-France, KNMI
Land Surface:
IPSL (Orchidée), MetOffice, MPG-
Coupler: Regional Climate:
CERFACS, NEC, SHMI, DMI, MetOffice
IMET, UREADMY, Météo-France
CCRLE, FECIT, SGI,
(ISBA)
MPI-MAD
Sea Ice:
Ocean NERSC, UCL-ASTR,
Biogeochemistry: MetOffice, IPSL, MPG-IMET
MPI-BGC, IPSL, MPG- Ocean:
IMET, MetOffice UREADMY, MetOffice
(FOAM), MPI-M (HOPE),
IPSL (OPA/ORCA)
ESMF - PRISM
Running environment
PRISM
Superstructure
ESMF User code
Code Infrastructure
Coupling software and its
evolution in PRISM
• Coupler: OASIS 3.0/4.0 (~10 years
experience)
• Prism system model interface library:
PSMILe
• MPI1 or MPI2 direct communication
between models with same grid otherwise
repartitioning using a transformer
• modularity: prism_put() and prism_get() to
implement in existing models
OASIS coupler: Ocean Atmosphere
Sea Ice Soil
Historical review: Developed since 1991 in CERFACS to couple
existing GCMs. At the time:
•Models at relatively low resolution (~10000-20000 pts)
•Small number of 2D coupling fields (~10)
•Low coupling frequency (~once/day)
flexibility was very important, efficiency not so much!
1991 2001
|-- |--- PRISM
OASIS 1 OASIS 2 OASIS3
OASIS4
OASIS community
•CGAM-Reading (UK) HadAM3 - ORCA2
•Southampton University (UK) Inter. Atm - OCCAM lite
•UCL (Belgium) LMDz - CLIO
•SMHI (Sweden) ECHAM - RCA RCA(region.) – RCO(region.)
•U. of Bergen (Norway) MM5 - ROMS
•KNMI (Netherlands) ECHAM5 - MPI-OM
•DMI (Danemark) ECHAM - HIRLAM
•INGV (Italy) ECHAM5 – MPI-OM
•IRI (USA) ECHAM4 - MOM3
•JAMSTEC (Japan) ECHAM4 - OPA 8.2
•BMRC (Australia) BAM - MOM4 BAM3 - ACOM2
•U. of Tasmania (Australia) Data Atm. - MOM4
•CAS,IIT Delhi (India) MM5 - POM
OASIS community
•CERFACS (France) ARPEGE3 - ORCA2LIM
ARPEGE3 - OPA 8.1 ARPEGE3 - OPAICE
•METEO-FRANCE (France)
ARPEGE4 - ORCA2 ARPEGE medias -OPAmed
ARPEGE3 - OPA8.1 ARPEGE2 - OPA TDH
•IPSL- LODYC, LMD, LSCE (France) LMDz - ORCA2LIM
LMDz - ORCA4 LMDz - OPA ATL3/ATL1
IFS - OPA 8.1 ECHAM4 - ORCA2
•MERCATOR (France) PAM(OPA)
•MPI - M&D (Germany)
ECHAM5 - MPI-OM ECHAM5 - C-HOPE
PUMA - C-HOPE EMAD - E-HOPE
ECHAM5 - E-HOPE ECHAM4 - E-HOPE
•ECMWF (UK)
IFS Cy23r4 - E-HOPE IFS Cy15r8 - E-HOPE
Oasis2 and Oasis3
O
A
•Mono-process coupler O
A
•2D scalar coupling fields interpolation (SCRIP1.4) Oasis3 O
A
•PRISM System Model Interface Library PSMILe O
• coupling fields exchange (MPI1 & MPI2) A
• I/O actions (GFDL mpp_io) A file
A
Flexibility, modularity:
Coupler and PSMILe act according to user-defined coupling
configuration (text file):
• number of models and coupling fields
• coupling frequencies and transformations for each field
• I/O or coupling mode (transparent for model)
Oasis4 – new demands
• Higher resolution, parallel and scalable models
• Higher coupling frequencies desirable
• Higher number of models and (3D) coupling
fields
• Massively parallel platforms
Need to optimise and parallelise the coupler
OASIS4
is composed of:
• a Driver
• a Transformer
• a new PRISM System Model Interface Library
• Interface and data
flow
OASIS4
•MPI parallel communication including repartitioning OB C
•parallel multigrid 3D neighbourhood search and OB C
C O1
calculation of communication patterns in each source
process PSMILe OB C C O1
•extraction of useful part of source field only
•parallel I/O: single file, distributed files: GFDL OB C
mpp_io parallel file: parNetCDF C O2
T
OB C
•parallel Transformer: loops over PSMILe requests
C O2
•flexibility and modularity same as Oasis3 OB C
ATM-LAND AD
Driver user
user ATM PMIOD
OCE AD V1: in, metadata V1
V2: out, metadata V2
OCE PMIOD V3: out, metadata V3
SCC
V1: out, metadata V1 ATM:...
V2: in, metadata V2 OCE:...
user
LAND:... ATM SMIOC
user V1 : from OCE, T1
V2: to OCE, T2
OCE SMIOC ATM V3 : to LAND
V1 : to ATM, T1 V2
to file V1 OCE V2
V3
V2 : from ATM, T2 T V1
LAND
V1
LAND PMIOD
V3: in, metadata V3
V4 V4: in, metadata V4
fileV1
Definition Phase fileV4 user
LAND SMIOC
Composition Phase V3 : from ATM
V4 : from fileV4
Deployment Phase
Oasis – Current status
•OASIS3_prism_2-2 available
•OASIS4 prototype available
•OASIS4 final PRISM version due 12/2004
File formats and grids
• NetCDF for grid and restart auxiliary files
• CF convention under development,
extending the COARDS conventions
• XML for model and script meta-data input
(Fortran namelist and shell replacement)
System Architecture and User
Interface
PRISM architecture to provide an efficient climate
modelling infrastructure to users and developers
through:
• Standardised interfaces
• Remote functionality
• Centralised administration
• Distributed resources
Standard compile (SCE) and run
environment (SRE)
• Finalizing the SCE and SRE for the PRISM models
• The system comprises 15 models (arpege_climat4
echam5 hamocc lim lmdz mozart mpi-om oasis3
opa orchidee pisces toy4opa toyatm toyche toyoce)
which are adapted to a varying degree to the PRISM
standards and can run in several combinations.
• Most of the models have been tested on a variety of
platforms (NEC (SX), SGI (MIPS or IA64SGI),
FUJITSU (VPP), IBM (power4))
A tedious task without flashy graphics but very useful !!!
PrepIFS
PrepIFS – earth system modeling
via the Internet …
OASIS4 –
GUI support
XML designed
to be read by machines
not humans !!!
SMS/WebCdp job scheduling
and monitoring
• Complex automated
scheduling
• Macro-parallelism
• Flexible inter-
dependencies
• Interactive control
• Visual structure of
large systems
• Used for all
operational and
research activities at
ECMWF ( ~10 years)
Further tools …
• Diagnostics – Prism processing and
visualization software (COCO, CDAT/VCS
and VTK), at ECMWF MARS/Vis5D/Metview
• Web GUI – database and diagnostics web
interface using web-access server
technology, (DODS and LAS), at ECMWF
Web-MARS
A PRISM sustained team
•Document by PRISM Steering Group proposing establishment
of a PRISM sustained team of 7 people sent to European Climate
modelling Community (June 04)
•First preparation meeting: August 17th, 2004
•Target: Signature of Consortium Agreement: 01/2005
•MPI (Germany), CERFACS (France), ECMWF (EU), CNRS (France),
MetOffice (UK), NCAS(UK), CCRLE (Germany) already expressed strong
interest.
•Additional FP6 funding (March 2005)?
Enthusiasm is still high !!!
Further information
http://prism.enes.org
Oasis4: in the prototype (05/2004)
•Access and use of XML information
•Coupling and I/O of n parallel applications with m components.
•Coupling exchange with repartitioning, direct or through the
Transformer
•Interpolations:
•PSMILe (non-exact) parallel neighbourhood search
•3D 2D nearest-neighbour, 3D linear, 2D linear
•I/O:
•single and parallel mode
•Coupling and I/O exchange from one source to many targets
•Local transformations (scatter, gather, add or mult scalar, statistics)
•Basic time transformation (average, accumul, min, max)
Oasis4: still to be done
•PSMILe API for model access to SCC and SMIOC information
•Interpolation:
•More schemes (conservative, 3D, etc.)
•Exact parallel neighbour search
•Transformer parallelisation (almost completed)
•Field reduction, combination
•Full support of vector and bundle fields (I/O OK)
•I/O: distributed mode (parNetCDF)
•Adaptive grids
•Unstructured grids