construction scheme of meteorological application grid portal

Document Sample
construction scheme of meteorological application grid portal Powered By Docstoc
					Construction Scheme of Meteorological Application Grid Portal1
Weimin Zhang National Laboratory for Parallel and Distributed Processing,changsha,hunan 410073,China Xuesheng Yang,Dehui Chen Chinese Academy of Meteorological Sciences



Numerical weather prediction operational system is strongly dependant on computer power. Although China Meteorological Administration including NMC, CAMS, NMSC and regional meteorological centers, installed some supercomputers during the past few years, such as IBM SP, COMPAQ, Origin, Galaxy-III, SW-1, DAWN 2000 etc., the current computing capability and communication resource still can’t satisfy the increasing service demand and the new developing NWP system, especially the climate model system. With the advent of the grid computing, we can get substantial benefits by implementing grids in the existed operational NWP system and research environment, for example, it can aggregate the geographically distributed computing resources and give end users remote access to the computing, data and storage resources they need, can establish a Virtual Research Center and a uniform platform which make good use of expertise to develop the new NWP system, can accelerate the procedure from research to operation and can help solve problems that were previously unsolvable. Some progress of grid application in meteorology has been made during the past few years, especially the ECACCESS[1] system of European Center for Medium-range Weather Forecast. NASA develop the modeling environment for atmospheric discovery(MEAD)[2] . After thoroughly investigation of the status and outlook of operational NWP system and research environment, Meteorological Application Grid (MAG) is proposed based on the grid computing. Grid computing enables the virtualization of distributed computing and data resource such as processing, network bandwidth and storage capacity to create a single system image, granting users and applications seamless access to vast IT capabilities.Grid portal is the administrative interface of MAG.

2 Construction scheme of the MAG portal
Totally three portals will be constructed, which include Resource Access Portal (RAP), Product Announcement Portal (PAP) and Research Collaborative Portal (RCP). In order to manage the resource and service in the grid, a grid administrative node was introduced, which is used to manage the account, service and resource. Account management includes account registration, priority and authentication. Service management includes registration of the service, authorization and fees. Resource management consists of the registration of the computing node, meteorological data management and node monitoring etc. All these application was completed by the Service Domain (SD), which is one of the
This work is sponsored by National 863 Project of Department of Science and Technology under Contract Number 2002AA104210.

PDF created with FinePrint pdfFactory trial version

IBM grid software. Client’s service request was sent from portal to SD, and SD select the corresponding application service to satisfy user’s request, such as offering the NWP products, downloading NWP source codes, uploading meteorological modules, constructing the experimental workflow. SD and Service Provisioning Management (SPM) can complete the service balance, engagement, authorization and log functions. In the early stage, only one SD will be installed, but when more resource, service and client are involved, one SD may become a bottleneck and more SD will be installed.

2.1 Resource Access Portal(RAP)
RAP provides an approach to access grid resources via Web browser for internal users, and reports status of grid resource, includes: · user management: application and cancellation of user account · remote command: such as telnet, file and directory command, compilation and link command · queue job management: submission, deletion and monitoring of the job, queue · resource information report: configuration of resource, status of resource, queue, job, CPU, etc. · grid bulletin: such as resource maintenance agenda, new resource introduction, etc. · grid consulting: resource documentation and consulting RAP uses a single-sign-on to access all the grid resource distributed in Beijing, Shanghai, etc.

2.2 Product Announcement Portal (PAP)
PAP disseminates its service to the public and professional users via Web browser. User can submit its service request via PAP, then PAP passes this request to IBM SD(Service Domain), and SD returns the appropriate service according user’s priority. For public request, the service is returned from data set directly in the image format and no additional calculation needed. While for professional users, he can choose desired NWP workflow according to his request and identity, and the results can be returned in the format of image or data. For administrator user, he can perform user management, grid internal service management, etc. MAG will provide the product announcement services for the public user and professional users with web page, E-mail and Web service, as well as provide the capability for remote data access, visual image visiting and services. Public consumers which can get the services from appointed Web website. Professional customers can order the forecast service, and get the services via the user designated requests, via the Web service customer's software, E-mail, etc., or via the executable procedures.

2.3 Remote Collaborative Portal (RCP)
Researchers often want to aggregate not only data and computing power, but also human expertise. For example, a meteorologist who has performed a climate simulation might want colleagues around the world to visualize the results in the same way and at the same time so that the group can discuss the results in real time. RCP was designed to manage the development of NWP modules and experimental workflow. Users can do research with RCP to develop and improve the NWP system. A web-oriented interface was designed to access RCP. With this interface, users can access all the resource within MAG., include NWP models, data, graphics, etc. They can input initial data, select desired model parameters and modules to submit NWP jobs, and to browse and modify NWP modules. Figure 1 shows the flowchart of RCP.

PDF created with FinePrint pdfFactory trial version

Model developers can get the updated source codes and make improvement, and can test them with other modules. They can also define new modules in the system and describe its implement environmental requirements. Internal users can select the registered NWP modules and define his own NWP workflow, search module information of the workflow, submit his NWP job and display job status and system status, receive the notification after job completed.
Professional user YH Compaq

Operation user

Resource Registry:

Modular Registration



application workflow


Modular codes

Repository Developer

Figure 1:Flowchart of RCP
Source codes were managed by CVS. For every NWP module, a corresponding web service should be defined, and every web service must define a query interface to describe the function of the module to respond user’s request. Users can choose the given modules and to define his own NWP workflow. NWP workflow can be defined via web, every defined NWP flow can be considered as a combination of web service, every request to the flow can be thought of as a job. Through these combined web service, user’s requests are completed. RCP provides following functions: l Modular register service: users can register a new developed module in the grid and module’s attribute should be described, such as version, hardware request, operation system requirement and other software requirement, etc. l l l l l l l l l l Module announcement: users can announce his ready module in the grid, and install it to the platform. Module search: given the search conditions, the matched modules returned. Data search service: given search conditions, grid returns the matched data information . Workflow definition and edition: users can define its own workflow and the attribute of the modules will be checked to see whether there have any conflictions between the modules. Job submission service: users can select modules and submit NWP job. Job status monitoring: users can check job status given job ID. Job notification: after job completed, grid will notify users by e-mail, etc. Resource registration and management: perform registration and management of the resource Source code sharing: users can upload, download, modify, maintain and search source codes. Visualization tools: to provide common used meteorological visualization tools, such as isoline, streamline, 2D, 3D graph making, etc.

PDF created with FinePrint pdfFactory trial version

3. Collaborative Enviroment
Workflow management and source code management are used to construct the remote collaborative environment based on the web service technique. Workflow technique realizes the definition of complex job-flow. For every NWP module, a corresponding web service should be defined, and every web service must define a query interface to describe the function of the module to respond user’s request. Users can select the given modules and to define his own NWP workflow. NWP workflow can be defined via web, user can choose the desired modules and insert them to his own flow. Every defined NWP flow can be considered a combination of web service, every request to the flow can be thought of a job, and through this combined web service to complete user’s request.

SOAP Server
4.SOAP Message 6.Result

5.Invoke Service

Representation of Service

Construction of NWP Workflow
Construct new services using registered service

2.deploy service 3.Inquiry

UDDI Registry

Forecast Web service Preprocess 3DVAR Analysis

1. Register Retrieve Service/workflows /delivers output to user

XMLdata description WSDL description

Model forecast Postprocess Visualization

Workflow execution

Product announce

Figure 2: Framework of NWP workflow

The source code manager realizes the function of CVS with WEB technology by three layers frames: portal、Web Service layer and CVS system. User manages the source code in manner of B/S. The web service is written in JSP. In order to response user’s request, web service is performed interactively and simultaneously with command interpreter and CVS.

[1] A.P.McNally,E.Andersson and G.kelly,PrepIFS-Global Modeling via the Internet, ECMWF Newsletter Number83,spring 1999. [2] MEAD: [3] Ian Foster, et al.The Physiology of the Grid:an Open Grid Services Architecture for Distributed Systems Integration.

PDF created with FinePrint pdfFactory trial version

Shared By:
Description: construction scheme of meteorological application grid portal