Di_Grid_Status by panniuniu

VIEWS: 4 PAGES: 37

									  Current Status of GMU Grid Project and OGC OWS
                     2.0 Update

                                    Liping Di

         Laboratory for Advanced Information Technology and Standards (LAITS)
                            George Mason University (GMU)
                                      ldi@gmu.edu
                                   http://laits.gmu.edu



                                     September 8, 2004




                                                                           LAITS
                                                                  Laboratory for Advanced Information
9/6/04                                                                Technology and Standards      Page   1
                         Content of the presentation

         • Introduction
         • Status of Software Development and Integration.
            – Data and service catalogs
                 • Grid-based catalog service for web (GWRS)
            –   Metadata
            –   Services
            –   BPEL engine
            –   Web service client
         • Status of Testbed deployment
            –   Development environment at GMU/LAITS
            –   Creditable testbed to be deployed.
            –   Data at the testbed
            –   Services available at the testbed.
         • OGC OWS 2.0 Update
                                                                        LAITS
                                                               Laboratory for Advanced Information
9/6/04                                                             Technology and Standards      Page   2
                                    Introduction

     • GMU’s OGC Grid project is to provide Geospatial web services,
       which are compliant with OGC specifications, at the Grid
       environment
         – Enable interoperable data access to large data archives.
         – Provide standard-based chainable geospatial analysis services
         – Provide geospatial modeling and virtual data product generation.
     • Partners in the project
         – NASA Ames Research Center (NASA Information Power Grid).
         – DOE Lawrence Livermore National Lab (LLNL) (Earth System Grid).
     • This project is also a part of CEOS Grid testbed.
     • Funded by NASA Earth Science Technology Office (ESTO)
         – Three-year project from May 2003-April 2006
     • This presentation gives an update on recent progress of this
       project since last WGISS meeting.




                                                                           LAITS
                                                                  Laboratory for Advanced Information
9/6/04                                                                Technology and Standards      Page   3
                       The Grid-enable CSW (GCSW)

         • In the past several months, we developed a data and
           service catalog server based on OGC Catalog Service for
           Web (CSW) specification.
            – written in Java with JDBC connection to MySQL.
         • Any OGC CSW clients can access the server for catalog
           service (web service).
         • The server is wrapped as a Grid catalog service.
            – Provide both data (both virtual and non-virtual) and grid-
              enabled service catalogs.
            – Enable other Grid services to access the catalogs.




                                                                          LAITS
                                                                 Laboratory for Advanced Information
9/6/04                                                               Technology and Standards      Page   4
                          Current implementation of OGC CSW
                                                                                                         System Environment
                                                    App. Client/User Interface
                                                                                                         Solaris/Linux

                                                                                      OGC Interface      Tomcat/Ant
                                Grid Service Interface
                                                                                                         GT 3.2 (Grid Service)
                                                                                      as Web Service
                                                                                                         MySQL

         Grid Tools (Globus 3.2)                  GCSW                          CSW
          (GSI, GridFTP, RLS)                as Grid Service              as Web Service



   Grid Service Specifications                     Web Service (AXIS) Server
             (OGSA/OGSI)

                                               Catalogue Service for Web IM
           Grid Technologies
                                                              repositoryItem
                                                                                                      Service Type
                                          Dataset             cross-reference           Service Catalogue entry
                                      Catalogue entry                              Catalogue entry

                                                 Metadata generating                         Metadata generating
                                                          data served by service
         IM: Information Model                                                           Service
                                       Data object                                      Instance
                                                         service referenced by data
                                                                                                                 LAITS
                                                                                                        Laboratory for Advanced Information
9/6/04                                                                                                      Technology and Standards      Page   5
              Design of the Information Model (IM) of GCSW


                 RegistryObject              ClassificationSchema


                                             Service           Service IM (ISO 19119)
                      Classification

                                             ExtrinsicObject
                      Association


                      ServiceBindings         CSWExtrinsicObject

                          ……                  RepositoryItem

                      RegistryEntry
                                                          Dataset IM (ISO 19115)

                                                          NASA EOS Core System (ECS)


         ebRIM + OGC Catalogue Service for CSW + OGC Catalogue Service for ISO 19115/19119



                                                                                             LAITS
                                                                                    Laboratory for Advanced Information
9/6/04                                                                                  Technology and Standards      Page   6
                   Integration for OGC & Grid w/GCSW
         Server                                                                    Client
              Query GCSW
                                                            GWCS               1
              Job submit                                                            Grid
              Job status query         2           Mod
                                                                               8    Client
              Job result return                    i         WC
                                           7
                               iGSM                           S
                                                                           1
                                               3
                                           4                                        OGC
                                                            CS
                                                                                    Client
                           5                                W
                                  6
                                                         GCSW
                                                                   RLS
          Local

          Remote
                         WC                                 WMS
                          S
                         GWCS                               GWMS



          GWCS: Grid enabled WCS working as Grid Service
          iGSM: intelligent Grid Service Mediator

                  Grid Services              OGC Services

                  OGC request and response           Grid request and response


                                                                                                      LAITS
                                                                                             Laboratory for Advanced Information
9/6/04                                                                                           Technology and Standards      Page   7
               Data Flow Diagram for Integration w/GCSW

                                     MPGC request or                        remote
              User Data Request
                                     Other standard OGC request            GWMS
                                                                          remote
                                                                          GWCS
                   local                        Security Authentication                   iGSM
                  GWCS                                Job submit
                                                 Remote secure copy
         Is data local?         No    iGSM

                          Yes




                GMU HDF Data                      GCSW
                                                                     Ames HDF Data




                                                                                          LAITS
                                                                                 Laboratory for Advanced Information
9/6/04                                                                               Technology and Standards      Page   8
                      Design of the Metadata IM for GCSW
                                         Dataset Instance Metadata (dt_md) (IM11)
         ISO 19115 Part one              •     datasetTypeID (dtTypeID) M
                                         •     productName -- LongName (resTitle) M
                                         •     productDateTime (resRefDate -> resProDate) M
         ISO 19115 Part two              •     responsibleParty (respParty) (to IM12) M
                                         •     BBOX (westBL, eastBL, northBL, southBL) M
         (FGDC extenstion)               •     datasetLanguage (dataLang) (Default: English) M
                                         •     datasetCharacterSet (dataChar) (Default: utf8)
                                         •     topicCategory -- ECSTopicKeyword (tpCat) M
         NASA ECS                        •     spatialResolution (dataScaleDist)
                                         •     datasetAbstract -- CollectionDescription (idAbs) M
                                         •     distributionFormat (disFormatName) (Default: hdf)
         ISO 19119                       •     distributionVersion (disFormatVersion) (Default: 1.0)
                                         •     collectionRange – (beginDateTime, endDateTime)
                                         •     spatialRepresentationType (spatRpType) (Default:
                                               grid)
                                         •     referenceSystemInfo (refSysInfo) (to IM13)
           Abstract/Extract              •     lineageStatement (lnStatement)
                                         •     onlineResourceLinkage (olResLink)
                                         •     metadataFileIdentifier (mdFileID)
                                         •     metadataStandardName (mdStanName)
            Dataset Type Metadata        •     metadataStandardVersion (mdStandVer)
                                         •     metadataLanguage (mdLang)
                                         •     metadataCharacterSet (mdChar)
            Dataset Instance Metadata    •     mdResponsibleParty (mdRespParty) (to IM12) M
                                         •     mdDateStamp (mdDateSt) M
                                         •     instrumentLongName (instflnm)
                                         •     instrumentShortName (instshnm) M
            Service Type Metadata        •     platformLongName (platflnm)
                                         •     platformShortName (flatshnm) M
                                         •     processingLevelID (prolevid) M
            Service Instance Metadata    •     ECSTopicKeyword (tpKeyword) M
                                         •     ECSDisciplineKeyword (dsKeyword) M
                                         •     ECSVariableKeyword (vrKeyword)
                                         •
                                         •
                                               ECSTermKeyword (tmKeyword)
                                                                                          LAITS
                                               datasetStatus -- CollectionState (idStatus) … …
                                                                                 Laboratory for Advanced Information
9/6/04                                                                               Technology and Standards      Page   9
                     Extraction of HDF-EOS Metadata for GCSW

                       HDF File                                                        XML File
                       (MODIS,                                                        (Based on
                      ASTER, …)
                          Read out                                                    ISO19139)



                      1


                                             2                                       3
                     ECS Metadata                            ISO Metadata
                                             Map                                  Modify
                      Key/Value                                Key/Value


                                                                                       Template
                                                                                       XML File
                                         ECS – ISO       a                                         b
                                                                                      (Based on
                                        Mapping Table
                                                                                      ISO19139)

                      a Create a mapping table between ECS metadata and ISO19139 metadata. Store it in MySQL database.
         Pre-steps
                      b Create a template XML file based on the selected ISO19139 schemas.
                      1 Read out the ECS metadata from HDF files (MODIS, ASTER, …).
          Steps       2 Map the ECS metadata elements to ISO elements according to the selected ISO19139 schemas.
                      3 Modify the template XML file by inserting the mapped ISO elements.



                                                                                                                LAITS
                                                                                                       Laboratory for Advanced Information
9/6/04                                                                                                     Technology and Standards      Page   10
                     Analysis/Transformation Services

         • Several services are developed as a part of OGC OWS 2.0 and
           GeoBrain Projects.
            – Web Image Classification Service (WICS)
                • Provide both unsupervised and supervised multi-band classification
                  services.
            – Web Coordinate Transformation Service (WCTS)
                • Provide reprojection services for 12 map projections supported by
                  NASA HDF-EOS.
            – Both services are chainable
            – Draft OGC WICS and WCTS specifications are based on those two
              implementations.
         • A chainable reformatting service is being developed
            – Support about 40 formats.
         • More chainable geospatial analysis services are going to be
           developed in the near future
            – SOAP Interface, WSDL description.
         • All services are being wrapped as Grid services.
                                                                                    LAITS
                                                                           Laboratory for Advanced Information
9/6/04                                                                         Technology and Standards      Page   11
                                 BPELPower

         • LAITS has developed a BPEL management and execution
           engine called BPELPower.
            – Based on Open Source software such as BPWS4J, WSIF,
              WSDL4J, AXIS and SOAP which only provide the function
              library.
            – Running on the top of application servers, such as Tomcat,
              J2EE, JBoss, Weblogic, and WebSphere.
         • Integrates services into collaborative and transactional
           business processes.
         • Supports the various invocation of web services, e.g.,
           EJB, JMS, JCA, SOAP, MIME, HTTP GET / POST and
           Java classes.



                                                                       LAITS
                                                              Laboratory for Advanced Information
9/6/04                                                            Technology and Standards      Page   12
                            BPELPower Functions

         • Displaying processes and services (including part of IBM
           BPWS4J functions);
         • Deploying and un-deploying processes and services(including
           part of IBM BPWS4J functions);
         • Executing simple processes and services automatically;
         • Executing simple processes and services manually;
         • Executing complex processes and services automatically;
         • Executing complex processes and services manually;
         • Managing processes and services;
         • Managing instances and activities;
         • Web addresses with examples of workflows
            – simple interface
              http://bpel.laits.gmu.edu/bpelpower/admin/index.html
            – new interface, under constructing
              http://bpel.laits.gmu.edu/bpelpower/admin/index.jsp

                                                                              LAITS
                                                                     Laboratory for Advanced Information
9/6/04                                                                   Technology and Standards      Page   13
                             Web Service Client

         • Our Grid system will provide OGC Interfaces.
            – OGC client will be able to access the Grid the same as OGC
              Web services.
         • Our OGC compliant client--Multi-protocol Geospatial
           Client (MPGC) has been updated to support
            – OWS 2 WCS enhancement (WCS 1.0.2.0)
            – Support the CSW specification
            – Dynamic construction of geospatial process models (Service
              chaining and virtual data generation)




                                                                      LAITS
                                                             Laboratory for Advanced Information
9/6/04                                                           Technology and Standards      Page   14
                                    Current VO for the Testbed


                           GMU Laits (Solaris) (laits.gmu.edu)
                             Globus 3.2.1 with IPG CEOS Certs.




           GMU CA center
         GMU Llinux (Linux)                                          Ames Evelyn      (Irix)
        (llinux.laits.gmu.edu)                                     Globus 2.4 with IPG Certs.
     Globus 2.2 with Laits Certs.


                        GMU Data (Linux)                   Ames Angel     (Linux)
                         (data.laits.gmu.edu)
                                                         Globus 3.0 with IPG Certs.
                     Globus 3.2.1 with Laits Certs.


                GMU LAITS VO                                     NASA IPG VO


                                                                                       LAITS
                                                                              Laboratory for Advanced Information
9/6/04                                                                            Technology and Standards      Page   15
                               Testbed Update

      Installed gt3.2.1 and gt3.9.2 on Sun Solaris and one of Linux
       machines, and gt2.2 on another Linux machine at GMU LAITS.

      Installed gt3.0 on a Linux machine and gt2.4 on an Irix machine
       at NASA Ames. Now Ames is upgrading machines with new
       security, when it is finished, we will upgrade to gt3.2.1 and
       gt3.9.2.
      Issued IPG certificates to the above machines at GMU LAITS
       and NASA Ames, and test and debug their authentication to each
       other among those boxes.
      Setup MCS/RLS at GMU LAITS and NASA Ames.
      Setup MySQL server at GMU LAITS and NASA Ames.
      Work is in progress with DOE LLNL to join the virtual
       organization.
      In order to build a credible test environment, LAITS is adding a
       new computer system into the testbed                      LAITS
                                                               Laboratory for Advanced Information
9/6/04                                                             Technology and Standards      Page   16
                         The New Computer System

         • The new computer system is used to build a creditable testbed
           for both OGC Grid and GeoBrain projects.
            – The system will be arrived at mid September.
         • The system consists of
            – 3 Apple X servers, each having 2 G5 CPUs (1 head node and 2
              cluster nodes)
            – Each server has an unique Internet address while the head node
              can distribute tasks to the cluster nodes
            – 3 X Raid storages, 3.5 TB each (Total 11.5 TB disk space)
            – The Raid storage is linked to servers through a fiber channel
              switch.
            – The fiber channel switch is also linked to GSFC SAN.
            – The system is configured to allow adding more cluster nodes and
              Raid disks.
         • The system will be hosted by NASA GSFC
            – 1 Gb/s Internet connection is provided to the system.

                                                                               LAITS
                                                                      Laboratory for Advanced Information
9/6/04                                                                    Technology and Standards      Page   17
                            Datasets at the testbed

         • We will put variety of data generated by NASA on the
           system.
            –   MODIS low and high level products.
            –   Global topographic data generated by Shuttle.
            –   Landsat 1990 and 2000 global datasets.
            –   Aster, MISR, etc.
         • Those data will be the foundation for the services provided
           by our system.




                                                                         LAITS
                                                                Laboratory for Advanced Information
9/6/04                                                              Technology and Standards      Page   18
                           Services at the system

         • OGC-compliant data access and process services will be
           available at the system
            – Data access: WCS, WMS, CS/W-available at the end of this
              year.
            – Process services: WCTS, WICS, Reformating, ….
         • All services are available both to CEOS Grid testbed and
           as general web services.
         • Virtual geospatial products will be provided through
           geospatial modeling and workflow executions
            – All services available at the system are chainable and can
              be bond with OGC-compliant data sources dynamically.
         • The services will be available to all Internet users in the
           form of web services and to all users in the Grid virtual
           organization through Grid services.

                                                                        LAITS
                                                               Laboratory for Advanced Information
9/6/04                                                             Technology and Standards      Page   19
                    Screenshot of the Prototype System

         http://grid.laits.gmu.edu




             Grid Environment and Virtual Organization (VO)

                                                                       LAITS
                                                              Laboratory for Advanced Information
9/6/04                                                            Technology and Standards      Page   20
         End User Interface – MPGC WCS

                                         This is a WCS client
                                         portal to access Grid-
                                         Enabled OGC compliant
                                         Services.




                                                     LAITS
                                            Laboratory for Advanced Information
9/6/04                                          Technology and Standards      Page   21
              End User Interface – MPGC WRS
                                                      URL of WRS



  Query
  Theme




Query Items
based on
Metadata                       Query Results – Registered Objects
                               Registered Object Detail




                                                                      LAITS
                                                             Laboratory for Advanced Information
 9/6/04                                                          Technology and Standards      Page   22
                  End User Interface – WRS web interface 1/2
         Query Interface http://llinux.laits.gmu.edu:8080/WRS




                                                                         LAITS
                                                                Laboratory for Advanced Information
9/6/04                                                              Technology and Standards      Page   23
                  End User Interface – WRS web interface 2/2
         Query Result Return Interface




                                                                 LAITS
                                                        Laboratory for Advanced Information
9/6/04                                                      Technology and Standards      Page   24
                  End User Interface – CSW web interface 1/4
         OpenGIS Catalogue Services for Web (CSW) http://www.laits.gmu.edu:8099/csw




                                                                                        LAITS
                                                                               Laboratory for Advanced Information
9/6/04                                                                             Technology and Standards      Page   25
                  End User Interface – CSW web interface 2/4
         CSW getCapabilities()’s GET & POST methods interface




                                                                         LAITS
                                                                Laboratory for Advanced Information
9/6/04                                                              Technology and Standards      Page   26
                   End User Interface – CSW web interface 3/4
         Result return interface for CSW getCapabilities()




                                                                      LAITS
                                                             Laboratory for Advanced Information
9/6/04                                                           Technology and Standards      Page   27
                  End User Interface – CSW web interface 4/4
         CSW getRecords() POST method interface




                                                                 LAITS
                                                        Laboratory for Advanced Information
9/6/04                                                      Technology and Standards      Page   28
                              OGC OWS 2.0 Update

         • OGC Web Service Initiative 2.0 (OWS 2.0)
            – March 2004 to September 2004
         • OWS 2.0 Threads
            –   OWS Common Architecture
            –   Information Interoperability
            –   Imagery Handling Services/Decision Support (IH4DS)
            –   Technical Baseline Maturation
            –   GeoScout
            –   OpenLS™ services.
         • Two Demonstration
            – Natural Resource Demo supported by IH4DS
            – Defense Demo supported by GeoScout
         • LAITS involved in IH4DS
            – Many software development reported in this presentation is also
              part of the OWS 2.0


                                                                              LAITS
                                                                     Laboratory for Advanced Information
9/6/04                                                                   Technology and Standards      Page   29
                         OWS Common Architecture

         • The Common Architecture (CA) thread addresses issues and
           requirements driving the further elaboration of the of the core set
           of OGC specifications to bring them inline with the industry
           mainstream Web Services implementations.
             – Define a consistent adaptation and integration of Web Services
               standards, defined by leading organizations such as W3C, OASIS,
               Microsoft, IBM, Sun Microsystems, and the open-source
               community, with the OWS service framework in general and the set
               of core OGC specifications (WMS, WFS, CS-W, WCS, and WOS).
             – Address issues of fundamental architectural significance to OWS,
               including
                 • metadata about resources (services and data)
                 • how to adopt Web Service (specifically WSDL, SOAP, UDDI)
                   technologies Common Architecture for WMS, WFS, CS-W, WCS,
                   WOS.



                                                                              LAITS
                                                                     Laboratory for Advanced Information
9/6/04                                                                   Technology and Standards      Page   30
                          Information Interoperability

         • The Information Interoperability (II) thread of OWS2
           focuses on the interoperable expression, exchange and
           access of geographic information within and across
           information communities (Information communities are
           groups that share common geographic terms and
           common spatial feature definitions).
         • The work of OWS2 for this thread includes
            – Application Schema from GML Schema
            – WFS Support for GML Level 0 Profile
            – Integrated Reference Client supporting:
               •   WFS
               •   WMS
               •   CS/W
               •   WCS


                                                                  LAITS
                                                         Laboratory for Advanced Information
9/6/04                                                       Technology and Standards      Page   31
                       Technical Baseline Maturation

         • The OGC Technical Baseline contains a list of all OGC
           specifications and specification candidates, their type, title,
           current version, document number, description, and
           primary author/editor.
         • This information provides the reader with a current
           snapshot of work within the OGC.
         • OWS 2.0 develops conformance tests for the core set of
           the Technical Baseline that include: WMS, WFS, CS-W,
           WCS, and WOS.




                                                                        LAITS
                                                               Laboratory for Advanced Information
9/6/04                                                             Technology and Standards      Page   32
                                   GeoScout

         • The purpose of the GeoScout task is to mature the OGC
           technology base so that the GeoScout program can
           purchase and deploy SCOTS products with a high
           confidence of success.
            – This thread leverages the work being done under the
              Information Interoperability (II) and Technical Baseline
              Maturity (TBM) tasks and applies it to the GeoScout specific
              requirements.
         • Only open to US participants
            – Application Schemas
            – SCOTS Assessment and Enhancement




                                                                        LAITS
                                                               Laboratory for Advanced Information
9/6/04                                                             Technology and Standards      Page   33
                              OpenLS™ services

         • The OpenGIS Location Services (OpenLS) represent an
           open (middleware) platform for location-based application
           services for mobile assets and terminals.
         • The primary goal of the OpenLS initiative series is to
           define the specifications for the “Core Services and
           Abstract Data Types (ADT)” that comprise this platform.
            – The services are defined in two OGC specifications.
         • The OWS 2 work is to
            – Enhance these specs (add minor features).
            – Add new specifications for traffic and tracking services.




                                                                          LAITS
                                                                 Laboratory for Advanced Information
9/6/04                                                               Technology and Standards      Page   34
            Imagery Handling Services for Decision Support
                               (IH4DS)

         • Working on the following areas:
            – Image Archive Server (WCS enhancements) (GMU lead)
            – Image Metadata (GMU lead)
            – Image processing services
                • Web coordinate transformation service (WCTS) (PCI & GMU lead)
                • Web Image classification service (WICS) (GMU lead)
            – Chaining and Orchestration (Intergraph & GMU lead)
            – Decision Support Client (Intergraph lead)
         • Participants:
            – GMU/LAITS, Ionic, Raytheon (NASA Data Pools), Intergraph, PCI,
              SPOT Image
         • Interoperability Program Reports (IPR)
            – WCS change proposal (WCS 1.0.2.0)
            – IM, WICS, WCTS IPR


                                                                               LAITS
                                                                      Laboratory for Advanced Information
9/6/04                                                                    Technology and Standards      Page   35
                                 OWS 2 IH4DS Demo

         • The IH4DS demo was conducted on September 2 at Raytheon
           Landover facility
         • The demo scenario is forest fire near LA
         • The demo shows how OGC specifications can help the decision
           support for fighting the forest fires in 3 phases
             – Pre-fire : Evaluate the fire danger through estimating the availability
               of fuels- using MODIS and Landsat data.
             – During fire: Detecting fires using MODIS for early detection, SPOT,
               and air-borne remote sensors for hotspot validation and monitoring.
             – After fire: Damage estimation by using SPOT data (Burn scar
               classification).
         • Shows the individual services.
         • Shows chain of WCS, WCTS, and WICS for fire detection,
           locating, mapping, and fire damage estimation (virtual products).
         • Interoperability of workflows are also shown.


                                                                                  LAITS
                                                                         Laboratory for Advanced Information
9/6/04                                                                       Technology and Standards      Page   36
         Thank You for your attention !

             Any Questions ?

                                           LAITS
                                  Laboratory for Advanced Information
9/6/04                                Technology and Standards      Page   37

								
To top