OGF slide presentation template

Document Sample
OGF slide presentation template Powered By Docstoc
					               OGF-24



        Grid-Enabled
      Geospatial Systems

Craig Lee            Mark Reichardt
lee@aero.org    mreichardt@opengeospatial.org
 Geospatial Data: Immense Applicability
  Estimates vary, but ~80-90% of all data collected or
produced by the human race is geospatially referenced
  Image: E. Gennai, C. Terbough, ESRI



 • Natural exploration, e.g., oil & gas
 • Public administration
 • Civic planning and engineering
 • Weather and aviation
 • Environmental planning
 • Disaster management
 • Satellite ground systems
                                        2
            An Example: GEO Grid
Global Earth
Observation
    Grid
S. Sekiguchi,
    AIST




                      3
  The OGC-OGF Collaboration



• Promote best practices and international standardization for
  distributed geospatial data processing capabilities that is:
   • Transparent -- users is not aware of the infrastructure
   • Interoperable -- the resources work together
   • Scalable -- small local, to massive distributed platforms
  Directly Relevent to Satellite Ground Systems
                              4
   Two Issues
• What geospatial and
  computing technologies
  must be integrated?
• How do we manage the
  process to make progress
  on doing so?

          5
    Some Key OGC Standards
• Web Map Service (WMS)
   • Display of registered and superimposed map-like views of
     information that come simultaneously from multiple remote
     and heterogeneous sources.
• Web Feature Service (WFS)
   • Retrieval and update of digital representations of real-world
     objects referenced to the Earth’s surface.
• Web Coverage Service (WCS)
   • Access to a grid coverage, usually encoded in a binary
     format, and offered by a server.
• Catalogue Service for the Web (CSW)
   • Common interfaces to publish, discover, browse, and query
     metadata about data, services, and other resources


                                6
             Key OGC Standards
• Sensor Observation Service (SOS)
    • Access observations for a sensor or sensor constellation, whether
      in-situ or dynamic (e.g, water monitoring or satellite imaging).
      Optionally access associated sensor and platform data
• Sensor Planning Service (SPS)
    • An interface to task sensors or models. Using SPS, sensors can be
      reprogrammed or reconfigured, sensor missions can be started or
      changed, simulation models executed and controlled.
• Sensor Alert Service (SAS)
    • Defines an interface for publishing and subscribing to alerts from
      sensors. If an event occurs the SAS will notify all clients subscribed
      to this event type.
• Observations and Measurements Encoding Standard (O&M)
    • Defines an abstract model and an XML schema encoding for
      observations and it provides support for common sampling
      strategies. It also provides a general framework for systems that
      deal in technical measurements in science and engineering.


                                     7
           Key OGC Standards
• Transducer Markup Language (TML)
   • TML defines the conceptual model and XML Schema for
     describing transducers and supporting real-time streaming of
     data to and from sensor systems

• Web Processing Service (WPS)
   • WPS defines a standardized interface to facilitate the
     publishing of geospatial processes, and the discovery of an
     binding to those processes by clients




                               8
             OGC Sensor Web Enablement
                 Framework (SWE)              Registered sensor and observation metadata




                                        CSW
                                              Metadata for a sensor and observations, and a URL
                                              Metadata for a sensor observations, and a URL
                                              Metadata for a sensor observations, and a URL
                                              Metadata for a sensor observations, and a URL
                                              Metadata for a sensor observations, and a URL
                                              ………




                                                    Mission




                                        SPS
                                                    Control
                                                    System




                                              




                                        WNS
SWE Architecture Encodings:
–Observations & Measurements




                                        SOS
                                                                                                  IEE1451
–SensorML
–Transducer Markup Language (TML)


                                                      
                                        SAS
                                                      !



                                    9
OGC Web Services Combined




            10
           Key OGF Standards
• HPC-Basic Profile
   • Cluster scheduling interoperability layer supported by
     Microsoft, Altair Computing, Platform Computing, and others
• Simple API for Grid Applications (SAGA)
   • Complete distributed programming environment with C++
     and Java bindings
• GridRPC
   • Grid-enabled Remote Procedure Call
• Data Access and Integration specs (WS-DAI-*)
   • Set of specs for accessing remote files, databases, XML
     documents
• Web Services Resource Framework (WSRF)
   • Allow remote data and services to be independently
     managed


                               11
An Example of a Global Grid: EGEE
     Bob Jones, EGEE Project Director




                         QuickTime™ an d a
                     TIFF (LZW) decomp resso r
                  are need ed to see this picture .




                                  12
   Coalescing Service Categories
       to Specification Areas
• Infrastructure Service Categories          • Specification Areas
  • Catalogues & Registries                    • Catalogues & Registries
  • Discovery                                  • Discovery
  • Applications                               • Applications
  • Job Submission/Execution Svcs              • Execution Services
  • Workflow Management                        • Workflow/Transactions
  • Data Product Management                    • Data Management
  • Storage & Archiving                        • State Management
  • Messaging                                  • Messaging, Routing, Addressing
  • Event Notification                         • Event Notification
  • User Management                            • Metadata Schemas & Ontologies
  • Information Assurance/Security             • Portals and User Interfaces
  • Monitor & Control                          • Security
  • Enterprise Svc Mgmt/Software Mgmt          • Policy & Agreement
  • APIs/Ops Systems                           • System Architecture
  • Financial Accounting                       • System Management


                                        13
   Populating the Specification Areas
                                  (There are essentially an unbounded number of “applications”
                                  that could run as part of the application domain. OGC WMS,
Applications
                                  WFS & WCS are listed here since they would be part of what
                                  an end-user sees and interacts with.)
                                  WSDL, SOAP, (WS-I Basic Profile), CORBA, JSDL, OGSA-
Job Submission/Execution Svcs
                                  BES (OGF HPC Basic Profile), OGC Web Processing Service
                                  UDDI, (WS-I Basic Profile), DDMS v1.3 (DoD Discovery
Discovery
                                  Metadata Specification), WS-Discovery
                                  LDAP, OpenGIS Cat 2.0, OGC CSW, ebRIM, ebXML RS,
                                  EO Products Ext., NATO DFDD 2007-1 (DGIWB Data
Catalogues & Registries
                                  Feature Data Dictionary), NAS v1.8 (NSG Application
                                  Schema), NEC V1.8 (NSG Entity Catalog)
                                  XML, CIM/GLUE, FGDC, GML 3.1, ISO 15836 (Dublin
Metadata Schemas & Ontologies     Core), ISO 19115 (metadata), ISO 19119 (services), ISO
                                  19130 (sensors), OWL, OWL-S, SWRL, (many other ISO docs)

                                  HTTP, WS-Addressing, WS-MessageDelivery,                  WS-
Messaging, Routing, Addressing
                                  Reliability, WS-ReliableMessaging

Event Notification                WS-Notification, WS-Eventing, Sensor Alert Service

     Updated from Fox, Ho, Pierce – U. Indiana        Green: On DISR Baseline 07-3.0

                                                 14
  Populating the Specification Areas
Workflow/Transactions                BPEL, WS-Coordination, (Kepler, Triana, Pegasus, …)

Data Management                      FTP, GridFTP, OGSA-DAI, (SRB, iRODS)
                                     IETF RFC 2695 (HTTP State Management (cookies)),
State Management                     WS-ResourceFramework, WS-Context, WS-I+,
                                     WS-MetadataExchange
Portals and User Interfaces          WS-RemotePortlets

                                     SSL/TLS, Kerberos, WS-Security, ID-WSF 2.0 (Liberty
                                     Identity Web Services Framework), WS-Trust, SAML, GSI,
Security
                                     Shibboleth, VOMS (Virtual Organization Management System),
                                     OpenID, OAuth

Policy & Agreement                   WS-Policy, WS-Agreement, (iRODS), XACML, GeoXACML

                                     OGF Open Grid Services Architecture, OGC Sensor Web
System Architecture
                                     Enablement (SWE), Community Sensor Model v2.A (CSM)

                                     WS-DistributedManagement, WS-Management,
System Management                    WS-Transfer, OGF CDDLM (Configuration Description,
                                     Deployment and Lifecycle Management)
           Updated from Fox, Ho, Pierce – U. Indiana   Green: On DISR Baseline 07-3.0

                                                 15
How Do We Manage Progress for
Distributed Geospatial Systems?
 • Targeted Projects on Decisive Issues
 • Build Critical Mass of Key Stakeholders
   • Continual polling and coordination across the
     community
 • They must agree on:
   •   Clear Goals
   •   Clear Schedule (“time-box” the process)
   •   Clear Responsibilities
   •   Properly Provisioning the Effort


                        16
         A General Process Model
   Task A            *RFQ/CFP = Request for Quotation/Call for Participation
   Concept
 Development
                 Task B
                RFQ/CFP*
               Development
                                 Task C
                              Selection &
                               Kick-off
                                                    Task D

                                                  Develop &
                                                    Test
Clear Schedule, Deliverables and                                    Task E

    Project Responsibilities                                       Deploy &
                                                                    Persist
                                    17
                                                                               17
   Return on Investment
• What are the “carrots” to build the critical
  mass of stakeholders?
   • Get early influence in specification development,
     early skills building, visibility, and opportunity for
     early market deployment of standards, but just
     as important…
   • Return on Investment (ROI)
• Investment
   • Time, Money & People
   • Both Monetary and In-Kind (labor & materials)
• Timely Connection to Concrete Results
   • Stakeholders benefit from collaboration
   • Get more than they put in
                        18
A Case Study: HPC Basic Profile
 • 2006
    • Key stakeholders decided to demonstrate interoperability
       between their existing job submission infrastructures
 • November 2006 (SC06)
    • Prototype implementations demonstrated
 • 28-August-2007
    • HPC Basic Profile, Version 1.0, published
    • http://www.ogf.org/documents/GFD.114.pdf
 • November 2007 (SC07)
    • Interoperability demonstrated by Altair, Microsoft, Platform,
        OMII-UK, OMII-Europe, EGEE, UVa
 • 21-February-2008
    • Interoperability Experiences with HPCBP, Version 1.0, published
      http://www.ogf.org/documents/GFD.124.pdf
 • Commercial adoption plans by Altair, Microsoft & Platform

                                 19
OGC Web Service Testbed
• OWS is an annual process where sponsors identify
  specific demonstration targets
   • Larger number of participants supply in-kind resources
• OWS-5 recently finished (March 28, 2008)
• OWS-5 Sponsors
   •   BAE Systems - National Security Solutions
   •   Federal Geographic Data Committee (FGDC/USGS)
   •   Google
   •   Lockheed Martin Integrated Systems & Solutions (Lead Org.)
   •   Northrop Grumman
   •   US National Aeronautic and Space Administration (NASA)
   •   US National Geospatial-Intelligence Agency (NGA)
• ~35 Participating Organizations
• 3x-4x Return on Investment
   • Strength of Collaboration
                            20
              OWS-5
• Five Threads
  •   Sensor Web Enablement (SWE)
  •   Geo Processing Workflow (GPW)
  •   Information Communities and Semantics (ICS)
  •   Agile Geography
  •   Compliance Testing (CITE)
• Outbrief Videos
  •   NASA Sensor Web 2.0 Experiments (7:43)
  •   Data Architecture Views (10:37)
  •   Conflation Processing (5:07)
  •   Web Coverage Processing Service (5:58)

                     21
 Still from OWS-5 NASA Sensor Web Video
http://www.opengeospatial.org/pub/www/ows5/index.html




                                         Satellite EO-1 tasked to collect imagery
                                         on Northern San Diego County Wildfires
                                         that was integrated with UAV track data
                                         (red lines) in Google Earth. Demo drove
                                         issues around sensor networks, data
                                         interoperability, and command & control.

                                    22
        OWS-6: RFQ/CFP Topics
• Sensor Web Enablement (SWE)
   •   Georeferenced sensors, event notification, security
• Aviation Information
   •   Aeronautical Information Exchange Model (AIXM) for next generation flight
       control system (FAA and EuroControl)
• GeoProcessing Workflows (GPW)
   •   Web Processing Service (WPS), Workflow and Grids (“WPS to Grid”)
   •   Distributed resource management, e.g., data, services, hosts, networks, for
       geospatial applications
• Decision Support Services
   •   3D Visualization with Fly Through
   •   Open Location Services
   •   Integrated Client for multiple OWS services
• Compliance, Interoperability & Test Evaluation (CITE)
   •   Suite of tests and reference implementations for all approved standards




                                         23
   Possible Geospatial/Grid Topics
• Integration, interoperability of geospatial and computing resource
   • Information models (CIM/GLUE)
   • Distributed and federated catalogs (ebRIM, MDS, etc.)
   • Discovery services
• Implementation of WPS on various grid tools
   • SAGA, GridRPC, HPC-Basic Profile
• Integration of WS-Eventing, WS-Notification, INFOD
   • Sensor Web Enablement
• Integration of configuration and lifecycle management tools
   • Configuration Description, Deployment and Lifecycle Management (CDDLM)
• Integration of WPS with workflow management tools
   • BPEL, Kepler, Taverna, Triana, Pegasus, etc.
   • Workflow design tools, execution engines, planning (data virtualization), data
     provenance
• Integration of WPS with grid security models
   • SAML, XACML, VOMS, GSI
   • Support for virtual organizations
• Integration of geospatial data with rule-based, data management policy
  engines
   • iRODS
                                         24
    Digital Repositories/Libraries
• Digital Repositories/Libraries represent a growing part of the
  “infomass”
    • Federated catalogues and federated storage
• OGF-Europe Workshop at OGF-23
    • 4 sessions, ~70 participants from industry, academia and European
      Commission
• Proposed OGF-Europe collaboration with Global Research Library 2020
• Proposed OGF-Europe involvement in GENESI-DR Digital Repositories
  Seminar in Autumn 2008, Frascati
• OGF-Europe PC membership for the DReSNet special DR session at
  IEEE e-Science conference, Dec 2008
• Eventual development of an OGF Working Group dedicated to issues
  surrounding DRs


                                    25
     Another Example: Green Grids
          Managing Energy Policy
                                                • Extensive use of
                                                  virtualization and
                                                  fiber optics to put
                                                  the computing
                                                  where the power is
                                                • Enable the
                                                  workload to “follow
                                                  the wind” or
                                                  “follow the sun”


             Source: www.Bastionhost.com

Bill St. Arnaud

                                           26
     Another Example: Green Grids
          Managing Energy Policy
                                                • Extensive use of
                                                  virtualization and
                                                  fiber optics to put
                                                  the computing
                                                  where the power is
                                                • Enable the
                                                  workload to “follow
                                                  the wind” or
                                                  “follow the sun”


             Source: www.Bastionhost.com

Bill St. Arnaud

                                           27
     Another Example: Green Grids
          Managing Energy Policy
                                                • Extensive use of
                                                  virtualization and
                                                  fiber optics to put
                                                  the computing
                                                  where the power is
                                                • Enable the
                                                  workload to “follow
                                                  the wind” or
                                                  “follow the sun”
                                                • A geospatial sensor
                                                  network
             Source: www.Bastionhost.com

Bill St. Arnaud

                                           28
              Summary
• Significant work already underway for
  standard, distributed geospatial computing
  infrastructures
• Highly relevant to many application domains
• Opportunity to leverage the work of key
  participants through collaboration
• How to engage application groups and
  industries needing distributed geospatial
  processing?

                     29
   Back-Ups and
Other Draft Material
                       A “Cloudy” Solution Space
Data Collection    Data Collection
                                                                                                   Vendors of high-level
                                                                                                   application configuration
map map map map map map map map
                                                                  OS Virtualization                tools are potential
     reduce            reduce
                                                                 Amazon
                                                                                                   “cloud framework providers”
              reduce                                             S3/EC2
                                                             RightScale, GigaSpaces,
                                                                  Elastra, 3Tera

                                                        Hadoop over EC2           Cohesive

                                                                                  Sun’s Caroline

                                                    Hadoop
                                                                                          MS Astoria

                                                                                                   Mesh
                                                  MapReduce,
                                                  GFS, BigTable                  Google AppEngine
                                       Parallel                                                           Software as a
                                     Frameworks                                                              Service

              …..


    Courtesy of Dennis Gannon, www.extreme.indiana.edu/~gannon/science-data-center.pdf
                                                                          31
           A Host of Cloud Issues
• Data access and interoperability
    • Will have to be approached at the application domain level, by the
      domain users involved
• Security
    • Can cloud providers provide sufficient security for sensitive applications?
• Reliability
    • Can cloud providers provide sufficient reliability for critical applications?
• Frameworks
    • How to manage sets of VMs and VOs?
    • These are essentially “cloud frameworks” and call for "cloud framework
      providers"
• Performance management
    • Clouds tend to abstract away location
    • Managing the compute-data locality (affinity) can be important
• Costing models
    • How to compare your own infrastructure costs with a cloud computer?
    • How to compare two clouds?
                                        32

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:10
posted:11/4/2011
language:English
pages:32