Grid Computing

Document Sample
Grid Computing Powered By Docstoc
					      Grid Computing

            Ian Foster
Mathematics and Computer Science Division
       Argonne National Laboratory
                  and
     Department of Computer Science
        The University of Chicago
     http://www.mcs.anl.gov/~foster
                                                        2

            Partial Acknowledgements
     Open Grid Services Architecture design
         Carl Kesselman, Karl Czajkowski @ USC/ISI
         Steve Tuecke @ANL
         Jeff Nick, Steve Graham, Jeff Frey @ IBM
     Grid services collaborators at ANL
         Kate Keahey, Gregor von Laszewski
         Thomas Sandholm, Jarek Gawor, John Bresnahan
     Globus Toolkit R&D also involves many fine
      scientists & engineers at ANL, USC/ISI, and
      elsewhere (see www.globus.org)
     Strong links with many EU, UK, US Grid projects
     Support from    DOE, NASA, NSF, IBM, Microsoft
foster@mcs.anl.gov                        ARGONNE  CHICAGO
                                                             3



                             Goals
        Communicate the purpose, significance,
         state, adoption, & future of Grid technology
        Persuade you that Grid technology
         represents an opportunity
            Grids aren’t (particularly) about science or
             servers—themes of virtualization, service
             discovery, service management, and QoS
             delivery are universal
            Rapid uptake in industry & science represents
             an exceptional opportunity for impact


foster@mcs.anl.gov                            ARGONNE  CHICAGO
                                                           4



                          Overview
        Origins: Resource sharing within scientific
         collaborations
            Science drivers & science Grid projects
            Globus Toolkit
        Evolution: Resource virtualization
            Commercial drivers
            OGSA: Grid meets Web services




foster@mcs.anl.gov                           ARGONNE  CHICAGO
                                                       5



                         Overview
        Origins: Resource sharing within
         scientific collaborations
            Science drivers & science Grid projects
            Globus Toolkit
        Evolution: Resource virtualization
            Commercial drivers
            OGSA: Grid meets Web services




foster@mcs.anl.gov                       ARGONNE  CHICAGO
                                                          6


     E-Science: The Original Grid Driver
        Pre-electronic science
            Theorize &/or experiment, in small teams
        Post-electronic science
            Construct and mine very large databases
            Develop computer simulations & analyses
            Access specialized devices remotely
            Exchange information within distributed
             multidisciplinary teams
             Need to manage dynamic, distributed
             infrastructures, services, and applications

foster@mcs.anl.gov                          ARGONNE  CHICAGO
                                                                           7
                      eScience Application:
                 Sloan Digital Sky Survey Analysis
Size distribution of
 galaxy clusters?




             Galaxy cluster
100000
             size distribution
10000
                                               Chimera Virtual Data System
 1000                                          + GriPhyN Virtual Data Toolkit
  100
                                              + iVDGL Data Grid (many CPUs)
   10




foster@mcs.anl.gov
    1
         1                10
                   Number of Galaxies
                                        100          www.griphyn.org/chimera
                                                         ARGONNE  CHICAGO
                                                                                                                8
             NASA’s Information Power Grid:
                     Aviation Safety
                                       Wing Models


                                                         •Lift Capabilities
                                                         •Drag Capabilities            Stabilizer Models
                                                         •Responsiveness
                     Airframe Models




                                                                                           •Deflection capabilities
                                                                                           •Responsiveness
 Crew Capabilities
 - accuracy
 - perception
 - stamina
 - re-action times
 - SOPs
                                                                              Engine Models




  Human Models                              •Braking performance
                                            •Steering capabilities
                                                                               •Thrust performance
                                            •Traction
                                                                               •Reverse Thrust performance
                                            •Dampening capabilities
                                                                               •Responsiveness
                                                                               •Fuel Consumption
                              Landing Gear Models


foster@mcs.anl.gov                                                            ARGONNE  CHICAGO
                                                                 9

         Life Sciences: Telemicroscopy
         DATA ACQUISITION
                            PROCESSING,     ADVANCED
                              ANALYSIS    VISUALIZATION




                                                    NETWORK




     IMAGING                 COMPUTATIONAL
   INSTRUMENTS                 RESOURCES
                                               LARGE DATABASES
foster@mcs.anl.gov                            ARGONNE  CHICAGO
                                                       10


                 And Thus: The Grid
 “Resource sharing & coordinated
  problem solving in dynamic, multi-
  institutional virtual organizations”




foster@mcs.anl.gov                       ARGONNE  CHICAGO
                                                         11




     Underlying Technical Requirements
        Dynamic formation and management of
         virtual organizations
        Online negotiation of access to services:
         who, what, why, when, how
        Configuration of applications and systems
         able to deliver multiple qualities of service
        Autonomic management of distributed
         infrastructures, services, and applications



foster@mcs.anl.gov                        ARGONNE  CHICAGO
                                                           12



         The Grid World: Current Status
        Dozens of major Grid projects in scientific &
         technical computing/research & education
        Open source Globus Toolkit™ a de facto
         standard for major protocols & services
            Simple protocols & APIs for authentication,
             discovery, access, etc.: infrastructure
            Information-centric design
            Large user and developer base
            Multiple commercial support providers
            Enabler of numerous tools and applications
        Global Grid Forum: community & standards
foster@mcs.anl.gov                           ARGONNE  CHICAGO
                                                           13



                          Overview
        Origins: Resource sharing within scientific
         collaborations
            Science drivers & science Grid projects
            Globus Toolkit
        Evolution: Resource virtualization
            Commercial drivers
            OGSA: Grid meets Web services




foster@mcs.anl.gov                           ARGONNE  CHICAGO
                                                           14

        Resource Sharing within “VOs” is
            Not Unique to Science!
       Fragmentation of enterprise infrastructure
           Driven by cheap servers, fast nets,
            ubiquitous Internet, eBusiness workloads
           Need to configure distributed collections of
            services to deliver specified QoS
       Virtualization
           Emerging service infrastructure, utility
            computing models, economies of scale
           Services dynamically instantiated across
            device spectrum
       B2B, B2C,    C2C interactions
foster@mcs.anl.gov                           ARGONNE  CHICAGO
                                                                     15

                Virtualization and
        Distributed Service Management
Less capable, integrated                         Larger, more integrated
    Less connected                                   More connected
   User service locus      Device Continuum      Dynamically provisioned

                 Distributed service
                    management                Resource &
                                                service
                                              aggregation


                                                  Delivery of virtualized
                                                   services with QoS
                                                       guarantees
Dynamic, secure
service discovery
 & composition
foster@mcs.anl.gov                                 ARGONNE  CHICAGO
                                                        16

             Realizing the Promise
         Requires Significant Innovation
        Automation of infrastructure operation to
         achieve economies of scale
        Management and component models for
         service discovery, composition, provisioning
        New applications and tools powered by
         distributed services and resources
        Business and service models to support
         specialization of function



foster@mcs.anl.gov                      ARGONNE  CHICAGO
                                                       17

                Grid Evolution:
         Open Grid Services Architecture
        Refactor Globus protocol suite to enable
         common base and expose key capabilities
        Service orientation to virtualize resources
         and unify resources/services/information
        Embrace key Web services technologies:
         WSDL as IDL, leverage commercial efforts
        Result: standard interfaces & behaviors for
         distributed system management



foster@mcs.anl.gov                       ARGONNE  CHICAGO
                                                              18



               OGSA System Structure
        A standard substrate: the Grid service
            Standard interfaces and behaviors that
             address key distributed system issues
            The “Grid Service Specification”
        … supports standard service specifications
            Resource management, databases,
             workflow, security, diagnostics, etc., etc.
            Target of current & planned GGF efforts
        … and arbitrary application-specific
         services based on these & other definitions
foster@mcs.anl.gov                              ARGONNE  CHICAGO
                                                                19


            Transient Service Instances
      “Web services” address discovery & invocation
       of persistent services
          Interface to persistent state of entire enterprise
      In Grids, must also support transient service
       instances, created/destroyed dynamically
          Interfaces to the states of distributed activities
          E.g. workflow, video conferencing, distributed
           data analysis, workload management
      Significant implications for how services are
       named, discovered, managed, and used

foster@mcs.anl.gov                            ARGONNE  CHICAGO
                                                                              20



         OGSI, OGSA, and Web Services
        OGSI (I = Infrastructure)
            Small extensions to WSDL
                 Nested serviceType & serviceDataDescription
            Conventions for naming service instances
                 Handles and references
            portTypes for common behavior
                 Instance creation, lifetime management, introspection and
                  monitoring, registration, notification, …

        OGSA (A = Architecture) built on OGSI
            A collection of Grid service interfaces
                 Resource description & provisioning
              Higher-level services: messaging services, logging, etc.
foster@mcs.anl.gov                                    ARGONNE  CHICAGO
                                                                                  21

             The Grid Service =
    Interfaces/Behaviors + Service Data

                        GridService      … other interfaces …
Service data access     (required)            (optional)           Standard:
Explicit destruction                                               - Notification
Soft-state lifetime                                                - Authorization
                            Service   Service      Service
                                                                   - Service creation
                              data      data         data          - Service registry
Binding properties:         element   element      element
                                                                   - Manageability
- Reliable invocation                                              - Concurrency
- Authentication
                                Implementation
                                                                   + application-
                                                                   specific interfaces

                        Hosting environment/runtime
                             (“C”, J2EE, .NET, …)



foster@mcs.anl.gov                                              ARGONNE  CHICAGO
                                                                              22


                             Service Data
        A Grid service instance maintains a set of
         service data elements
            Described in WSDL extension
            XML element encapsulated in standard
             container: name, type, lifetime, etc.
            Includes basic introspection information,
             interface-specific data, and application state
        Pull and push models for information query
            GridService::FindServiceData operation
                 Pull: queries this information via extensible query language
            NotificationSource::SubscribeServiceData
                 Push: Subscribe to notification of changes to information
foster@mcs.anl.gov                                       ARGONNE  CHICAGO
                                                                        23

                    Notification Interfaces
        NotificationSource for client subscription
            Subscription expression describes which service
             data element changes are of interest
            Creates a subscription manager service
                 Manages the lifetime and properties of subscription

        NotificationSink for asynchronous delivery
         of notification messages
        Simple, flexible base with wide variety of uses
            Dynamic discovery/registry services,
             monitoring, application error notification, etc.
            Intermediaries: filter, aggregate, archive, et.c
            Can integrate commercial messaging services
foster@mcs.anl.gov                                      ARGONNE  CHICAGO
                                                                       24
                 Grid Service Example:
                   Database Service
        A DBaccess Grid service will support at
         least two portTypes          Grid
                                          Service   DBaccess
            GridService
            DBaccess                           Name, lifetime, etc.

        Each has service data                  DB info


            GridService: basic introspection
             information, lifetime, …
            DBaccess: database type, query languages
             supported, current load, …, …
        Maybe other portTypes as well
            E.g., NotificationSource
foster@mcs.anl.gov                              ARGONNE  CHICAGO
                                                              25



                 Lifetime Management
        GS instances created by factory or manually;
         destroyed explicitly or via soft state
            Negotiation of initial lifetime with a factory
             (=service supporting Factory interface)
        GridService interface supports
            Destroy operation for explicit destruction
            SetTerminationTime operation for keepalive
        Soft state lifetime management avoids
            Explicit client teardown of complex state
            Resource “leaks” in hosting environments

foster@mcs.anl.gov                             ARGONNE  CHICAGO
                                                            26



                            Factory
       Factory interface’s CreateService operation
        creates a new Grid service instance
           Reliable creation (once-and-only-once)
       CreateService operation can be extended to
        accept service-specific creation parameters
       Returns a Grid Service Handle (GSH)
           A globally unique URL, resolves to GSR
           Uniquely identifies the instance for all time
           Based on name of a handle resolver
       Or Grid Service Reference (GSR)
foster@mcs.anl.gov                            ARGONNE  CHICAGO
                                                                        28

                        Example:
               Transient Database Services
“What services         “Create a database
can you create?”       service”

                   Grid                     Grid
                                DBaccess
                   Service                  Service
                                Factory               DBaccess


                      Instance name, etc.        Name, lifetime, etc.

“What database         Factory info              DB info
services exist?”

                   Grid                     Grid
                   Service       Registry   Service   DBaccess


                      Instance name, etc.       Name, lifetime, etc.

                      Registry info             DB info



foster@mcs.anl.gov                               ARGONNE  CHICAGO
                                                           29



         OGSA Design & Implementation
        OGSI (I=Infrastructure) WG in GGF
         defining core Grid service specification
            (At least) three implementation efforts
        Globus Toolkit => GT3 (alpha end 2002)
            GT3 Core: Grid service specification
            GT3 Base: Globus Toolkit behaviors
            CIM resource model, GRAM-2 SLA
             negotiation, database services, …
        Other GGF WGs address OGSA security,
         OGSA-compliant database services, etc.
foster@mcs.anl.gov                           ARGONNE  CHICAGO
                                                             30



                       Recap: Goals
        Communicate the purpose, significance,
         state, adoption, & future of Grid technology
        Persuade you that Grid technology
         represents a significant opportunity
            Grids aren’t only (or particularly) about
             science and servers—themes of virtualization,
             service discovery, service management, and
             QoS delivery are universal
            Rapid uptake in industry & science represents
             an exceptional opportunity for impact

foster@mcs.anl.gov                          ARGONNE  CHICAGO
                                                 31


                 For More Information
    The Globus Project™
        www.globus.org
    Context & research articles
        www.mcs.anl.gov/~foster
    Open Grid Services
     Architecture
        www.globus.org/ogsa
    Global Grid Forum
      www.gridforum.org
      Edinburgh, July 22-24

      Chicago, Oct 15-17

foster@mcs.anl.gov                 ARGONNE  CHICAGO
                                                             32


                OGSA Implementation
 1) OGSA builds on infrastructure
       Plumbing: WSDL, WS-Security,
                                        OGSI/OGSA Interfaces
        WS-Routing/Referral, reliable      service description,
        messaging, transactions, etc.    service provisioning, …

       Hosting environments
                                        Standard        Web
 2) to enable virtualization via          OGSI        services
                                        container      various
       Service description
       Service provisioning
                                        Hosting Environment
 3) Standard container avoids           Resource virtualization
                                          and QoS support
   implementing OGSI features
   in every service instance
foster@mcs.anl.gov                         ARGONNE  CHICAGO
                                                                             33



             Building an OGSI Container
        Service data mgmt, query, subscription
            Container should provide simple interface for
             interacting with an instance’s implementation
             to get and manage dynamic service data
                 Service instance = CLR object
            Container should handle query processing
                 .NET support for XPath & Xquery allows for rich functionality
            Container manages notification subscriptions,
             and drives asynchronous notification messages
        Soft-state lifetime management
        Soft-state registration
foster@mcs.anl.gov                                       ARGONNE  CHICAGO

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:10/22/2012
language:English
pages:32