Docstoc

Ian Foster

Document Sample
Ian Foster Powered By Docstoc
					Globus       Toolkit ®

       Ian Foster
 Argonne National Laboratory
    University of Chicago
      Univa Corporation
                                 2



               Overview
Motivation
GT4
Future plans
The Globus community
TeraGrid and Open Science Grid
                                                 3



The Application-Infrastructure Gap

              Dynamic
               and/or
             Distributed
             Applications
   Shared Distributed Infrastructure
                             A           B


                         1           1



                                 9           9
                                                 4

        Bridging the Gap:
 Service-Oriented Infrastructure
                                  Users
Service-oriented applications        Composition
  Wrap applications as          Workflows
  services                             Invocation
  Compose applications
  into workflows           Appln        Appln
Service-oriented          Service      Service
infrastructure                  Provisioning
  Provision physical
  resources to support
  application workloads
                                               5

   Globus is Service-Oriented
   Infrastructure Technology
Software for service-oriented infrastructure
  Service enable new & existing resources
  E.g., GRAM on computer, GridFTP on
  storage system, custom application service
  Uniform abstractions & mechanisms
Tools to build applications that exploit
service-oriented infrastructure
  Registries, security, data management, …
Open source & open standards
  Each empowers the other
Enabler of a rich tool & service ecosystem
                                                                               6
               Globus as
     Service-Oriented Infrastructure
                             User                        User
         User                                          Application
                           Application
       Application                                        Tool
          Tool
                                    Workflow
                                                       User Svc
 Uniform interfaces,                            Host Env
security mechanisms,     Registry
Web service transport,                                      Credent.
     monitoring
                                                              DAIS
                       User Svc
        GRAM                           GridFTP                    IBM




                      Host Env
                                                                        IB M




                                          IBM


                                                IB M




                                                             Database
     Computers       Specialized       Storage
                      resource
                                     7
   A Typical eScience Use of Globus:
Network for Earthquake Eng. Simulation




Links instruments, data,
   computers, people
                                                                                                8
   An eBusiness Use of Globus:
SAP Demonstration @ GlobusWorld
3 Globus-enabled applns:
  CRM: Internet Pricing Configurator (IPC)
  CRM: Workforce           Web Browsers / Batch Processes
                             (typically several thousand requests)

  Management (WFM)                                                                            IPC
                                                                                             Server
  SCM: Advanced Planner                                    Request:
                                                          Price Query
                                                                                       2


  & Optimizer (APO)                                                  1        IPC
                                                                           Dispatcher
                                                                                      Delegation of
                                                                                        Request

Applications modified to:                                                              2

                                                                                             IPC
  Adjust to varying                                                      Response: PricelistServer
                                                                         Depending on:
  demand & resources                                                     - Time
                                                                         - Discount

  Use Globus to discover
                                                                         - Number of Items
                                                                         -… 3

  & provision resources                  SAP AG R/3 Internet Pricing
                                            & Configurator (IPC)
                                                                  9
                  Cheaper and/or Faster ...
 18 Servers                               11 Servers
                 Strategy A:
       A         Reduce TCO         A      1              B

   1
                                1   2                7    1   2

           9

                                            1
       B

                                    A                     B
   1

                  Strategy B:
           9
                   Improve      1   2                     1   2
                                                14
                 Performance
9 Servers Each                          16 Servers Each
                                 10



               Overview
Motivation
GT4
Future plans
The Globus community
TeraGrid and Open Science Grid
                                                 11



             Globus Toolkit
Core Web services
  Infrastructure for building new services
Security
  Apply uniform policy across distinct systems
Execution management
  Provision, deploy, & manage services
Data management
  Discover, transfer, & access large data
Monitoring
  Discover & monitor dynamic services
      Globus Toolkit version 2 (GT2)



                                                                         Web
                                                                       Services
                                                                     Components




   Pre-WS                    Grid Resource Monitoring
                                                        C Common
Authentication    GridFTP     Alloc. Mgmt & Discovery
Authorization                  (GRAM)                    Libraries
                                            (MDS)                      Non-WS
                                                                     Components




                              Execution      Info       Common
  Security       Data Mgmt
                                Mgmt       Services     Runtime
      Globus Toolkit version 3 (GT3)


 Community Data Access
Authorization & Integration                                               Web
                                                                        Services
     WS           Reliable    Grid Resource                           Components
                                                           Java
Authentication      File       Alloc. Mgmt     MDS3
Authorization                  (WS GRAM)                  WS Core
                  Transfer

   Pre-WS                     Grid Resource Monitoring
                                                         C Common
Authentication    GridFTP      Alloc. Mgmt & Discovery
Authorization                   (GRAM)                    Libraries
                                              (MDS)                     Non-WS
                                                                      Components
                   Replica                               eXtensible
                  Location                                IO (XIO)


                               Execution        Info     Common
  Security       Data Mgmt
                                 Mgmt         Services   Runtime
      Globus Toolkit version 4 (GT4)
                               Community
                   Data                                         C
 Delegation                    Scheduling       WebMDS
                 Replication                                  WS Core
                               Framework

 Community Data Access Workspace                  MDS-        Python
Authorization & Integration Management           Trigger      WS Core         Web
                                                                            Services
     WS           Reliable     Grid Resource                              Components
                                                 MDS-          Java
Authentication      File        Alloc. Mgmt
Authorization                   (WS GRAM)       Registry      WS Core
                  Transfer

   Pre-WS                                        Pre-WS
                                 Pre-WS                      C Common
Authentication    GridFTP                      Monitoring
Authorization                     GRAM                        Libraries
                                               & Discovery                  Non-WS
                                                                          Components
 Credential        Replica                                   eXtensible
   Mgmt           Location                                    IO (XIO)


                                Execution         Info       Common
  Security       Data Mgmt
                                  Mgmt          Services     Runtime
      Globus Toolkit version 4 (GT4)
                               Community
                   Data                                         C
 Delegation                    Scheduling       WebMDS
                 Replication                                  WS Core
                               Framework

 Community Data Access Workspace                  MDS-        Python
Authorization & Integration Management           Trigger      WS Core         Web
                                                                            Services
     WS           Reliable     Grid Resource                              Components
                                                 MDS-          Java
Authentication      File        Alloc. Mgmt
Authorization                   (WS GRAM)       Registry      WS Core
                  Transfer

   Pre-WS                                        Pre-WS
                                 Pre-WS                      C Common
Authentication    GridFTP                      Monitoring
Authorization                     GRAM                        Libraries
                                               & Discovery                  Non-WS
                                                                          Components
 Credential        Replica                                   eXtensible
   Mgmt           Location                                    IO (XIO)


                                Execution         Info       Common
  Security       Data Mgmt
                                  Mgmt          Services     Runtime
                                                                                                                               16



                           GT4 Components
                       Your      Your      Your                           Your                   Your
                                                                                                Your              Your
                                                                                                                  Your
CLIENT                Your      Your       Your                          Your
                                                                          Java                     C             Python
                       Java       CC      Python
                                          Python                         Java                     C              Python
                      Java                                                Client                 Client           Client
                       Client    Client
                                Client     Client
                                          Client                         Client                 Client           Client
                      Client




               Interoperable
                                                        X.509 credentials =
             WS-I-compliant
                                                        common authentication
            SOAP messaging


                                                     Your       Your




                                                                                                                               Pre-WS MDS
                                                                                                                 Pre-WS GRAM
    Your
                                     OGSA-DAI
                      Delegation




   Your




                                                                                     SimpleCA
                       Archiver




                                                                                                 MyProxy
                                                                           GridFTP
                                                    Python       C
                       Trigger




    Java
               GRAM




   Java                               GTCP
                        Index



                                       CAS
                RFT




                                                                                                           RLS
  Service                                           Service    Service
  Service
                                                    pyGlobus   C WS
                                                    WS Core    Core


             Java Services in Apache Axis Python hosting,                   C Services using GT
SERVER
            Plus GT Libraries and Handlers GT Libraries                    Libraries and Handlers
                                              17



          Our Goals for GT4
Usability, reliability, scalability, …
  Web service components have quality equal
  or superior to pre-WS components
  Documentation at acceptable quality level
Consistency with latest standards (WS-*,
WSRF, WS-N, etc.) and Apache platform
  WS-I Basic Profile compliant
  WS-I Basic Security Profile compliant
New components, platforms, languages
  And links to larger Globus ecosystem
                                               18



     GT4 Web Services Core
Supports both Globus services (GRAM, RFT,
Delegation, etc.) & user-developed services
Redesign to enhance scalability, modularity,
performance, usability
Leverages existing WS standards
  WS-I Basic Profile: WSDL, SOAP, etc.
  WS-Security, WS-Addressing
Adds support for emerging WS standards
  WS-Resource Framework, WS-Notification
Java, Python, & C hosting environments
  Java is standard Apache
                                                                   19



                GT4 Web Services Core

                           User Applications



                            Custom     GT4




                                                  Administration
                           WSRF Web WSRF Web




                                                    Registry
                Custom
GT4 Container




                            Services Services
                 Web
                Services
                           WS-Addressing, WSRF,
                              WS-Notification

                       WSDL, SOAP, WS-Security
                                               20


             Grid Security
Control access to shared services
  Address autonomous management, e.g.,
  different policy in different work-groups
Support multi-user collaborations
  Federate through mutually trusted services
  Local policy authorities rule
Allow users to set up dynamic trust domains
  Personal collection of resources working
  together based on trust of user
                                                                       21


                         GT4 Security
                             SSL/WS-Security
                             with Proxy
Authz Callout
                             Certificates
                                             Services (running
                                             on user’s behalf)

                         Access

  Compute                   CAS or VOMS
                                                                 Rights
   Center                   issuing SAML
                            or X.509 ACs

                                                            Users
                Rights
Local policy                                                     MyProxy
on VO identity             VO
or attribute
authority                                  Rights’
                                                                 KCA
                                                               22


              GT4 Security
Extensible authorization framework
based on Web services standards
  SAML-based authorization callout
    As specified in GGF OGSA-Authz WG

  Integrated policy decision engine
    XACML policy language, per-operation policies, pluggable

Credential management service
  MyProxy (One time password support)
Community Authorization Service
Standalone delegation service
  Implements IETF RFC
                                  23



GT4’s Use of Security Standards
                                               24



     GT4 Data Management
Stage large data to/from nodes
Replicate data for performance & reliability
Locate data of interest
Provide access to diverse data sources
  File systems, parallel file systems,
  hierarchical storage (GridFTP)
  Databases (OGSA DAI)
                                                             25



         GT4 Data Functions
Find your data: Replica Location Service
  Managing ~40M files in production settings
Move/access your data: GridFTP, RFT
  High-performance striped data movement
    27 Gbit/s memory-to-memory on a 30 Gbit/s link (90%
    utilization) with 32 IBM TeraGrid nodes.
    17.5 Gbit/s disk-to-disk limited by the storage system
    Reliable movement of 120,000 files (so far)

Couple data & execution management
  GRAM uses GridFTP and RFT for staging
                                                                               p g
                                                                                                          26
                                           20000
                                           18000
                                                          Disk-to-disk on
                                           16000
                                                          TeraGrid




                        Bandwidth (Mbps)
GridFTP in GT4
                                           14000
                                           12000
                                           10000
                                            8000

 100% Globus code
                                            6000
                                            4000
                                            2000
   No licensing issues                         0
                                                   0     10      20      30      40        50      60          70

   Stable, extensible                                                 Degree of Striping

                                                       # Stream = 1       # Stream = 2          # Stream = 4
 IPv6 Support                                          # Stream = 8       # Stream = 16         # Stream = 32



 XIO for different transports
 Striping    multi-Gb/sec wide area transport
 Pluggable
   Front-end: e.g., future WS control channel
   Back-end: e.g., HPSS, cluster file systems
   Transfer: e.g., UDP, NetBLT transport
                                                                             27
                     Reliable File Transfer:
                      Third Party Transfer
 Fire-and-forget transfer                        RFT Client
 Web services interface               SOAP                    Notifications
 Many files & directories            Messages                  (Optional)

 Integrated failure recovery                     RFT Service

 Has transferred 900K files
GridFTP Server                                           GridFTP Server

   Master      Protocol     Data        Data          Protocol    Master
    DSI      Interpreter   Channel     Channel      Interpreter    DSI

       IPC Link                                          IPC Link

    IPC           Slave     Data        Data          Slave         IPC
  Receiver         DSI     Channel     Channel         DSI        Receiver
                                                         28



        Replica Location Service
Identify location of files
                                   Index    Index
via logical to physical
name map
Distributed indexing of
names, fault tolerant
update protocols             Local Update Bloom Bloom
                              DB    send   filter filter
GT4 version scalable &
                                   (secs) (secs) (bits)
stable                       10K     <1      2      1M
Managing ~40 million         1M       2     24      10 M
files across ~10 sites
                             5M       7     175     50 M
                                        29



Execution Management (GRAM)
Common WS interface to schedulers
  Unix, Condor, LSF, PBS, SGE, …
More generally: interface for process
execution management
  Lay down execution environment
  Stage data
  Monitor & manage lifecycle
  Kill it, clean up
A basis for application-driven
provisioning
                                          30



              GT4 GRAM
2nd-generation WS implementation
optimized for performance, flexibility,
stability, scalability
Streamlined critical path
  Use only what you need
Flexible credential management
  Credential cache & delegation service
GridFTP & RFT used for data operations
  Data staging & streaming output
  Eliminates redundant GASS code
                                                                                              31

                       GT4 GRAM Structure:
                      WSRF/WSN Poster Child
                                  Service host(s) and compute element(s)

                     GT4 Java Container                                      Compute element
                                             Local job
                             GRAM
                            GRAM              control
                            services                                               Local
                           services
         Job tions




                                                           sudo
                                             Dele                 GRAM           scheduler
          func                                      gate          adapter
Client




                                  Transfer
                     Delegation   request
         Delegate
                                                                  GridFTP        User
                           RFT File
                                              FTP                                 job
                           Transfer
                                              control
                                                                      FTP data
                                                                                 Remote
                                                                  GridFTP        storage
                                                                                 element(s)
                    32



Workspace Service
                                              33



    Monitoring and Discovery
“Every service should be monitorable and
discoverable using common mechanisms”
  WSRF/WSN provides those mechanisms
A common aggregator framework for
collecting information from services, thus:
  MDS-Index: Xpath queries, with caching
  MDS-Trigger: perform action on condition
  (MDS-Archiver: Xpath on historical data)
Deep integration with Globus containers &
services: every GT4 service is discoverable
  GRAM, RFT, GridFTP, CAS, …
                                                              34
                           GT4
                  Monitoring & Discovery                Clients
                                                   (e.g., WebMDS)
                     GT4 Container
WS-ServiceGroup
                         MDS-
                         Index
 Registration &
WSRF/WSN Access
                          adapter

                                                  GT4 Cont.
   GT4 Container
          MDS-              Custom protocols
                                                   MDS-
          Index           for non-WSRF entities    Index
    Automated
   registration                      GridFTP
   in container
                                                    RFT
   GRAM           User
     GT4
Documentation
      is
Much Improved!
                                                36



         Working with GT4
Download and use the software, and provide
feedback
  Join gt4friends@globus.org mail list
Review, critique, add to documentation
  Globus Doc Project: http://gdp.globus.org
Tell us about your GT4-related tool, service,
or application
  Email foster@mcs.anl.gov
                                 37



               Overview
Motivation
GT4
Future plans
The Globus community
TeraGrid and Open Science Grid
                                               38



          2005 and Beyond
We have a solid Web services base
We now want to build, on that base, a
open source service-oriented infrastructure
  Virtualization
  New services for provisioning, data
  management, security, VO management
  End-user tools for application development
  Etc., etc.
                                            39

      Short-Term Priorities:
            Security
Improve GSI error reporting & diagnostics
Secure password, one-time password,
Kerberos support for initial log on
Trust roots, use of GridLogon
Identity/attribute assertions in GT auth.
callouts (e.g., Shib, PERMIS, VOMS, SAML)
Extend CAS admin & policy support
Security logging with management control
for audit purposes
                                          40

      Short-Term Priorities:
       Data Management
Space & bandwidth management in GridFTP
Concurrency in globus-url-copy
Priorities in RFT
Data replication service
Enhance policy support in data services
Physical file name creation service
Scalable & distributed metadata manager
                                             41

      Short-Term Priorities:
     Execution Management
Implement GGF JSDL once finalized
Advance reservation support
Policy-driven restart of “persistent” jobs
Improved information collection for jobs
Improved management of job collections
Credential refresh
Virtual workspace service
Integration of virtual machines (Xen,
VMware) and associated services
                                            42

      Short-Term Priorities:
      Information Services
Many more information sources, including
gateways to other systems
Automated configuration of monitoring
Specialized monitoring displays
Performance optimization of registry
Archiver service
Helper tools to streamline integration of
new information sources
                                                43

      Short-Term Priorities:
              Core
Streamlined container configuration
Remote management interface
Dynamic service deployment
Service isolation: multiple service instances
WS-Notification, subscription performance
Full functionality in C WS Core
Optimized WS-ServiceGroup support
WS-SecureConversation support
                                 44



               Overview
Motivation
GT4
Future plans
The Globus community
TeraGrid and Open Science Grid
                                                45


        How Globus Works
Globus is a distributed open source
community with many contributors & users
  CVS, documentation, bugzilla, email lists
  Modular structure allows many to contribute
Globus Alliance Board provides
governance when needed
  Meritocracy: individuals who demonstrate
  ongoing contributions & commitment
  Primarily: what to include, when to release
Globus Alliance is an informal partnership
of organizations led by Board members
                                             46



Evolution of the Globus Alliance
Argonne/U.Chicago (Childers, Foster): 1995
USC/ISI (Kesselman): 1995
Edinburgh (Atkinson, Parsons): 2003
Swedish PDC (Johnsson, Mulmo): 2003
NCSA (Welch): 2004
Univa (Czajkowski, Tuecke): 2004


Other contributors will surely be added
                                                47



  From eScience to eBusiness
Since ~2001, growing interest in Globus
for commercial use
  Enterprises, IT vendors, ISVs asking Globus
  leaders to address commercial needs
  But hard to do in a research laboratory
In response, we have created two new
organizations
  Globus Consortium
  Univa
                                                 48

     Globus Consortium
 (www.globusconsortium.com)
Nonprofit organization funded by companies
to advance Globus Toolkit for enterprise use
  Initial sponsor members: HP, IBM, Intel, Sun
  Initial contributors: Nortel, Univa
First two projects already identified
  Member-driven software quality improvements
  Contributions to job submission standards
Other projects to be defined, e.g.
  Develop new features key to enterprise use
  Education & outreach
                                              49




Provider of commercial support, services, &
products around open source Globus
  Commercial distribution of GT4 & beyond
  Integration with enterprise systems
  Committed to open source & open standards
Founded by Tuecke, Foster, Kesselman
  Tuecke left Argonne to be CEO
  Foster, Kesselman remain at Argonne, ISI
Experienced management team
  Rich Miller, Vas Vasiliadis, Paul Davé,
  Bob Mandel
26776 U.S.                                          68 Belgium     20 Finland            7 Ecuador      3 Mongolia           1 Mozambique




               38669 downloads in 2004 from globus.org
2753 China                                          66 Venezuela   20 New Zealand        7 Macedonia    3 Sri Lanka          1 Tokelau
1318 Japan                                          66 Romania     17 Nigeria            6 Bolivia      3 Wallis & Futuna Is. 1 Greenland
1017 India                                          64 Indonesia   17 South Africa       6 Comoros      3 Belarus            1 Nepal
750 U.K.                                            62 Mexico      16 Jordan             6 Zaire        3 Bangladesh         1 Swaziland
495 Italy                                           61 Turkey      16 Slovenia           6 Lebanon      2 Falkland Islands   1 Iraq
488 Germany                                         60 Malaysia    16 Afghanistan        5 Puerto Rico 2 Kuwait              1 Serbia
391 Brazil                                          58 Portugal    15 Denmark            5 Namibia      2 Sierra Leone       1 Barbados
328 S. Korea                                        57 Austria     15 Philippines        5 Togo         2 Trinidad & Tobago 1 Cambodia
306 Taiwan                                          54 Ireland     14 Vanuatu            5 Tajikistan   2 Guyana             1 Qatar
268 France                                          44 Hong Kong 14 Luxembourg           5 Paraguay     2 American Samoa     1 Saint Vincent
241 Canada                                          40 Hungary     14 Tunisia            5 Albania      2 Andorra            1 Laos
211 Viet Nam                                        38 Egypt       12 Virgin Is. (U.K.) 5 Sudan         2 Georgia            1 San Marino
211 Spain                                           38 Argentina   12 Peru               4 Estonia      2 Cook Islands       1 Libya
202 Russia                                          34 Uruguay     12 Yemen              4 Camaroon     2 Turkmenistan       1 Benin
187 Pakistan                                        31 Ukraine     11 Norway             4 Ghana        2 Gabon              1 Angola
159 Australia                                       29 Slovakia    11 Algeria            4 Tuvalu       2 The Gambia         1 Chad
142 Singapore                                       25 Israel      11 Armenia            4 Costa Rica   2 Kazakhstan         1 Gibraltar
131 Greece                                          23 Yugoslavia 10 Iceland             4 Cuba         2 Macau              1 Haiti
119 Colombia                                        23 Iran        9 Zambia              4 UAE          2 Malta              1 Guatemala
111 Poland                                          22 Bulgaria    9 Virgin Is. (U.S.)   4 Tonga        2 Jamaica            1 Malawi
109 Netherlands 22 Uzbekistan 9 Uganda                                                   4 W. Samoa     2 Latvia             1 Equatorial Guinea
107 Thailand                                        22 Czech Rep. 9 Bosnia & Herz.       4 Tanzania     2 Turks & Caicos     1 Palau
94 Switzerland                                      22 N. Korea    8 Kenya               3 Syria        1 Bhutan             1 Bermuda
77 Chile                                            21 Lithuania   7 Zimbabwe            3 Bahamas      1 Ascension Island   1 Botswana
74 Sweden                                           21 Croatia     7 Saudi Arabia        3 Ethiopia     1 Cyprus             1 Suriname
26776 U.S.                                          68 Belgium     20 Finland        7 Ecuador     3 Mongolia           1 Mozambique




               38669 downloads in 2004 from globus.org
2753 China                                          66 Venezuela   20 New Zealand    7 Macedonia   3 Sri Lanka          1 Tokelau
1318 Japan                                          66 Romania     17 Nigeria        6 Bolivia     3 Wallis & Futuna Is. 1 Greenland
1017 India                                          64 Indonesia   17 South Africa   6 Comoros     3 Belarus            1 Nepal
750 U.K.                                            62 Mexico      16 Jordan         6 Zaire       3 Bangladesh         1 Swaziland
495 Italy                                           61 Turkey      16 Slovenia       6 Lebanon     2 Falkland Islands   1 Iraq
488 Germany                                                Top1610
                                                    60 Malaysia Afghanistan          5 Puerto Rico 2 Kuwait             1 Serbia
391 Brazil                                          58 Portugal    15 Denmark        5 Namibia     2 Sierra Leone       1 Barbados
328 S. Korea
306 Taiwan
                        26776 U.S. 5 Togo
                54 Ireland
                              15 Philippines
                              14 Vanuatu
                                                    57 Austria
                                                   5 Tajikistan
                                                                                                   2 Trinidad & Tobago 1 Cambodia
                                                                                                   2 Guyana             1 Qatar
268 France                 2753 China 5 Paraguay
                44 Hong Kong 14 Luxembourg                                                         2 American Samoa     1 Saint Vincent


                38 Egypt 1318 Japan 5 Sudan
241 Canada      40 Hungary    14 Tunisia           5 Albania                                       2 Andorra            1 Laos
211 Viet Nam                  12 Virgin Is. (U.K.)                                                 2 Georgia            1 San Marino
211 Spain
202 Russia      34 Uruguay
                           1017 India 4 Estonia
                38 Argentina 12 Peru
                              12 Yemen             4 Camaroon
                                                                                                   2 Cook Islands
                                                                                                   2 Turkmenistan
                                                                                                                        1 Libya
                                                                                                                        1 Benin
187 Pakistan    31 Ukraine 750 U.K.
                              11 Norway            4 Ghana                                         2 Gabon              1 Angola
159 Australia   29 Slovakia
142 Singapore 25 Israel
                             495 Italy 4 Costa Rica
                              11 Algeria
                              11 Armenia
                                                   4 Tuvalu                                        2 The Gambia
                                                                                                   2 Kazakhstan
                                                                                                                        1 Chad
                                                                                                                        1 Gibraltar
131 Greece      23 Yugoslavia488 Germany
                              10 Iceland           4 Cuba                                          2 Macau              1 Haiti
119 Colombia    23 Iran       9 Zambia             4 UAE                                           2 Malta              1 Guatemala
111 Poland      22 Bulgaria 391 Brazil 4 Tonga
                              9 Virgin Is. (U.S.)                                                  2 Jamaica            1 Malawi


107 Thailand
                             328 S. Korea
109 Netherlands 22 Uzbekistan 9 Uganda
                22 Czech Rep. 9 Bosnia & Herz.
                                                   4 W. Samoa
                                                   4 Tanzania
                                                                                                   2 Latvia
                                                                                                   2 Turks & Caicos
                                                                                                                        1 Equatorial Guinea
                                                                                                                        1 Palau
94 Switzerland 22 N. Korea 306 Taiwan
                              8 Kenya              3 Syria                                         1 Bhutan             1 Bermuda
77 Chile                                            21 Lithuania   7 Zimbabwe        3 Bahamas     1 Ascension Island   1 Botswana
74 Sweden                                           21 Croatia     7 Saudi Arabia    3 Ethiopia    1 Cyprus             1 Suriname
  Global
Community
                                            53

      The Globus Ecosystem
Globus components address core issues
relating to resource access, monitoring,
discovery, security, data movement, etc.
  GT4 being the latest version
A larger Globus ecosystem of open
source and proprietary components
provide complementary components
  A growing list of components
These components can be combined to
produce solutions to Grid problems
  We’re building a list of such solutions
                                              54
    Many Tools Build on, or Can
   Contribute to, GT4-Based Grids
Condor-G, DAGman              VOMS
MPICH-G2                      PERMIS
GRMS                          GT4IDE
Nimrod-G                      Sun Grid Engine
Ninf-G                        PBS scheduler
Open Grid Computing Env.      LSF scheduler
Commodity Grid Toolkit        GridBus
GriPhyN Virtual Data System   TeraGrid CTSS
Virtual Data Toolkit          NEES
GridXpert Synergy             IBM Grid Toolbox
Platform Globus Toolkit       …
                                                      55
                                     Documenting
                                       The Grid
                                      Ecosystem

The Grid Ecosystem: Software Components for Grid Systems
And Applications




                                 www.grids-center.org
                                        56



        Example Solutions
Portal-based User Reg. System (PURSE)
VO Management Registration Service
Service Monitoring Service
TeraGrid TGCP Tool
Lightweight Data Replicator
GriPhyN Virtual Data System
                                                              57



The Globus Developer Community
 We want to facilitate community
 contributions to the Globus software
   Increasingly open governance & mechanisms
   Documentation of larger ecosystem
   What else can we do?
 Tell us how you think we should evolve
   E.g., adopt Apache processes?
   Contribute more of Globus to Apache?
     We’ve already made major contributions to Apache Axis,
     WS-Addressing, and WS-Security
     Also WSRF/WSN     Apache (Apollo, Hermes)
                                               58



Globus and its User Community
How can “we” best support “you”?
  We try to provide the best software we can
  We use bugzilla & other community tools
  We work to grow the set of contributors
How can “you” best support “us”?
  Become a contributor: of software, bug
  fixes, answers to questions, documentation
  Provide us with success stories that can
  justify continued Globus development
  Promote Globus within your communities
Contact me at foster@mcs.anl.gov
                                 59



               Overview
Motivation
GT4
Future plans
The Globus community
TeraGrid and Open Science Grid
Grid2003     Open Science Grid                 60

 30 sites (2100-2800 CPUs) & growing
 400-1300 concurrent jobs
 8 substantial applications + CS experiments
 Running since October 2003




           Korea

             http://www.ivdgl.org/grid2003
                                                            61

 Open Science Grid Components
Computers & storage at 30+ sites
  3000+ CPUs
Uniform service environment at each site
  Globus Toolkit provides basic authentication, execution
  management, data movement
  Pacman installation system enables installation of
  numerous other VDT and application services
Global & virtual organization services
  Certification & registration authorities, VO membership
  services, monitoring services
Client-side tools for data access & analysis
  Virtual data, execution planning, DAG management,
  execution management, monitoring
IGOC: iVDGL Grid Operations Center
                                              62


    Example OSG Applications

CMS proton-proton collision simulation
ATLAS proton-proton collision simulation
LIGO gravitational wave search
SDSS galaxy cluster detection
ATLAS interactive analysis
BTeV proton-antiproton collision simulation
SnB biomolecular analysis
GADU/Gnare genone analysis
Various computer science experiments
  www.ivdgl.org/grid2003/applications
                Genome 63

               sequence
                analysis
   Example
     OSG
   Workflows




   Sloan
                Physics
digital sky
                 data
  survey        analysis
                                                             64

           Open Science Grid :
      Astronomy Mosaic Construction
   Construct custom mosaics on demand from multiple
   data sources
   User specifies projection, coordinates, size, rotation,
   spatial sampling




NVO/NASA Montage: A small (1200 node) workflow
Work by Ewa Deelman et al., USC/ISI and Caltech

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:8/14/2011
language:French
pages:64