gt seminar

					                     Globus Toolkit 4

                          By:
           Jeton Bacaj, Joel Lathrop, Fugang
              Wang & Andrew Younge

           http://blackrose02.rit.edu/wiki/doku.php?id=grid:seminar1:gt4




                  Rochester Institute of Technology
21:11:16              Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   1
                                              Outline
• Introduction
• Components
       –   Execution Management
       –   Information Services
       –   Data Management
       –   Security
• Globus Implementation
       –   Architectural overview
       –   Globus and Rocks
       –   Todo
       –   Job submission
• Summary

21:11:16            Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   2
            Introduction to Globus Toolkit
• What is Globus?
     – A “grid” toolkit
     – The goal is, “Letting people share computing power, databases,
       and other tools securely online across corporate, institutional,
       and geographic boundaries without sacrificing local autonomy.”
• Modular Architecture
     – Made of many components
     – Not all components are inter-dependent
     – Can install single, isolated components
• Globus Advantages
     –      Scalable
     –      Well documented API
     –      Open source
     –      Widely used

 21:11:16            Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   3
                     Introduction (cont)
• Components organized into classes
   – Execution Management
           • Initiation, monitoring, management, scheduling, and coordination
             of remote computational tasks
       – Information Services
           • Monitoring, managing logs, archiving data, etc.
       – Data Management
           • Data location, transfer, and management
       – Security
           • Authentication, mapping Globus credentials with other forms of
             credentials
       – Common Runtime
           • A collection of APIs allowing integration with different languages
             and platforms

21:11:16            Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   4
                Execution Management
• Community Scheduler Framework
       – Interface to resource schedulers
• Grid TeleControl Protocol
       – WRSF-enabled service interface for control of remote
         instruments
• Workspace Management
       – Dynamic fostering of managing workspaces on hosts
• Grid Resource Allocation & Management
       – “GRAM”; the bread „n butter of Globus Toolkit‟s Execution
         Management – deploying and monitoring jobs on a Grid
           • Used to incorporate a wide variety of other grid and cluster services such as
             Condor, PBS, LSF, SGE, BOINC and more!


21:11:16               Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   5
                   Information Services
Some of the services that are included on Globus Toolkit:
• MDS
     – Collects data about different resources about the grid
     – By default, uses WSRF for polling operations
     – WebMDS uses MDS that formats data in XML into HTML using XSLT
       style sheets
• Indexing
     – Services register with this component so others can utilize those
       resources
     – Caches resources of the registered services (speed obviously)
• Trigger
     – What it sounds like - if there is a condition that‟s met, an alert is raised
       (notification via email for example)
     – Monitors behavior - looks at the data with some preconfigured tests to
       see if trigger conditions are met


 21:11:16            Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   6
Data Management and Replication
• Data Management
       – GridFTP
           • A Souped up version of FTP that‟s secure, robust and
             efficient
           • Good for bulk data.
       – Reliable File Tranfter (RFT)
           • More reliable, job based file transfer using web services
• Data Replication
       – Replica Location Service (RLS)
           • Keeps track of location of the copies in the grid
       – Data Replication Service (DRS)
           • Higher level merge of RFT and RLS
21:11:16           Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   7
    Grid Security Infrastructure - GSI

• Conform to X.509 based PKI standard
• Certificate
       – Used to identify and authenticate
       – Structure
          • (subjID, publicKey, CA‟s ID, CA‟s signature)
          • Coded in X.509 format
       – Host, service, user, proxy
• Mutual authentication
       – Use SSL/TLS to exchange certificates
       – Prove identity by signing a message with private key


21:11:16            Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   8
    Grid Security Infrastructure - GSI

• Confidential communication
       – By default no encryption
       – Easy to setup a shared key encryption
• Securing private key
       – Pass phrase protected private key
• Proxy and certificate delegation
       – Chain of trust
       – RFC 3820


21:11:16         Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   9
                         GSI - MyProxy
• MyProxy credential management service




                                                                                  Grid
                   User_home
                                                                                Front End


                                                   MyProxy
           CA
                                                    Server




21:11:16        Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu        10
                    GSI - MyProxy
• MyProxy credential management service

                                          User_mobile



                                                                             Grid
                                                                           Front End


                                              MyProxy
                                               Server




21:11:16   Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu        11
                 MyProxy Commands
• MyProxy related commands in GT4
       myproxy-init: store proxy credential
       myproxy-store: store end-entity credential
       myproxy-logon: retrieve a stored credential
       myproxy-retrieve: retrieve end-entity credential

       myproxy-destroy: remove credential
       myproxy-info: query stored credentials

       myproxy-change-pass-phrase:
       change password encrypting private key



21:11:16            Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   12
                                GSI - In Practice
• User certificate generation
       User generates certificate request and private key




           USER$ echo export GLOBUS_LOCATION=/usr/local/globus-4.0.6 >> ~/.bashrc
           USER$ $GLOBUS_LOCATION/etc/globus-user-env.sh
           USER$ grid-cert-request

           Private key(passphrase protected):
                      ~/.globus/userkey.pem
           Certificate request:
                      ~/.globus/usercert_request.pem

           ! Mail the certificate request to the administrator as prompted




21:11:16                   Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   13
                           GSI - In Practice
• User certificate generation
       User generates certificate request and private key


                    CA signs the certificate




    globus$ grid-ca-sign –in usercert-request.pem -out signed_USER.pem
    globus$ sudo cp signed_USER.pem /home/USER/.globus/usercert.pem




21:11:16              Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   14
                             GSI - In Practice
• User certificate generation
       User generates certificate request and private key


                      CA signs the certificate


                     Add entry in gridmap file


           /etc/grid-security/grid-mapfile


           "/O=Grid/OU=GlobusTest/OU=simpleca-lilac03.rit.edu/OU=rit.edu
           /CN=FIRST_NAME LAST_NAME" USER




21:11:16                Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   15
                                     Grid Architecture
                                                                 Grid users
                                                                     lilac03.rit.edu
                                                             Main Globus Server
                                                                     globus-4.0.6
                                                                    grid scheduler
                                                     Certificate Authority




               lilac04.rit.edu                                 clover.rit.edu                       Condor/PBS/BOINC   ...
              Rocks frontend                                      VALinux
                globus 4.0.3                                      Rocks?
           Rocks queue manager

     lilac02.rit.edu   lilac01.rit.edu                                                                    ??
      compute-0-0       compute-0-2       clover02.rit.edu    clover03.rit.edu   clover04.rit.edu
                                                                                                          ...
     lilac05.rit.edu   lilac08.rit.edu
      compute-0-1       compute-0-3



21:11:16                         Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu                        16
                                          Progress
• Main Globus installation
       –   Used Ubuntu Linux 7.10 Server
       –   Compiled and installed version 4.0.6 of Globus
       –   Setup security using Simple-CA
       –   Configured postgresql RFT Database
       –   Started Globus container
       –   Added test users and submitted test jobs
• Rocks Cluster
       – Installed the frontend on lilac04.rit.edu
       – Added Globus, Ganglia, and SGE roles to the frontend
       – Installed clients on other lilac machines.


21:11:16            Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   17
                                      Todo List
• Link Rocks frontend to main Globus installation
       – Configure the Certificates
       – Setup and test GridFTP between machines
       – Update /etc/grid-security/grid-mapfile
• Test the grid using by submitting a job to a
  remote resource (Rocks)
• Setup independent CA behind a firewall using
  MyProxy
       – Would enhance security and simplify usage



21:11:16         Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   18
                         Submitting a Job
• Submitting a simple job
     xero@lilac03:~$ globusrun-ws -submit -c /bin/echo ROAR
     Submitting job...Done.
     Job ID: uuid:a07e6908-fbb8-11dc-884b-000874d2a480
     Termination time: 03/28/2008 04:45 GMT
     Current job state: Active
     Current job state: CleanUp
     Current job state: Done
     Destroying job...Done.
• Job submission in Globus gets more complicated quickly
       – Have to use large xml (RSL) files to specify job details
       – Use a job_epr.xml file to monitor job status
       – Make sure job results are retrieved

21:11:16             Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   19
                                      Summary
• The Globus Toolkit is a powerful middleware
  application that allows one to build a multi-level grid
  computing system that can span spatially and
  administratively different computing systems.
• Setting up and managing a grid using Globus is a lot
  of work!
• Globus can work with a wide variety of clusters,
  queuing systems or other grid technologies in order
  to provide the largest amount of resources
       – System integration is unmatched by anything else
       – Many tools already available for free
       – Easy API to create custom tools for specific needs

21:11:16          Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   20
                           Questions?




• Ask, and ye shall receive …




21:11:16   Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu   21
                                      References
Foster, "A Globus Primer: Describing Globus Toolkit 4," 2005.
I. Foster, "Globus Toolkit 4: Software for Service-Oriented Systems," in Proc. Network And
          Parallel Computing: IFIP International Conference, 2005, pp. 2-13.
C. K. Ian Foster, "Globus: a Metacomputing Infrastructure Toolkit," Iternational Journal of High
          Performance Computing Applications, vol. 11, iss. 2, pp. 115-128, 1997.
L. P. N. M. C. K. I. F. M. D. Jennifer M. Schopf Ioan Raicu, "Monitoring and Discovery in a Web
          Services Framework: Functionality and Performance of Globus Toolkit MDS4,"
          Argonne National Laboratory2005.
W. Allcock, J. Bresnahan, R. Kettimuthu, and M. Link, "The Globus Striped GridFTP Framework
          and Server," in Proc. SC '05: Proceedings of the 2005 ACM/IEEE conference on
          Supercomputing, Washington, DC, USA, 2005, p. 54.
H. Kanemitsu and Y. Urano, "On the method for realizing globally and locally accessible
          resource management for WS-GRAM," in Proc. Web Services, 2007. ICWS 2007.
          IEEE International Conference on, 2007, pp. 1201-1204.
X. Zhang and J. M. Schopf, Performance Analysis of the Globus Toolkit Monitoring and
          Discovery Service, MDS2, 2004.
I. N. Borja Sotomayor Lisa Childers, Globus Toolkit 4 Programming Java Services, Morgan
          Kaufmann, 2006.


21:11:16              Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu     22

				
DOCUMENT INFO