Docstoc

R-GMA Revisited

Document Sample
R-GMA Revisited Powered By Docstoc
					                                            WP3

RGMA Deployment




  Laurence Field / RAL <l.field@rl.ac.uk>
 Steve Fisher / RAL <s.m.fisher@rl.ac.uk>
    R-GMA                                                                   WP3
                                           Producer

•   Uses the Grid
    Monitoring Architecture
    from Global Grid Forum
                                                             Registry
•   R-GMA is a relational
    implementation

•   Applied to both
    information and
    monitoring
                                                              Information flow
•   Creates impression that                                   Meta-data flow
    you have one RDBMS
    per Virtual Organisation
                                         Consumer

R-GMA Deployment               Laurence Field   30/01/2003                     2
   SiteInfo Example                                                                       WP3
                     dn: in=siteinfo,Mds-Vo-name=ralrgma,o=grid
                     objectclass: SiteInfo

   Gin               objectclass: DataGridTop
                     objectclass: DynamicObject
                     siteName: RALRGMA
                                                                                   Gout
                     sysAdminContact: edg-site-admin@gridpp.rl.ac.uk
                     userSupportContact: edg-site-admin@gridpp.rl.ac.uk
                     siteSecurityContact: edg-site-admin@gridpp.rl.ac.uk
                     dataGridVersion: v1_4_3
                     installationDate: 20021111103000Z
                     createTimestamp: 20030129111208Z
                     modifyTimestamp: 20030129111208Z
                     entryTtl: 1200



                              dn                      siteName    sysAdminContact


             in=siteinfo,Mds-Vo-name=ralrgma,o=grid    RAL       admin@gridpp.rl.ac.uk




R-GMA Deployment                                Laurence Field    30/01/2003              3
   Gin                                                                  WP3




   • Uses the same information              R-GMA                MDS
     provider as MDS

   • Can run MDS and R-GMA in           CircularBuffer
                                          Producer
     parallel                                                    GRIS
                                             Gin

   • Can compare information
     systems
                                                Information Provider




R-GMA Deployment           Laurence Field   30/01/2003                  4
   Gout                                                                    WP3

   • LDAP Server is queried by RB

   • Two RBs should be used currently in the UK                  LDAP
        – one for MDS and one for R-GMA                          Server



                              Consumer               Archiver
                                (CE)
                                                                Consumer
                                                   DataBase
                              Consumer                            API
                                                   Producer
             R-GMA              (SE)

                              Consumer
                              (SiteInfo)            RDBMS

                            GOUT


R-GMA Deployment              Laurence Field   30/01/2003                  5
   CE and SE Tables                                                                  WP3

                                    CloseStorage                    StorageElement
 ComputingElement
                                      Element                            status
 dn                             dn                                 dn
 CEId                           CEId                               SEId
 TotalCPUs                      CloseSE                            SEfreespace
 FreeCPUs                       ……                                 ……
 TotalJobs
 RunningJobs
 ……


“Select a ComputingElement with at least 1 free CPU that also has a
CloseStorageElement with at least 1000 MB of free space”

SELECT DISTINCT ComputingElement.CEId FROM
ComputingElement, CloseStorageElement,StorageElementStatus WHERE
ComputingElement.FreeCPUs > 0 AND
(ComputingElement.CEId = CloseStorageElement.CEId AND
CloseStorageElement.CloseSE = StorageElementStatus.SEId AND
StorageElementStatus.SEfreespace > 1000)

R-GMA Deployment                    Laurence Field   30/01/2003                      6
   InfoCat                                                WP3

   • One logical registry per VO
        – Replication/Distribution system ready soon
   • At the moment only using one for all VOs
   • Located at RAL
        – Infocat.gridpp.rl.ac.uk
   • InfoCat machine also hosts BrowserServlet




R-GMA Deployment            Laurence Field   30/01/2003   7
   R-GMA Deployment                                                            WP3
        Information flow        Information Catalogue

        MetaData flow          Registry           Schema
                               Servlet            Servlet
     Server Config                                              Resource Broker
                                         Tomcat
        Producer                                                   Archiver
         Servlet                                                   Servlet
         Tomcat                  Site A                             Tomcat
                               Client Config

                                                                   Archiever
      Producer API            Producer API                            API

          Gin                      Gin                               Gout


   Information Provider    Information Provider                     LDAP


R-GMA Deployment                 Laurence Field    30/01/2003                  8
   R-GMA Tools                                                      WP3
• R-GMA CLI
     – Command Line Interface (similar to MySQL)
     – Supports single query and interactive modes
• R-GMA Browser
     – JSP application dynamically generating web pages
     – Supports pre-defined and user-defined queries
• Pulse
     – R-GMA Java client-based GUI
     – Supports streaming and simple graphical displays
• Nagios
     – R-GMA plug-in instantiates an Archiver and uses that information
       to populate Nagios displays
     – Users can benefit from Nagios alert mechanism

R-GMA Deployment              Laurence Field   30/01/2003            9
   Service Status                                                               WP3

   • Monitors EDG components
   • Provides a standard interface for plug-in
     service status scripts
        – Return 0 or 1 OK or fail
        – Returns message from standard out
   • Browsing via the web

       hostName      name     VO        status   message     measurementTime


      gpp.rl.ac.uk   R-GMA   GridPP      OK       Running      20021111103000




R-GMA Deployment                      Laurence Field   30/01/2003               10
   How to upgrade                                   WP3




   • Based on edg version 1_4_3
   • Install rpms need for R-GMA
   • Configure via lcfg (or manually)




R-GMA Deployment      Laurence Field   30/01/2003   11
   RPMS                                                               WP3
   EDG RPMS                             R-GMA
   edg-info-main-2.0.1-0                edg-rgma-api-c-2.2.8-1
   CE                                   edg-rgma-common-2.2.8-1
   edg-info-siteinfo-1.0.0-1            edg-rgma-api-cpp-2.2.8-1
   SE                                   edg-rgma-demo-2.2.8-1
   edg-info-se-2.0.0-4                  edg-rgma-api-java-2.2.8-1
                                        edg-rgma-servlets-2.2.8-5
   Externals                            edg-rgma-api-perl-2.2.8-1
   j2sdk-1.4.1-fcs                      edg-rgma-tools-2.2.8-5
   log4cpp-0.2.7-3                      edg-rgma-api-python-2.2.8-1
   mm.mysql-2.0.6-2
   xerces-c-1.7.0-1                     LCFG
   MySQL-python2.1-0.9.2-1              obj-rgma-2.2.8-5
   python2-2.1.3-2                      obj-infoproviders-2.0.1-9
   log4j-1.2.6-1jpp
   SWIG-1.3.11-edg1

R-GMA Deployment               Laurence Field   30/01/2003            12
   LCFG R-GMA                                                          WP3
   rgma.servletlocation localhost
   rgma.registrylocation INFOCAT_HOSTNAME
   rgma.GOUT no

   rgma.service RGMA MDS GIN

   rgma.VO_RGMA SE_GDMP_VOS
   rgma.cmd_RGMA /opt/edg/bin/rgma-servlet-status /etc/tomcat4/server.xml

   rgma.VO_MDS SE_GDMP_VOS
   rgma.cmd_MDS /etc/rc.d/init.d/globus-mds status

   rgma.VO_GIN SE_GDMP_VOS
   rgma.cmd_GIN /etc/rc.d/init.d/edg-rgma-gin status


R-GMA Deployment                Laurence Field   30/01/2003             13
   LCFG InfoProviders                                                    WP3
   infoproviders.GRID_INFO_USER edginfo
   infoproviders.INFO_LOCATION   info
   infoproviders.INFO_PROVIDER_LOG /opt/edg/var/info/info-provider-log
   infoproviders.SIenabled yes

   infoproviders.siteName            SITE_NAME_
   infoproviders.sysAdminContact     SITE_MAILROOT
   infoproviders.userSupportContact SITE_MAILROOT
   infoproviders.siteSecurityContact SITE_MAILROOT
   infoproviders.dataGridVersion     SITE_EDG_VERSION
   infoproviders.installationDate    SITE_INSTALLATION_DATE_
   infoproviders.SITE_DN             Mds-Vo-name=local,o=grid
   infoproviders.SITE_GIN_UPDATE_PERIOD 600

   infoproviders.CEenabled   yes




R-GMA Deployment               Laurence Field   30/01/2003               14
    CMS - BOSS                                                      WP3

• Batch Object Submission System
    – A wrapper for any jobs a user wants to run
• A BOSS job is submitted containing the user’s
  job, whereupon BOSS carries out the following:
    – Forks the job off to a worker node
    – Parses (i.e. filters) ‘standard output’ and writes selected
      data from that output to a local database
• User application using R-GMA to stream data
• CircularBufferProducer used to publish parsed
  Standard Output
• Archiver used to collect and store information
  from multiple jobs

 R-GMA Deployment                Laurence Field   30/01/2003        15
   R-GMA Deployment
   and Performance                                                   WP3
   • Deployment of WP3 Demo Testbed (used in EU
     review)
        – Revealed R-GMA bug with multiple Tomcat instances –
          subsequently fixed
        – UK and INFN deployment following WP3 Demo Testbed
   • R-GMA is being used by:
        – WP1 (logging and book keeping)
        – WP2 (replica manager using R-GMA as the information
          provider)
        – WP7 (network monitoring)
        – CMS (Imperial College, Brunel) (BOSS)
   • Performance and Optimisation
        – Using NetLogger and profiling tools to identify possible
          bottlenecks


R-GMA Deployment               Laurence Field   30/01/2003           16
   Programme for 2003                                                  WP3

 • Functionality
     – Enhanced Mediator
     – Complete GRM integration for monitoring parallel applications
     – Nagios Integration
 • Resilience and Scalability
     – Consider all failure modes
     – Complete Registry and Schema replication
 • Security
     – Authorisation
 • Performance Optimisation
 • OGSA Compliant Implementation
     – GT3 inter-operability
     – Include wrapper for backwards compatibility



R-GMA Deployment              Laurence Field   30/01/2003              17
   Summary                                               WP3

   • R-GMA works!
   • Need to deploy to enable R-GMA to be tested
     in a real grid environment
        – Within UK and Italy initially
        – Problems found will be fixed quickly
   • Currently focusing on reliability, stability and
     performance




R-GMA Deployment           Laurence Field   30/01/2003   18

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:8/31/2012
language:English
pages:18