SRM-DRM by wuxiangyu


   Disk Resource Manager
    A Berkeley SRM for Disk Cache

ITB Service Integration Plan
   Jorge Rodriguez/Alex Sim
    OSG Integration Workshop at UC
           Feb 15-18, 2005
             Service Deployment Goals
          • To provide managed “tactical” storage to an
            OSG site
            – Apply SRM interface to $DATA… shared space
            – Storage space reservation
            – Storage request scheduling
          • Should function in the OSG environment
            – OSG Packaging
               • pacman
            – OSG MIS
               • Publish relevant service parameters (port #s, Service
            – OSG Authorization
               • Multi-VO, VOMS managed gridmap-file
2/15/05                              Jorge Rodriguez                     2
                 Service Description
          • DRM dependencies
            – Functions on top of ordinary UNIX file
              systems: local, NFS…
            – Binaries available for RH7.x and RHEL 3
            – Statically linked and self contained
              • Requires JAVA 1.4+
              • Requires data movement services: gTK, gsiftp

2/15/05                         Jorge Rodriguez                3
                 Service Description
          • Brief overview of the DRM Service
            – CORBA based service
              • Based on SRM v.1.1 specification
              • Movement of files into and out of SRM cache
              • Default Lifetime, space quota, caching and
                queuing policies are supported
              • Can be used through command-line DRM clients
            – Web Service component is provided by
              JAVA based wrapper
              • interoperable with other SRMs (e.g. dCache-
              • Can be used through SRMcp fermilab client
2/15/05                         Jorge Rodriguez                4
          Installation and Configuration
          • Installation:
            – DRM is included a part of VDT1.3.1+
               • DRM vers 1.2.3 in the VDT
               • Autoconfig CORBA component
            – Can be deployed with everything else…
               • How its currently deployed in DRM testbed
               • Canonical Grid3 site…
            – Can also be deployed stand alone
               • VDT will handle dependencies
               • Also being tried in DRM testbed

2/15/05                          Jorge Rodriguez             5
          Installation and Configuration
          • Configuration:
            – Should install as a non-privileged user (srm)
               • Service provider recommends this for security reasons
            – File system configuration
               • These recommendations are a work in progress!
               • Allocate a separate partition for the cache
               • RW for the service uid, all others RO
            – Can and should use service certificates
            – Auto configuration and sysV startup script
               • Hand entered parameters will be most likely be required
               • Web Service configuration is a work in progress

2/15/05                             Jorge Rodriguez                        6
Testing and Validation
                        Site Level Testing
          •       CORBA and Java Web Services are running
          •       Simple File movement exercise
              –     Move files into and out of the DRM cache via the
                    SRM interface
              –     Move files into and out of from a remote DRM
              –     Move files into and out of DRM cache from a
                    remote SRM-dCache client
          •       DRM test harness (from LBL team)
              –     Included with DRM vers 1.2.4+
              –     Will be part of the overall site verification

2/15/05                               Jorge Rodriguez                  8
                      Application Level Test
          •   Simplified application test should include:
              1. stage file into DRM cache from remote DRM/SRM-dCache
              2. Execute job that access “staged in” file in DRM cache via
                 POSIX calls
              3. Job creates an output file on local storage ($WNTMP or
              4. Job moves file from local storage into local DRM cache
              5. Job moves file from DRM cache to SRM managed or regular
                 remote location
          •   Scientific Application(s) (able to participate)
              –   Make and document recommendations for use of DRM
                  services in job work flow
              –   Make DRM enabled sites available for application testers
                  •   Schedule with application administrators
                  •   Monitor activity by both site and application administrators

2/15/05                                    Jorge Rodriguez                           9
                       Service Monitoring
          •       Will need input from Service Provider as to
                  what to monitor and what to measure
          •       Performance of storage requests
              –     How does the reservation system work
              –     How does storage request scheduling function
          •       Resource load from DRM activities
              –     Impact on server load
              –     Impact on other running services

2/15/05                              Jorge Rodriguez               10
          Support and Documentation

          –   Installation instructions
          –   Meetings and minutes
          –   Collection of interesting document
          –   email list
              • List is archived, linked from plone and “at work”
              • List is closed, but maybe possible to open to all
                osg lists?
          – Site information for those in DRM testbed

2/15/05                          Jorge Rodriguez                    11
Pending Issues
 Next Steps
                  Where Are We Now?
          • Communication channels created
            – plone activity, email lists and meetings
            – Coordinating with OSG-TG
          • Four sites in the readiness tesbed
            –   Argonne: Ed May
            –   Buffalo: Mark Green
            –   Florida: Jorge Rodriguez
            –   LBL: Alex Sim (DRM developers)

2/15/05                        Jorge Rodriguez           13
                 Where Are We Now?
          • Activities focused on deploying DRM software
            – Changed to DRM vers 1.2.3 ~ 1 week ago
               • New core features, including srm-remove
               • New configuration template added but not fully
               • Incorporated into VDT 1.3.1
            – New DRM version 1.2.4 released this Friday
               • No new core features but…
               • Improvements made to the auto config scripts
               • New testing tools added, will test installation and
                 interoperability with other SRMs, including SRM-dCache

2/15/05                             Jorge Rodriguez                       14
                 Where Are We Now?
          • Activities focused on using DRM software
            – CORBA services installed and used
               • Exercised file movement with gsiftp and SRM
               • Files moved into DRM cache worked OK
            – Web Services
               • None of the VDT based sites were successful in getting
                 these to run properly
               • Problems with installation and/or configuration
            – Pretty sure these difficulties can be ironed this

2/15/05                             Jorge Rodriguez                       15
                             Next Steps?
          • Continue to work out kinks in the installation and
            configuration procedures
          • Feed those back into the VDT packaging
          • Run through the testing and validation steps outlined
            in slides 8 through 11
          • Understand, document and communicate the service
             – To OSG site admins:
                • Impact on an individual site and services
                • Impact on OSG grid wide services
             – To OSG application community:
                • How will it impact on previously used work flows?
                • What other changes are needed effectively use the new

2/15/05                                  Jorge Rodriguez                  16
          What constitutes success for ITB release 0.1.0?
          • First need to complete Readiness Activity
          • Service is deployed and verified at OSG ITB
             – Software deployment will conform to standard
               OSG procedure: pacman –get + minimal post install
             – Service passes the test harness successfully
          • Service is described sufficiently for generic
            Grid3/OSG applications to use effectively

2/15/05                          Jorge Rodriguez                   17
                   Tentative Schedule
          • End of 2/15/05 week
            – Complete an installation of DRM vrs 1.2.4 on at
              least two tesbed sites
               • All DRM services installed and functioning
               • Services pass test harness
               • Simple application test complete
          • End of 2/21/05 week
            – Complete readiness activity
          • First week in March
            – Propose deployment plan
            – OSG Integration and Deployment

2/15/05                              Jorge Rodriguez            18

To top