PowerPoint Presentation

							                             Storage Management

    • Owen Synge
        –    Developer, Packager, and first line support to System
             Administrators.
    • Talks Scope
        – GridPP for the year ahead.
    • Role
        – RAL employee, RAL data management Team.
        – Working within GridPP2.
        – Mass Storage and Local Data Management.
    • Project Members
        – Owen Synge, Jens Jensen, Tara Shah, Glen Johnson




Owen Synge                        Title of Talk                      Slide 1
                                 Summary

•   Background
•   History
•   Current State
•   Mass Storage
    –   Mass Storage Future
    –   Mass Storage Future : Upcoming Releases
    –   Mass Storage Future : SRM Release
    –   Mass Storage Future : SRM News
• Worker Node Disks
    – Local Storage Future : Time Scales
    – Local Storage Future : Priorities/Features
• Storage Future
• Conclusion
• Issues
Owen Synge                    Title of Talk        Slide 2
                               Background

• ADS Service
    – Peta-byte level Tape Storage Solution
• Grid Summary
    – Distributed Computing
    – Commodity Hardware
• European Data Grid Project (EDG)
    – Europe wide Grid Research Project
    – Provided much of LCG infrastructure
• SRM Collaboration
    – Collaboration between Fermilab, Jefferson Lab, Lawrence
      Berkeley, RAL, CERN
    – Contributed to the design of the SRM version 2 protocol

Owen Synge                    Title of Talk                     Slide 3
                                   History

• EDG SE History
    – V1
         • HTTP based Storage interface
    – V2
         • Java Web services Storage Interface
    – V3 (Not History as Still on V2.2)
         • SRM Standard Compliant Interface




Owen Synge                    Title of Talk      Slide 4
                                Current State

• EDG SE Status
    – Stable Version
         • Stable since November.
             – One Bug Fix (File Truncation)
             – One Administration Script created.
             – Atlas Data Transfer
                  » About 1500 files; 2 TB in total transferred to CERN
         • Going into LCG Production (Security bug on Client Side).
             – ADS Support
             – Not using Disk, Castor or HPSS interfaces
    – Current Development Version
         • Released this month.
         • Configuration Upgrade (Layered Template System).
         • Metadata Upgrade (See Later).
Owen Synge                       Title of Talk                            Slide 5
                        Mass Storage Future

• Addressing issues in EDG-SE
    –   Metadata/Configuration (Due in near future)
    –   Scalability (Number of Files)
    –   Performance (Time taken and resources used)
    –   SRM Compliance
• Addressing issues in GridPP
    – Disk Resources as MSS system
    – Checksums (Atlas request)
• Addressing issues for LCG (UK)
    – Clustering for Bandwidth
    – Testing frameworks
• Addressing issues (Generic)
    – Access Control
    – Job/User Namespaces
Owen Synge                    Title of Talk           Slide 6
             Mass Storage Future : Upcoming
                   Release Time Line
• Release 2.2
    – Renaming of LCG release
• Release 2.3 (May/June 2004)
    – Layered Configuration
    – Metadata upgrade
• Release 2.4 (Before August 2004)
    – LCG Release of 2.3 when Stable and bugs squashed
• Release 3.0 (Before August 2004)
    – SRM compliance
• Release 3.1
    – To be decided

Owen Synge               Title of Talk              Slide 7
                Mass Storage Future : SRM
                         Release
• Extra Features
    – Multiple Files acted upon with a single operation
    – Fully Asynchronous (srmStatusOfGetRequest)
• Benefits
    – Interoperability with major Storage solutions in Grid
      community.
    – GFAL and a large number of other Client tools
      available.




Owen Synge                Title of Talk                   Slide 8
                   Mass Storage Future : SRM
                             News
• SRM2
    – Finalised now working on 2.1 Revisions
• SRM and the GGF
    – SRM is going to be a GGF standard (Honolulu)
    – Specification of Basic/Advanced SRM
         • We will provide a service somewhere between these
           specifications for ADS on the first SRM release
• SRM and Other Storage solutions
    – SRB
         • On going work to support SRM API
    – DCache
         • Currently supports SRM v1 API

Owen Synge                    Title of Talk                    Slide 9
                         Worker Node Disks

• EDG/EGEE Local Data Management
    – Missing from EDG
         • Clean Down Worker Nodes
         • Reservation
    – Remote File Access
         • RFIO, SlashGRID, Replica Manager/EDG SE Client tools,
           GFAL.
• DCache
    – Aggregate Worker node space into storage
    – Mature system


Owen Synge                    Title of Talk                   Slide 10
             Local Storage Future : Time
                       Scales
• PM03 LD1: UK Site Coordinated Deployment and
  Support Plan for Local Storage Management (LSM)
  system.
• PM05 LD2: Release of LSM integrating with LCG3
  at UK sites. LCG 3 expected in PM05.
• PM07 SD3, LD3: Software prototype release
  integrating with EGEE DJRA1.3
• PM16 SD4, LD4: software prototype release
  integrating with EGEE DJRA1.6
• PM26 SD5, LD5: Release of MS and LSM
  integrating with EGEE follow-on.
Owen Synge           Title of Talk            Slide 11
                   Local Storage Future :
                    Priorities/Features
• Requirements
    – Disk Clean down?
    – Using Worker Node disk space?
    – Space management?
• Specification
    – Need to establish which requirements are highest
      priority.
• Implementation
    – Need to get the plan signed off.
    – Need to employ a new member of the team.

Owen Synge               Title of Talk               Slide 12
                       Storage Future

• SRM (De facto Grid Standard)
    – SRM1 moving to SRM2
    – GSM Basic moving to GSM Advanced
• Worker Nodes
    – Clean down after Jobs
    – DCache
• Must not forget Tier 2+3 have more storage space
  than Tier 0+1



Owen Synge              Title of Talk          Slide 13
                            Conclusion

• SRM
    – On going evolution of a standards management API
    – Not yet clear where access control is going exist in
      this area.
    – No SRM2/GSM Advanced implementations yet exist
• LSM
    – Local Storage management solutions not yet clear




Owen Synge                Title of Talk               Slide 14
                                   Issues

•   Tracking SRM scope and model
•   Local Storage Priorities
•   Recruiting and Training new member of the team
•   Testing environments for LCG/EGEE
•   Representation of files (Metadata Group?)
    – Trees/UID
    – Namespaces
         • Job/Service/User/VO/host based




Owen Synge                  Title of Talk        Slide 15