Docstoc

StoRM - CCR October 2005 - StoRM - Infn

Document Sample
StoRM - CCR October 2005 - StoRM - Infn Powered By Docstoc
					StoRM Service view


       Riccardo Zappi
        INFN-CNAF
          Bologna



                        Padova, 5 October
                        Outline

• Forewords
     – SNIA – SSM Model
     – Storage System example
     – Storage Element
• What SRM and StoRM are
• Aggregation within Application layer (difference
  with DPM and dCache)
• StoRM feasible tests
• Storage resource access scenario with StoRM


Page 2              Storage Task Force Meeting   Padova, 5 October
             The SNIA–SSM: Shared Storage Model

                              Application                       Mgmt Client
                                                                     SMIS
                        File/record layer                        Services
 Storage domain


                  Database                 File system         Capacity planning
                   (dbms)                       (fs)           Redundancy mgmt
                                                                 (backup, …)

                         Block aggregation                      High availability
                                                                 (fail-over, …)
                       Host

                                 Network                       Resource mgmt,
                                             Device             configuration

                                                                Security, billing
                     Storage devices (disks, …)                   Discovery,
                              Block layer                         monitoring


Page 3                            Storage Task Force Meeting             Padova, 5 October
              Clustering inter-box aggregation
                                  (example)

                     Application                           Purposes:
                                                           • load spreading across
File/record




                                                            peers (scalability)
                       Host        Host
                                                           • alternate paths (high
   layer




                                                GPFS        availability, scalability)
                              LAN
                                                Lustre
                     NASCluster FS
                                NAS
                     head       head                       Host block-aggregation


                                                           Network block-aggregation
                              SN
Block
layer




                                                           Device block-aggregation
                     Disk array




Page 4                        Storage Task Force Meeting                Padova, 5 October
         Aggregation within Application layer


                                                Files Aggregation
                   Application                       metadata


                                                                    DPM
     File/record




                    Host      Host
        layer




                                                   File
                                                   metadata
                            LAN                                     - No cluster-FS.

                     NAS          NAS                               dCache
                     head         head

                                                                    - Hierarchical MSS
     Block
     layer




Page 5                            Storage Task Force Meeting                 Padova, 5 October
                     Grid Storage Element
•   Storage Element (SE) represents a storage system
    abstraction in Grid.

                      I/O                                Service
          Data Transfer      Data Access      Management      Information

                                POSIX             SRM          Local Info
             GSIFTP                                             Provider
                                 RFIO             PBox
                                                                     ...
             HTTPS
                                 dCap            Admin

               ...                 ...             ...



          Storage         File/Record Layer
          System                                                   Services
                          Block Layer




Page 6                         Storage Task Force Meeting                     Padova, 5 October
    SRM : Storage Mgmt Middleware within the Grid
         Storage Resource Managers (SRMs)
•    SRMs are middleware components that manage shared
     storage resources on the grid and provide management
     functionalities like:
     – Uniform access to heterogeneous types of storage
     – File pinning
     – Disk space allocation and advanced disk space
        reservation
     – Protocol negotiation
     – Life time management of files
     – Management of security
•    SRMs implements a SRM interface specified by GSM-WG
     within GGF. (v.1.1, v.2.1.1 and v.3.0)


Page 7               Storage Task Force Meeting   Padova, 5 October
         Managent Access & Data Access to SE
                         Management Access
    SRM-Castor         SRM-dCACHE                                     StoRM
           Management Protocol                                     Management Protocol
                 SRM v.1.1                                           SRM v.2.1.1


    Storage System      Storage System                     Storage System    Storage System

                                 dC
     CASTOR             DCACHE                               GPFS              LUSTRE


          rfio               dCap                                      posix
    Access Protocol     Access Protocol                             Access Protocol


                                       Data Access
Page 8                                Storage Task Force Meeting                  Padova, 5 October
                     What StoRM is

• StoRM is a Storage Resource Manager.
• It is exposed as web service.
   – Web service description (wsdl) of StoRM is compliant with
     SRM specification version 2.1.1
• StoRM prefers Cluster File System with ACL
  support.
• Current StoRM version is built on top of GPFS
  (Plugin for Lustre FS is coming soon).
• StoRM allows direct access (native posix call) [i.e.
  StoRM works without server I/O too]. Aggregation
  is performed by underlying file system (better than
  StoRM could do).


Page 9                Storage Task Force Meeting     Padova, 5 October
          Feasible tests on Grid Storage System

                     Storage System
Tests on underlying Storage System with different
  Hardware/Software configurations.


                    Storage Services
• Data Access by Jobs running in WNs.
• Data Transfer by Transfer Service.
• Management calls by jobs, services, clients.
• Information provider… ( ? )




Page 10               Storage Task Force Meeting   Padova, 5 October
                      StoRM (SRM) test
• StoRM is a management service.
• StoRM is not a I/O service.
                                TESTs
•   Functionality tests
     – Well/Bad formed function call test
     – SRM interface compliance test
     – Security hacking
•   Stress tests
     – Bombing with well/bad formed function calls
     – Huge size requests
•   Integration test
     – Use of SRM by other Service (FTS, Agreement Service, …)
     – Use of SRM with other Service (LCMAPS, PBox, etc, … )
•   Failure test (network crash, database crash, power crash, ..)
•   Perverse test
Page 11                   Storage Task Force Meeting        Padova, 5 October
                StoRM test environment

•    Test bed #1 (V.Vagnoni) [build]
     1. Only StoRM ahead of Storage System based on GPFS.
     – Useful for [ Functionality, Stress, Failure ] tests
•    Test bed #2 (Certification INFN-Grid) [finishing]
     1. Storage Element (GridFTP server, LCMAPS, StoRM,
        Authorization service (PBox), ecc.. )
     2. Computing Element and some work nodes.
     – Useful for [ Integration / Stress / Failure ] tests
•    Test bed #3 (Certification INFN-Grid) [coming soon]
     1. Nr.2 (or more) Test bed #2
     2. External Service (FTS, LFC / FireMan, … )
     – Useful for [ Integration / Failure / Perverse ]

Page 12                  Storage Task Force Meeting      Padova, 5 October
Page 13   Storage Task Force Meeting   Padova, 5 October
StoRM use cases



        Backup slides




                        Padova, 5 October
                                SURL




                                   PBox                                   PBox
                StoRM Service      (SE Istance)                           (CE Istance)
                                                                                            CE



                                                     LCMAPS                              Gatekeeper
 StoRM Server




                 Permission        Privilege                                                LCAS
                 Component         Enforcer

                                                                   Shared
                                                                  Mapping
                                                                                          LCMAPS
                                                                  Policies &
                                                                 assignment
                       SURL

                                                                                 WN#k       Job
                                                                                          Manager
                                               ACL    PFN
                  Permission
                   Catalog                                                       User
                                                                                 Job
                                                                                           Computing
  Storage                                                                        GPFS      Element
  Element                               GPFS File System                                   and WNs
                                   (ACL Enforcement Mechanism)

Page 15                                   Storage Task Force Meeting                     Padova, 5 October
    References
•   DMTF                        •   SRM
•   SNIA
•   SMI-S
•   GPFS
•   LUSTRE




Page 16          Storage Task Force Meeting   Padova, 5 October

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:4/5/2013
language:Latin
pages:16