Objectivity - HPSS Integration

Document Sample
Objectivity - HPSS Integration Powered By Docstoc
					                       Marcin Nowak
                        RD45 Project
                   IT / Object Databases

                     Objectivity/DB - HPSS Interface




Objectivity Workshop 02.2000   Marcin Nowak   IT/DB/RD45   1
                               Interface Features
          Direct (POSIX) interface to HPSS
                abandoned due to low performance
          Staging interface
                currently used version
                requires staging pool management
                AMS-like performance for databases in the staging pool
                HPSS-like performance for databases on tape
                RPC timeouts need to be addressed
          Extended (extensible) AMS 5.2
                AMS protocol extended with GAP, OIP, RRP and DRP
                multi-threaded
                only one AMS version - no special linkable AMS
                uses shared libraries (liboofs.so, libooss.so, librfio.so)

Objectivity Workshop 02.2000     Marcin Nowak   IT/DB/RD45                    2
                           Direct HPSS Interface
        Standard Objectivity/DB configuration in
         distributed environment
                                                                      Objy Page Server
     Objectivity                                      Network
                                                       Layer          AMS
       Client             Network transfer
     Application           in DB pages              Filesystem                     DB
                                                   Access Layer                   files
                                                                       DB pages


        Objectivity/DB with HPSS FS level interface

     Objectivity                         Network            AMS
                         DB pages         Layer
       Client
     Application                       INTERFACE                              HPSS
                                                                DB pages
                                        HPSS client                           Server

Objectivity Workshop 02.2000        Marcin Nowak   IT/DB/RD45                             3
                    Staging Interface Overview
                   Migration          Migration (write) request
                   daemon


                                       DB files transfer
Stage Pool                                                      HPSS
 UNIX FS                                                        Server


                          DB pages
                           transfer

                AMS with HPSS           Staging (read) request
                  interface




Objectivity Workshop 02.2000      Marcin Nowak     IT/DB/RD45            4
                           Software Components
          OOAMS 5.2 executable
                liboofs.so
                liboss.so
                librfio.so (multithreaded, HPSS aware version)
                other shared libraries e.g. security
                ooss_Stage.pl script
                       moving files from HPSS to the local disk (stage pool)

          Migration/Purge daemon
                ooss_MigrPurg.pl
                     migrating idle database files from the local disk to HPSS
                     maintaining stage pool free disk space

          RFIO commands
                rfcp, rfmv

Objectivity Workshop 02.2000       Marcin Nowak   IT/DB/RD45                      5
                                    AMS Structure

       AMS 4.0.10              AMS 5.0         AMS 5.0             AMS 5.2
                               build 56        build 58

         OOFS v2               OOFS v3         OOFS v4            OOFS v5


                                          OOSS


                HPSS                     RFIO                    Other
                                       interface


                                RFIO                   UNIX FS


Objectivity Workshop 02.2000        Marcin Nowak   IT/DB/RD45                6
                      AMS/HPSS Interface at CERN


        CERN version of AMS/HPSS interface is using RFIO
              HPSS servers are running a special RFIO daemon used by
               SHIFT and Castor
              RFIO eliminates the need to link with HPSS libraries
                   no need for DCE on the host running the interface
                   porting to different platforms is easier
              no performance degradation compared to PFTP
                     full use of the HPSS parallel, point-to-point transfer protocol
              2GB file size limit
              current RFIO library does not support HPSS SetCOS call




Objectivity Workshop 02.2000       Marcin Nowak   IT/DB/RD45                            7
                         Staging Interface with RFIO
                        AMS
                        File & catalog
                                                 RFIO calls
                                                                         RFIO
                        management                                      daemon
                        Stage-in
                        requests
                                                                        HPSS
                        UNIX FS I/O                                     Server
                        DB pages


Disk
                                    Migration                         HPSS Mover
Server
                                    daemon
(Solaris)                                                                        Tape
                                                                                 Robot
                Disk
                Pool                            RFCP
                                                (RFIO copy)

     Objectivity Workshop 02.2000         Marcin Nowak   IT/DB/RD45                      8
                      Space Management in the
                            Staging Pool

          Space in the disk pool is controlled by the migration
           and purge daemon (Perl)
          Parameters of migration and purge are set via
           configuration file
          The process is inactive between space checks, but
           can be activated by AMS if free space is low
          File migration is done in parallel
          File locking is used to synchronize with AMS
                additional lock file for every database




Objectivity Workshop 02.2000    Marcin Nowak   IT/DB/RD45          9
                               Control Scripts

          startMaster          - start master script
                starts AMS and migration/purge daemon
          stopAll              - stops master script
                stops migration/purge daemon and tires to stop AMS
          rereadConf           -   reread configuration file
          suspendMigr          -   stop migrating files
          suspendPurg          -   stop deleting files
          restartMigr          -   start migrating files
          restartPurg          -   start deleting files


Objectivity Workshop 02.2000   Marcin Nowak   IT/DB/RD45              10
                                 Configuration
          Single configuration file for AMS and migration daemon
           specified by OOAMS_CONFIG environmental variable
          Dynamic configuration refresh on the SIGCONT signal
          Configurable parameters:
                local and remote root paths
                list of directories to scan for files to migate
                file purge thresholds (in percent of disk space)
                migration and purge frequency and delays
                selection of file transfer method (PFTP/RFCP)
                debug levels, log file location and other
          Timeout problems
                set OO_RPC_TIMEOUT environment variable


Objectivity Workshop 02.2000    Marcin Nowak   IT/DB/RD45           11
                               Database File Names
       Elements from the interface configuration file
        ($OOAMS_CONFIG)
             hpss_server
             local_root
             remote_root
       Elements from FDB catalog:
             host_name
             db_pathname
       Database name in:
FDB catalog:            host_name::db_pathname
„host_name‟:            db_pathname
HPSS:                   hpss_server:remote_root/ db_pathname
                        local_root is removed from db_pathname
Objectivity Workshop 02.2000     Marcin Nowak   IT/DB/RD45       12
                      Example of Path Handling

       Configuration values
   local_root:     /datapool
   hpss server: hpsssrv1
   remote root: /hpss/cern.ch/cms
       Command
   oonewdb -host dbsrv1
        -filepath /datapool/test/mydb.DB -db mydb
       Database names
   FDB catalog:            dbsrv1::/datapool/test/mydb.DB
   on dbsrv1:              /datapool/test/mydb.DB
   in HPSS:                hpsssrv1:/hpss/cern.ch/cms/test/mydb.DB
Objectivity Workshop 02.2000     Marcin Nowak   IT/DB/RD45           13
                    Protocol Enhancements in 5.2

          Generic Authentication Protocol (GAP)
                allows security information to be transferred from client
                 application to the AMS
                supports private and public key mechanisms
                information is used by most AMS operations
          Opaque Information Protocol (OIP)
                allows arbitrary information to be transferred from client
                 application to the AMS
                useful for passing hints to HPSS (like Class Of Service)
                used when a database is opened




Objectivity Workshop 02.2000    Marcin Nowak   IT/DB/RD45                     14
                    Protocol Enhancements in 5.2

          Defer Request Protocol (DRP)
                allows an AMS to control the timeout of a client‟s request
                designed to deal with filesystems with variable latencies
                handled internally by Objectivity
                should make the use of AMS/HPSS interface completely
                 transparent to the client application
          Request Redirection Protocol (RRP)
                allows a group of AMS‟ s to perform load balancing using
                 a redirection scheme
                assumes that such a group has a single “master” AMS that
                 is always contacted first and redirects the request



Objectivity Workshop 02.2000    Marcin Nowak   IT/DB/RD45                     15
                               Current Activities

          Testing Objy/HPSS interface with CMS
           production software
                ~Feb 14 starts large scale simulation on ~40 CPUs
                ~Mar 7 starts ORCA production. Initial step is transfer of
                 cmsim fz zebra format into ooHit format in Objy, ~2TB
          Linux version
          Synchronization with SLAC latest release
                remote run-time AMS monitor
                load-balancing disk pool manager (?)
          Implement request deferring
          Test other AMS protocol enhancements
                security

Objectivity Workshop 02.2000    Marcin Nowak   IT/DB/RD45                     16
                                   Distribution

          Complete package for Solaris 2.6
           /afs/cern/ch/rd45/amshpss
          Self-sufficient
                does not need local Objectivity installation
                uses /usr/spool/objy directory and /etc/services




Objectivity Workshop 02.2000    Marcin Nowak   IT/DB/RD45           17

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:1
posted:10/21/2011
language:Polish
pages:17