An Introduction to The Grid by moti

VIEWS: 81 PAGES: 55

									   An Introduction to
        The Grid
             Mike Wilde
Mathematics and Computer Science Division
       Argonne National Laboratory




Oak Park River Forest High School - 2002.0522
                            Topics
        Grids in a nutshell
         – What are Grids
         – Why are we building Grids?
        What Grids are made of
        The Globus Project and Toolkit
        How Grids are helping (big) Science




www.globus.org                 2           www.griphyn.org
                   A Grid to Share
                 Computing Resources
                  Major Archive
                    Facilities




                            Network caches &
                             regional centers




                                            Local
                                            sites
                   ?

www.globus.org                    3                 www.griphyn.org
                      Grid Applications
 •Authenticate once
 •Submit a grid computation
 (code, resources, data,…)
 •Locate resources
 •Negotiate authorization,
 acceptable use, etc.
 •Select and acquire resources
 •Initiate data transfers,
 computation
 •Monitor progress
 •Steer computation
 •Store and distribute results
 •Account for usage
www.globus.org                   4        www.griphyn.org
                 Natural Science
            drives Computer Science




www.globus.org         5          www.griphyn.org
      Scientists write software to probe
          the nature of the universe




www.globus.org        6           www.griphyn.org
      Data Grids for High Energy Physics
                                     ~PBytes/sec
                                                                                                                1 TIPS is approximately 25,000
                                                        Online System          ~100 MBytes/sec
                                                                                                                SpecInt95 equivalents

                                                                                    Offline Processor Farm
          There is a “bunch crossing” every 25 nsecs.
                                                                                           ~20 TIPS
          There are 100 “triggers” per second
                                                                                                         ~100 MBytes/sec
          Each triggered event is ~1 MByte in size

                                                       ~622 Mbits/sec
                                                                          Tier 0               CERN Computer Centre
                                        or Air Freight (deprecated)

 Tier 1
          France Regional                   Germany Regional                  Italy Regional                     FermiLab ~4 TIPS
              Centre                            Centre                           Centre
                                                                                                                               ~622 Mbits/sec


                                                            Tier 2            Caltech               Tier2   Tier2 Centre
                                                                                           Tier2 Centre Centre        Tier2 Centre
                                                                              ~1 TIPS        ~1 TIPS ~1 TIPS ~1 TIPS ~1 TIPS
                                             ~622 Mbits/sec


                                Institute
                                        Institute Institute       Institute
                               ~0.25TIPS                                                       Physicists work on analysis “channels”.
                                                                                               Each institute will have ~10 physicists working on one or more
      Physics data cache
                                                 ~1 MBytes/sec                                 channels; data for these channels should be cached by the
                                                                                               institute server
                                                                 Tier 4
                    Physicist workstations



Image courtesy
www.globus.org Harvey Newman, Caltech
                            7                                                                                              www.griphyn.org
                           The Grid
        Emerging computational and networking
         infrastructure
         – Pervasive, uniform, and reliable access to remote
           data, computational, sensor, and human resources
        Enable new approaches to applications and
         problem solving
         – Remote resources the rule, not the exception
        Challenges
         – Many different computers and operating systems
         – Failures are common – something is always broken
         – Different organizations have different rules for
           security and computer usage


www.globus.org                  8                    www.griphyn.org
                     Motivation
       Sharing the computing power of multiple
       organizations to help virtual organizations
       solve big problems




www.globus.org             9               www.griphyn.org
             Elements of the Problem
        Resource sharing
         – Computers, storage, sensors, networks, …
         – Sharing always conditional: issues of trust,
           policy, negotiation, payment, …
        Coordinated problem solving
         – Beyond client-server: distributed data
           analysis, computation, collaboration, …
        Dynamic, multi-institutional virtual orgs
         – Community overlays on classic org structures
         – Large or small, static or dynamic
www.globus.org               10                www.griphyn.org
                 Size of the problem
        Terflops of compute power
         – Equal to n,000 1GHz Pentiums
        Petabytes of data per year per experiment
         – 1 PB = 25,000 40 GB Disks
        40 Gb/sec of network bandwidth
         – 400 100Mb/sec LAN cables (streched across
           the country and the Atlantic)




www.globus.org             11               www.griphyn.org
      Sockets – the basic building block


             send              recv

             recv              send
                      IP
        Program     network
                              Program
           A                     B




www.globus.org        12              www.griphyn.org
          Services are built on Sockets

                     Protocol:
             send      http       recv

             recv                 send
                        IP
                     network
         Client:                 Server:
          Web                     Web
        Browser                  Server




www.globus.org          13               www.griphyn.org
                                  Client-Server Model

                                              Protocol:
                                                http       recv
                                                           send
           send                                  IP
           recv
                                              network
              send
              recv
                 send

      Client: recvsend                                    Server:
        Client: recv
       Web
                                                           Web
                     send

           Client: recv
          Web
     Browser
                        send

             Client: recv
            Web
       Browser
                           send

                Client: recv
               Web
          Browser
                  Client: recv
                 Web
            Browser
                              send
                                 send
                                                          Server
                     Client: recv
                    Web
               Browser
                                    send

                        Client: recv
                      Web
                 Browser
                                       send

                           Client: recv
                         Web
                    Browser
                              Client:
                            Web
                       Browser
                                 Client:
                               Web
                          Browser
                                  Web
                             Browser
                                Browser




www.globus.org                                   14               www.griphyn.org
           Familiar Client-Server Apps
        Email
         – Protocols: POP, SMTP
        File Copying
         – Protocol: FTP
        Logging in to remote computers
         – Protocol: Telnet




www.globus.org                15          www.griphyn.org
                        Peer-to-Peer Model
                                                          send
                                                          recv




                               Protocol:
             send
             recv
                                                      limewire


         limewire
                               gnutella
                                   IP
                               network
                                                           send
                                                           recv

                                    send
                                    recv
                                                      limewire
                        send
                        recv    limewire

                                               send
                                               recv
                    limewire


                                           limewire



www.globus.org                     16                             www.griphyn.org
            Familiar Peer-to-Peer Apps

        File (music) Sharing
         – Protocols: Napster, Gnutella
        Chat (sort of)
         – Protocols: IRC, Instant Messenger
        Video Conferencing
         – Protocols: H323




www.globus.org                17               www.griphyn.org
The Globus Project
       and
The Globus Toolkit
                   The Globus Toolkit:
                 Four Main Components
        Grid Security Infrastructure
         – A trustable digital ID for every user and computer
        Information Services
         – Find are all the computers and file servers I can use
        Resource Management
         – Select computers and run programs on them
        Data Management
         – Fast and secure data transfer (parallel)
         – Making and tracking replicas (copies) of files
        …plus Common Software Infrastructure
         – Libraries for writing Grid software applications

www.globus.org                    19                    www.griphyn.org
            Running Programs on the Grid
                             MDS client API calls
                             to locate resources
           Client                                    MDS: Grid Index Info Server
                             MDS client API calls                                    Site boundary
                             to get resource info


 GRAM client API calls to
request resource allocation                     MDS:   Grid Resource Info Server
   and process creation.                                            Query current status
                          GRAM client API state                     of resource
        Globus Security     change callbacks
        Infrastructure                              Local Resource Manager
                                                                                 Allocate &
                                                        Request
                                                                              create processes
                         Create      Job Manager

        Gatekeeper                  Parse
                                                                       Process
                                                        Monitor &
                                                         control       Process
                                      RSL Library
                                                                       Process


www.globus.org                                 20                            www.griphyn.org
         The Grid Information Problem




    Large numbers of distributed “sensors” with
     different properties
    Need for different “views” of this information,
     depending on community membership, security
     constraints, intended purpose, sensor type
www.globus.org            21               www.griphyn.org
               Grid Information Service
                                   hn= hostname
                           CPU     RAM       DISK    OS
                           CPU     VM        NET

                                                                          OS
                                                                  software=
                                                                     OS
                  CPUs
         dev group=
            CPU
            CPU

                                   memory
                          dev group=                  disk
                                             dev group=                net
                                                              dev group=
      dev=cpu 0 dev=cpu 1      RAM              DISK             NET
       CPU       CPU           VM


                         dev=RAM   dev=VM     dev=/scratch1    dev=eth0
                          RAM      VM            DISK           NET

www.globus.org                          22                         www.griphyn.org
                            GridFTP
                 Ubiquitous, Secure, High Performance
                         Data Access Protocol

        Common transfer protocol
         – all systems can exchange files with each other
        VERY Fast
         – Send files faster than 1 Gigabit per second
        Secure
         – Makes important data hard to damage or intercept
        Applications can tailor it to their needs
         – Building in security or “on the fly” processing
        Interfaces to many storage systems
         – Disk Farms, Tape Robots


www.globus.org                    23                     www.griphyn.org
                 Striped GridFTP Server
                GridFTP
                client                 To Client or Another Striped GridFTP Server



   GridFTP Control Channel                GridFTP Data Channels


                 mpirun                 GridFTP Server Parallel Backend
      GridFTP                                               MPI (Comm_World)
      server                 Control    Control   Control                      Control
                 Control                                            …
      master
                 socket      Plug-in    Plug-in   Plug-in                      Plug-in
                                                             MPI (Sub-Comm)



                                            MPI-IO


                     Parallel File System (e.g. PVFS, PFS, etc.)


                                                                …


www.globus.org                             24                           www.griphyn.org
          Striped
          GridFTP
        Application:
           Video
          Server




www.globus.org         25   www.griphyn.org
                     Replica Catalog Structure
                                                 Replica Catalog


                        Logical Collection                          Logical Collection
                        C02 measurements 1998                       C02 measurements 1999

                       Filename: Jan 1998
                       Filename: Feb 1998
                       …


                                                                        Logical
    Location                   Location
   jupiter.isi.edu            sprite.llnl.gov                           File Parent
 Filename: Mar 1998           Filename: Jan 1998
 Filename: Jun 1998           …                          Logical File                 Logical File
 Filename: Oct 1998           Filename: Dec 1998         Jan 1998                     Feb 1998
 Protocol: GridFTP            Protocol: ftp
 UrlConstructor:              UrlConstructor:          Size: 1468762
 GridFTP://jupiter.isi.edu/   ftp://sprite.llnl.gov/
   nfs/v6/climate                pub/pcmdi

www.globus.org                                           26                                  www.griphyn.org
             Programming with Globus
        UNIX based – Windows coming soon
         – Used by rest of Globus Toolkit
         – User can use for portability & convenience
         – Windows, UNIX, and Macintosh computers can all join
           the Grid
         – Portable programming very important
        Event Driving Programming
         – A way of writing programs that handle many things at
           once
        Parallel Programs
         – Wiriting programs that can utilize many computers to
           solve a single problem
         – MPI – A popular Message Passing Interface developed
           at Argonne and other laboratories
www.globus.org                  27                  www.griphyn.org
                 Grids and Applications




www.globus.org            28              www.griphyn.org
     Hunting for Gravity Waves
                                                              MyProxy
                                                               server




                                                    xml                       Frame
                                                                   HTTP
                                                                 frontend                                                                              Desired
                                                                                                                                                       Result
                                                                                                                                 :
                                                 Cgi interface
                                                                                                                  Single channel time series
     Transformation
        Catalog                                               Planner                          Monitoring


        Replica                                            G-DAG (DAGMan)
       Selection                            Executor
                                            CondorG/                                    Logs                                Prototype exclusive
                                            DAGMan                                                                              In design
                                                                                                                            Globus component
       Replica
       Catalog                                                                                                                        In integration


                                                    GridFTP       GridFTP      GRAM
                           GridFTP   GRAM/LDAS                                            GridFTP     GRAM/LDAS


                 GridCVS                                                     Compute           LDAS at Caltech
                              LDAS at UWM            UWM          SC floor
                                                                             Resource




www.globus.org                                                          29                                                www.griphyn.org
  Grid Communities and Applications:
Network for Earthquake Eng. Simulation
     NEESgrid: national
      infrastructure to couple
      earthquake engineers
      with experimental
      facilities, databases,
      computers, & each other
     On-demand access to
      experiments, data
      streams, computing,
      archives, collaboration

NEESgrid: Argonne, Michigan, NCSA, UIUC, USC
www.globus.org                30               www.neesgrid.org
                                                 www.griphyn.org
                              The 13.6 TF TeraGrid:
                              Computing at 40 Gb/s
                                    Site Resources   Site Resources
                          26
                                4      HPSS             HPSS
                          24

                                     External              External
                          8          Networks              Networks
                                                                      5

                                    Caltech          Argonne

                                                                             External
               External
                                                                             Networks
               Networks
   Site Resources                   SDSC             NCSA/PACI                          Site Resources
                                    4.1 TF           8 TF
      HPSS                          225 TB           240 TB                             UniTree




TeraGrid/DTF:
www.globus.org NCSA, SDSC, Caltech, Argonne
                               31                                         www.teragrid.org
                                                                            www.griphyn.org
          iVDGL Map Circa 2002-2003




                                      Tier0/1 facility
                                      Tier2 facility
                                      Tier3 facility
                                      10+ Gbps link
                                      2.5 Gbps link
                                      622 Mbps link
                                      Other link



www.globus.org       32         www.griphyn.org
     Whats it like to Work on the Grid?
        A fascinating problem on the frontiers of
         computer science
        Work with people from around the world
         and many branches of science
        Local Labs and Universities at the forefront
         – Argonne, Fermilab
         – Illinois (UIC and UIUC), U of Chicago,
           Northwestern
         – Wisconsin also very active!


www.globus.org               33                www.griphyn.org
           Access Grid
        Collaborative work
         among large groups
        ~50 sites worldwide
        Use Grid services for
         discovery, security
                                                           Presenter
                                                              mic

        See also
         www.scglobal.org           Presenter
                                     camera


                                      Ambient mic
                                       (tabletop)

                                    Audience camera


Access Grid: Argonne, others
www.globus.org                 34   www.mcs.anl.gov/FL/accessgrid
                                                   www.griphyn.org
                 Come Visit and Explore
        Argonne and Fermilab are right in our own
         backyard!
         – Visits
         – Summer programs




www.globus.org             35              www.griphyn.org
                 Supplementary Material




www.globus.org             36         www.griphyn.org
                         Executor Example:
                          Condor DAGMan
        Directed Acyclic Graph Manager

        Specify the dependencies between Condor jobs using
         DAG data structure

        Manage dependencies automatically
         – (e.g., “Don’t run job “B” until job “A” has completed
           successfully.”)                         Job A
        Each job is a “node” in DAG

        Any number of parent or         Job B           Job C
             children nodes

        No loops
                                                 Job D

                          Slide courtesy Miron Livny, U. Wisconsin
www.globus.org                      37                      www.griphyn.org
                       Executor Example:
                    Condor DAGMan (Cont.)
        DAGMan acts as a “meta-scheduler”
         – holds & submits jobs to the Condor queue at the
           appropriate times based on DAG dependencies

                                              A
               Condor B                  B          C
               Job    C
               Queue              DAGMan D
        If a job fails, DAGMan continues until it can no longer
         make progress and then creates a “rescue” file with the
         current state of the DAG
         – When failed job is ready to be re-run, the rescue file is
           used to restore the prior state of the DAG

                           Slide courtesy Miron Livny, U. Wisconsin
www.globus.org                      38                       www.griphyn.org
                 Virtual Data in CMS




           Virtual Data Long Term Vision of CMS:
           CMS Note 2001/047, GRIPHYN 2001-16
www.globus.org             39              www.griphyn.org
                               CMS Data Analysis
          Dominant use of Virtual Data in the Future
                                 Event 1                                    Event 2                  Event 3


                        100b          200b               Tag 2               100b            200b
       Reconstructed                                     Tag 1
           data

       (produced by                                 Jet finder 2
          physics                     7K                                                     7K
                          5K                                                   5K
       analysis jobs)                               Jet finder 1


                                                    Reconstruction
                                                      Algorithm
                               100K                                                   100K
                                                    Calibration
                                                       data


                        100K          100K 50K             100K              100K 50K
         Raw data
        (simulated             300K               200K               300K                    200K
          or real)      50K           100K 100K             50K              100K 100K



                               Uploaded data       Virtual data              Algorithms



www.globus.org                                           40                                         www.griphyn.org
                   Production Pipeline GriphyN-CMS
          pythia
                                 Demo
                                cmsim             writeHits               writeDigis
CPU:      2 min              8 hours               5 min                   45 min

 1 run


 1 run


 1 run

            .          .          .       .           .
                                                                              .
            .          .          .       .           .
                                                                              .
            .          .          .       .           .
                                                                              .
 1 run

Data:               0.5 MB              175 MB                  275 MB                      105 MB
                   truth.ntpl           hits.fz                 hits.DB                     digis.DB


                                                 1 run = 500 events
  SC2001 Demo Version:
1 event




www.globus.org                                             41                          www.griphyn.org
                        GriPhyN: Virtual Data
                    Tracking Complex Dependencies
                        file1                                               psearch –t 10 …       file8

 simulate –t 10 …
                                                          file1
                        file2                                 file1                            Requested
                                reformat –f fz …               File3,4,5
                                                                                                  file
                                                                                  file7




                                conv –I esd –o aod                         summarize –t 10 …
                                                             file6




             Dependency graph is:
                – Files: 8 < (1,3,4,5,7), 7 < 6, (3,4,5,6) < 2
                – Programs: 8 < psearch, 7 < summarize,
                  (3,4,5) < reformat, 6 < conv, (1,2) < simulate

www.globus.org                                       42                                   www.griphyn.org
                     Re-creating Virtual Data

                        file1                                               psearch –t 10 …       file8

 simulate –t 10 …
                                                          file1
                        file2                                 file1                            Requested
                                reformat –f fz …               File3,4,5
                                                                                                  file
                                                                                  file7




                                conv –I esd –o aod                         summarize –t 10 …
                                                             file6




             To recreate file 8: Step 1
                – simulate > file1, file2




www.globus.org                                       43                                   www.griphyn.org
                      Re-creating Virtual Data
                         file1                                                psearch –t 10 …       file8

 simulate –t 10 …
                                                            file1
                         file2                                  file1                            Requested
                                  reformat –f fz …               File3,4,5
                                                                                                    file
                                                                                    file7




                                  conv –I esd –o aod                         summarize –t 10 …
                                                               file6




             To re-create file8: Step 2
                – files 3, 4, 5, 6 derived from file 2
                – reformat > file3, file4, file5
                – conv > file 6
www.globus.org                                         44                                   www.griphyn.org
                     Re-creating Virtual Data
                        file1                                               psearch –t 10 …       file8

 simulate –t 10 …
                                                          file1
                        file2                                 file1                            Requested
                                reformat –f fz …               File3,4,5
                                                                                                  file
                                                                                  file7




                                conv –I esd –o aod                         summarize –t 10 …
                                                             file6




             To re-create file 8: step 3
                – File 7 depends on file 6
                – Summarize > file 7


www.globus.org                                       45                                   www.griphyn.org
                      Re-creating Virtual Data
                         file1                                                psearch –t 10 …       file8

 simulate –t 10 …
                                                            file1
                         file2                                  file1                            Requested
                                  reformat –f fz …               File3,4,5
                                                                                                    file
                                                                                    file7




                                  conv –I esd –o aod                         summarize –t 10 …
                                                               file6




             To re-create file 8: final step
                – File 8 depends on files 1, 3, 4, 5, 7
                – psearch < file1, file3, file4, file5, file 7 > file 8


www.globus.org                                         46                                   www.griphyn.org
                             Virtual Data Catalog
                               Conceptual Data Structure

                                                       TRANSFORMATION

                                                       /bin/physapp1
                                                       version 1.2.3b(2)
                                                       created on 12 Oct 1998
                                    DERIVATION         owned by physbld.orca

                                    ^ paramlist
                                    ^ transformation



      FILE                     PARAMETER LIST

      LFN=filename1               PARAMETER                     FILE
      PFN1=/store1/1234987        i filename1
      PFN2=/store9/2437218                                      LFN=filename2
      PFN3=/store4/8373636        PARAMETER                     PFN1=/store1/1234987
      ^derivation                 p -g                          PFN2=/store9/2437218
                                                                ^derivation
                                  PARAMETER
                                  E PTYPE=muon

                                  PARAMETER
                                  O filename2




www.globus.org                               47                           www.griphyn.org
                                     begin v /usr/local/demo/scripts/cmkin_input.csh

             CMS Pipeline in VDL
                                       file i ntpl_file_path
                                       file i template_file
                                       file i num_events
                                       stdout cmkin_param_file
                                     end
                   pythia_input
                                     begin v /usr/local/demo/binaries/kine_make_ntpl_pyt_cms121.exe
                                       pre    cms_env_var
                                       stdin cmkin_param_file
                    pythia.exe         stdout cmkin_log
                                       file o ntpl_file
                                     end

                                     begin v /usr/local/demo/scripts/cmsim_input.csh
                                       file i ntpl_file
                                       file i fz_file_path

                   cmsim_input         file i hbook_file_path
                                       file i num_trigs
                                       stdout cmsim_param_file
                                     end

                    cmsim.exe        begin v /usr/local/demo/binaries/cms121.exe
                                       condor copy_to_spool=false
                                       condor getenv=true
                                       stdin cmsim_param_file
                                       stdout cmsim_log
                                       file o fz_file
                                       file o hbook_file
                                     end

                                     begin v /usr/local/demo/binaries/writeHits.sh

                    writeHits           condor getenv=true
                                        pre orca_hits
                                        file i fz_file
                                        file i detinput
                                        file i condor_writeHits_log
                                        file i oo_fd_boot
                                        file i datasetname
                                        stdout writeHits_log
                                        file o hits_db
                                     end

                                     begin v /usr/local/demo/binaries/writeDigis.sh
                                        pre orca_digis
                                        file i hits_db
                    writeDigis          file i oo_fd_boot
                                        file i carf_input_dataset_name
                                        file i carf_output_dataset_name
                                        file i carf_input_owner
                                        file i carf_output_owner
                                        file i condor_writeDigis_log
                                        stdout writeDigis_log

www.globus.org                  48   end
                                        file o digis_db
                                                                  www.griphyn.org
                      Virtual Data for Real Science:
                     A Prototype Virtual Data Catalog

  Architecture of the System:

                                    Virtual Data
                                     Language
      Virtual Data
                                        VDL                                      Job Execution Site
        Catalog




                                                                                                               Pool
                                                                                                               Condor
                                                                                 U of Chicago

                                    Interpreter           Job
                                                                                                      Globus
                                                                                                      GRAM
     (PostgreSQL)                                         Sumission
                                                               Condor-G
                                                                                 GridFTP
                                       (VDLI)             Sites Agent
                                                          ANL, SC,
                                                                                  Client

                                                          …


                                                               Globus            Job Execution Site




                                                                                                               Pool
                                                                                                               Condor
                                                               Client            U of Florida
                                                                                                      Globus
                                                                                                      GRAM
                                                                                 GridFTP
                                                                                  Client
                                       Local File              GridFTP
                                                               Server

                                        Storage                                  Job Execution Site




                                                                                                               Pool
                                                                                                               Condor
                                                                                 U of Wisconsin
                                                                                                      Globus
                                                                                                      GRAM
                                                                                 GridFTP
                                                                                  Client




                                                                    Grid testbed
  Production DAG of Simulated CMS Data:
                       Simulate
     Simulate                           Copy flat-file   Simulate Digitization
                     CMS Detector
      Physics                            to OODBMS       of Electronic Signals
                      Response




www.globus.org                                     49                                www.griphyn.org
      Early GriPhyN Challenge Problem:
          CMS Data Reconstruction
                                                2) Launch secondary job on WI pool;
                             Master Condor      input files via Globus GASS
                             job running at                               Secondary
                                Caltech                                Condor job on WI
                                                5) Secondary                 pool
                                                reports complete
      Caltech                                   to master
     workstation
          6) Master starts
          reconstruction jobs                       3) 100 Monte
          via Globus                                Carlo jobs on
          jobmanager on                             Wisconsin Condor
          cluster                                   pool
                                    9) Reconstruction
                                    job reports
                                    complete to master
                                                                         4) 100 data files
                                                                         transferred via
                                        7) GridFTP fetches               GridFTP, ~ 1 GB
                                        data from UniTree                each
                   NCSA Linux cluster
                                                                    NCSA UniTree
                                        8) Processed                - GridFTP-
                                        objectivity                 enabled FTP
                                        database stored             server
                                        to UniTree

Scott Koranda,
www.globus.org Miron Livny, others
                                50                                                 www.griphyn.org
                   GriPhyN-LIGO SC2001 Demo
                                                              MyProxy
                                                               server




                                                    xml                       Frame
                                                                   HTTP
                                                                 frontend                                                                              Desired
                                                                                                                                                       Result
                                                                                                                                 :
                                                 Cgi interface
                                                                                                                  Single channel time series
     Transformation
        Catalog                                               Planner                          Monitoring


        Replica                                            G-DAG (DAGMan)
       Selection                            Executor
                                            CondorG/                                    Logs                                Prototype exclusive
                                            DAGMan                                                                              In design
                                                                                                                            Globus component
       Replica
       Catalog                                                                                                                        In integration


                                                    GridFTP       GridFTP      GRAM
                           GridFTP   GRAM/LDAS                                            GridFTP     GRAM/LDAS


                 GridCVS                                                     Compute           LDAS at Caltech
                              LDAS at UWM            UWM          SC floor
                                                                             Resource




www.globus.org                                                          51                                                www.griphyn.org
             GriPhyN CMS SC2001 Demo
              http://pcbunn.cacr.caltech.edu/Tier2/Tier2_Overall_JJB.htm



                                                                                                      Full Event
                      Full Event
                                                                                                     Database of
                      Database of
                     ~40,000 large                                                                   ~100,000
                                                                                                     large objects
                        objects


                                                            Request
                                                                                                 
                     Request
              
                                                                                          
         

         Parallel tuned GSI FTP              “Tag”                      Parallel tuned GSI FTP
                                           database of
                                            ~140,000
                                          small objects


      Bandwidth Greedy Grid-enabled Object Collection Analysis
                       for Particle Physics


www.globus.org                                52                                      www.griphyn.org
                            iVDGL
      International Virtual-Data Grid Laboratory
       – A place to conduct Data Grid tests at scale
       – Concrete manifestation of world-wide grid activity
       – Continuing activity that will drive Grid awareness
      Scale of effort
       – For national, intl scale Data Grid tests, operations
       – Computation & data intensive computing
      Who
       – Initially US-UK-Italy-EU; Japan, Australia
       – & Russia, China, Pakistan, India, South America?
       – StarLight and other international networks vital
U.S. Co-PIs: Avery, Foster, Gardner, Newman, Szalay www.griphyn.org
www.globus.org                  53
          iVDGL Map Circa 2002-2003




                                      Tier0/1 facility
                                      Tier2 facility
                                      Tier3 facility
                                      10+ Gbps link
                                      2.5 Gbps link
                                      622 Mbps link
                                      Other link



www.globus.org       54         www.griphyn.org
                       Summary
        “Grids”: Resource sharing & problem solving
         in dynamic virtual organizations
         – Many projects now working to develop, deploy,
           apply relevant technologies
        Common protocols and services are critical
         – Globus Toolkit a source of protocol and API
           definitions, reference implementations
        Rapid progress on definition, implementation,
         and application of Data Grid architecture
         – Harmonizing U.S. and E.U. efforts important


www.globus.org               55               www.griphyn.org

								
To top