IOP Conference, Lancaster, 2 April 2008

Document Sample
IOP Conference, Lancaster, 2 April 2008 Powered By Docstoc
					gLite adoption and opportunities
 for collaboration with industry




           Tony Doyle
   Distributed Computing Workshop
       Westminster, 21 May 2008
Introduction

    Context – PIPPS Projects

           Who are GridPP?

    Why do we need a Grid?

           What is our Grid?

          What do we offer?
                                PIPPS Projects

• David Sinclair and Chris Town (Cambridge Ontology Ltd) and Andy Parker
  (Cambridge e-Science Centre)
    – Mini-PIPSS to develop a Content Based Image Retrieval (CBIR) platform powered by gLite
    – On completion of the Mini-PIPSS project Cambridge Ontology received £535k private
       equity investment, changed its name to   Imense, and is now doing a PIPSS project with
       Andy Parker
• Oleg Soloviev (Econophysica) and Steve Lloyd (QMUL)
    – Mini-PIPSS to develop a Grid based automated trading platform for the financial industry
• Constellation Technologies Ltd and Neil Geddes (RAL)
    – PIPSS to develop a commercial version of gLite middleware
• DiGS and George Beckett (Edinburgh, EPCC)
    – PIPSS to develop a Data Grid for Cell Biology, sharing biological images between
      researchers (an example of inter-disciplinary use of software)
• Other EGEE-wide Projects
    – Total Oil testbed studies (Aberdeen)
    – EU-wide biomed docking studies (anti-malarial and bird-flu drug development)
                        Who are GridPP?
UK’s contribution to LHC computing:
- 19 UK Universities, STFC and CERN

GridPP1 (2001- 2004) £17m
“From Web to Grid”

GridPP2 (2004 - 2008) £16m
“From Prototype to Production”

GridPP3 (2008 – 2011) £25m
“From Production to Exploitation”




                                          4
              Why do particle physicists
                  need the Grid?

CERN LHC
The world’s most powerful particle accelerator




                                    4 Large Experiments
                      do particle GridPP?
                 Why Who are physicists need
                 the Grid?
Example from LHC:                     One year’s data   • ~100,000,000
starting from this event              from LHC would
                                                          electronic
                                      fill a stack of
                                      CDs 20km high       channels
                                                        • 800,000,000
                                                          proton-proton
                                       Concorde           interactions per
We are looking for this “signature”    (15 Km)            second
                                                        • 0.0002 Higgs
                                                          per second
                                                        • 10 PBytes of
                                                          data a year
Selectivity: 1 in 1013                     Mt. Blanc
                                                        • (10 Million
Like looking for 1 person in a             (4.8 Km)       GBytes = 14
thousand world populations                                Million CDs)
Or for a needle in 20 million
haystacks!
A question of scale
                  Solution Build a Grid
                 Solution – Builda Grid
                           •   Share more than information
                           •   Efficient use of resources at many institutes
                           •   Leverage over other sources of funding
                           •   Data, computing power, applications
                           •   Join local communities


Challenges:
    • share data between thousands of scientists with multiple interests
    • link major and minor computer centres
    • ensure all data accessible anywhere, anytime
    • grow rapidly, yet remain reliable for more than a decade
    • cope with different management policies of different centres
    • ensure data security
    • be up and running routinely in 2008
                             Middleware is the Key


                                                                  Your
                                                                  Program
Single PC                                Grid

 PROGRAMS               Your             MIDDLEWARE               User
                        Program
                                                                  Interface
    Word/Excel                                                    Machine
 Games                       Email/Web
                                           Resource                           Information
            OPERATING SYSTEM               Broker                             Service
            CPU
            Disks, CPU etc                                                    Replica
                                          Bookkeeping                         Catalogue
                                          Service
   Middleware is the Operating System
   of a distributed computing system
                                         Disk           CPU             CPU          CPU
                                         Server         Cluster         Cluster      Cluster
                         Something like this…


                   LFC                   VOMS      0    VOMS-proxy-init
                                                                                     gridui
    BDII                       3                                                     JDL


                                                   1
                                         WLMS
                                             RB
                         5




                                                                       Job Status?
                                                   11
4
                                                                                        2
                                             JS
 Grid Enabled Resources                                   7
  Grid Enabled Resources                 6
    Grid Enabled Resources                                            Logging &
     Grid Enabled Resources                                           Bookkeeping
                                       Submitter
                                   8                      9

CPU Nodes      Storage
 CPU Nodes       Storage
   CPU Nodes       Storage                               10
     CPU Nodes       Storage
                      Grid Infrastructure
                   Tier 0                 CERN computer centre
    Offline farm
                                                                              11 T1 centres

Online system      Tier 1             RAL,UK         Spain Germany    Italy    France
                   National centres


                   Tier 2                 ScotGrid     NorthGrid SouthGrid       London
                   Regional groups



                   Institutes              Glasgow      Edinburgh    Durham




                   Workstations



                       Structure chosen for particle physics.
                               Different for others.
                                   Middleware Validation:
                                         From Testbed to Production
                          Build                                    Development                                                      Certification                                                     Application
                          System                                   Testbed ~15CPU                                                   Testbed ~40CPU                                                    Testbed ~1000CPU

      Unit Test                     Build                             Integration                                                     Certification                                                      Production

                             Run nightly                                                                                                                                                               Certified public
   add unit                                                         Individual WP
                             build                                                                                                  Grid certification                                                 release
   tested code to                                                    tests
                             & auto. tests                                                                                                                                                             for use by apps.
   repository




                                                                                                                                                          Certified release selected for deployment
                                                  Tagged package
                                   Build system                           Integration                                                        Test Group                                                           Users




                                                                                        Tagged release selected for certification
                                                                          Team

                    WPs

                                                                    Overall
                                                                                                                                    Application
Fix problems                                                        release
                                                                                                                                    Certification
                                                                    tests
                  Process to test:
                                                                                                                                        Apps.
                  frameworks                                                                                                            Representatives

                  support
                  policies                                          Releases
                                                                      Tagged                                                        Releases
                                                                    candidate                                                        Certified
                  documentation                                        Releases                                                     candidate
                                                                                                                                     Releases
                  platforms/compilers
                                                                                                                                                                                                        24x7
                          Problem reports
                       Status
          March 2007                 March 2008




Status in 2007:
177 sites
                          Status in 2008:
32,412 CPUs               250 sites, 50 countries
~13 PB storage            55,094 CPUs
                          ~20PB storage
               GridPP & Industry
               What Do We Offer?
• Middleware Expertise

• Our Grid (for test purposes)
Examples:
• Adaptable User Interface (GANGA)

• Security tools (GridSite)

• Accounting tools (R-GMA & APEL)
                       Middleware Expertise

Grid Data Management        Network Monitoring   Workload Management




Information Services              Security         Storage Interfaces
                    Our Grid

• The UK Grid
  (via the individual
  research sites) has
  been used to test
  applications for
  other areas e.g.
• biomedical research
• financial modelling
• device modelling
• oil exploration
• image processing
                 Adaptable User Interface

                        Ganga GUI
                                            Scriptor
                          Job details




Logical
Folders


Job Monitoring         Job builder


                        Log window
                     Security Tools

                 Grid Security for the Web
                 Web platforms for Grids

• Digital Certificates
• Certification Authority
• Gridsite identifies users to websites with the digital
  certificates
• GridSiteWiki is an extension to the tool
• GridSite is open source (http://www.gridsite.org/)
                  Accounting tools


• Relational Grid Monitoring Architecture
  – An information and monitoring system for
    static and dynamic information about grid
    resources, applications and networks


• Accounting Processor for Event Logs
  – Provides a summary of the resources
    consumed based on attributes such as CPU
    time, Wall Clock Time, Memory and grid
    user identity
                         Knowledge Exchange

                             K
                Trust
                             n
                             o
             Security
                             w
     Business Models
                             l
                             e    Business
    Quality of Service
                             d    Community
                             g
                             e
          Accounting
                                  Research
                             E    Community
           Standards
                             x
         Applications
                             c
                             h
           Portability
                             a
                             n
                             g
Open Source Support
                             e
                                              Dissemination
   Software Licence
      Management
                     Knowledge Exchange

                          Productise
                          software for your
                          business




                                        Sustain
                                  software on
                             behalf of all users

“an essential
component within the                               Dissemination
innovation cycle of any
knowledge driven
economy”
              Summary
1. Opportunity for knowledge
   creation through improved IT skills
   and an enhanced research base

2. GridPP supports locally-led
   activities (based upon an
   international core of expertise and
   ongoing examples of collaboration)

3. GridPP will work with companies to
   examine different methods of
   technology transfer and identify
   the activities that can be used for
   industry and business

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:9/29/2012
language:English
pages:22