Lessons Learned

Document Sample
Lessons Learned Powered By Docstoc
					                                                Enabling Grids for E-sciencE




                          Lessons Learned

                          Steven Newhouse
                          EGEE-III Technical Director
                          EGI.eu Interim Director




www.eu-egee.org

EGEE-III INFO-RI-222667
                                                        EGEE and gLite are registered trademarks
                                                         European e-Infrastructure
                          Enabling Grids for E-sciencE



   •  European Data Grid
         –  Explore the concepts through testbeds and experimentation
   •  Enabling Grid for E-sciencE
         –  Moving from prototype to production
   •  European Grid Infrastructure
         –  Routine usage of a sustainable e-infrastructure




EGEE-III INFO-RI-222667                                       Lessons Learned - Steven Newhouse - OGF27   2
                                                         The EGEE-III Project
                          Enabling Grids for E-sciencE


17,000 users
139,000 LCPUs (cores)
25Pb disk
39Pb tape

12 million jobs/month
    +45% in a year
268 sites
    +5% in a year
48 countries
    +10% in a year
162 Virtual Organisations
    +29% in a year
Over 20 active communities in 112 VOs

EGEE-III INFO-RI-222667                                  Lessons Learned - Steven Newhouse - OGF27   3
                                   EGEE-III Management Structure
                          Enabling Grids for E-sciencE




EGEE-III INFO-RI-222667                                  Lessons Learned - Steven Newhouse - OGF27   4
                                                           EGI Project Ecosystem
                          Enabling Grids for E-sciencE




                                                         Supporting
                                                          Projects
                    SSC
                                                                                             SSC Support
                                                                                                 Centres

          SSC                                                                                      SSC




                                                          EGI                                SSC

   EC Projects


                                  Software               Software       Software
                                  Providers              Providers      Providers


EGEE-III INFO-RI-222667                                         Lessons Learned - Steven Newhouse - OGF27   5
                                                                                    The EGI Actors
                          Enabling Grids for E-sciencE




                                                                                                Research Teams
           Research Institutes                           EGI

                                                                     NGI2
                                   NGI1




                                                         EGI.eu
                                                                                NGIn




                                              …




                                                                                 National Grid Initiatives (NGIs)
           Resource Centres



EGEE-III INFO-RI-222667                            Lessons
Learned
‐
Steven
Newhouse
‐
OGF27
                       6

                                                   EGI Management Structure
                          Enabling Grids for E-sciencE



               EGI.eu
                                                                          NGIs

                                                     EGI Council

                   •  Votes proportional to membership fee
                                          EGI
         EAC                          Executive
                   •  Loosely based around GDP – ‘Terena Key’
                    Advises          Committee
                                                                                                 NGI
                   •  UK, Italy, Germany, France:70                                         Representative
                   • ....
   Advisory Groups •  Estonia, Cyprus:5
                          Advises      Director

    MCB         SPG
                                     COO            CTO     UCO         CAO                    NGI
   OMB         UFSC                                                                         Management
                                         Senior Management Team

    IPG        USAG
                                              Local & Remote Staff                             NGI Staff




EGEE-III INFO-RI-222667                                            Lessons Learned - Steven Newhouse - OGF27   7
                          Enabling Grids for E-sciencE




EGEE-III INFO-RI-222667                                  Lessons Learned - Steven Newhouse - OGF27   8
                                                         How much does it cost?
                          Enabling Grids for E-sciencE



                          Annual Running Costs EGEE-III (M€)
              & Staff Effort in EGEE-III
    Staff (EC % National)

    Hardware (National)

                                  25

                          14           8
                                                                                                       24
      27                                                   51




             Operations                    User Community
             Middleware                    Coordination

EGEE-III INFO-RI-222667                                         Lessons Learned - Steven Newhouse - OGF27   9
                                                                                               Supporting Science
                           Enabling Grids for E-sciencE


•            End-user activity Changes in Resource Utilisation
       Archeology                    Registered Applications
• •     13,000 end-users in 112 VOs
       Astronomy              60

•        •  +44% users in a year
       Astrophysics           50            Computational
                                              Chemistry
• •     23 Protection
       Civilcore VOs          40
                                             Life Sciences

•        •  A Chemistry
       Comp. core VO has >10% of
                              30

•           usage within its science
       Earth Sciences         20
                                           Multidisciplinary

            cluster                           Astronomy &
                                              Astrophysics
•      Finance                10
                                             Earth Science
•      Fusion                  0




                                                                                                                                                                    Astronomy, Astrophyics &
                                                      High Energy Physics



                                                                            Fusion



                                                                                     Earth Science



                                                                                                     Computational Chemistry



                                                                                                                               Life Sciences



                                                                                                                                               Computer Science &




                                                                                                                                                                                               Other
•      Geophysics                                    Fusion




                                                                                                                                                                     Astro Particle Physics
                                                                                                                                                  Mathematics
•      High Energy Physics                     Other Areas

•      Life Sciences                                         0    1   2     3      4     5     6      7

•      Multimedia                     March 2008 to February 2009 (%) March 2007 to February 2008 (%)
                                           Number of jobs x2 over the period
•      Material Sciences
                                                                                                 Proportion of HEP usage ~77%




EGEE-III INFO-RI-222667                                                                  Lessons Learned - Steven Newhouse - OGF27                                                                     10
                                                         The ESFRI Roadmap
                          Enabling Grids for E-sciencE



   •  First edition 2006 and updated in 2008 with 44 projects
   •  Preparatory phase funding for most with second round
      soon
   •  About 10 will fly by 2010
   •  European X-FEL first to go real – civil construction
      started in 2009 and International convention agreed in
      September

          ESFRI: European Strategy Forum on
              Research Infrastructures




EGEE-III INFO-RI-222667                                  Lessons Learned - Steven Newhouse - OGF27   11
                             ESFRI Projects for Env. Sciences
                          Enabling Grids for E-sciencE

                                                                        IAGOS-ERI
        EURO-ARGO



                    SIOS


                                        Status                              EUFAR-COPAL
                                        2009

 AURORA BOREALIS                                         LIFEWATCH
                                   EISCAT-3D




                                                                  EPOS
       EMSO
                                                                           ICOS


EGEE-III INFO-RI-222667                                   Lessons Learned - Steven Newhouse - OGF27   12
                                   User support – Cradle to Grave!
                          Enabling Grids for E-sciencE



   •  Dissemination
         –  What we do and how to get involved
   •  Training
         –  Trained trainers around Europe
         –  Material available on line
         –  Training infrastructure
   •  Application Porting
         –  Getting your application onto the infrastructure
   •  Application Database
         –  You may not need to port your application as its already ported!
         –  Expand the database with operational and user information
   •  Helpdesk
         –  To deal with operational issues from end-user use

EGEE-III INFO-RI-222667                                  Lessons Learned - Steven Newhouse - OGF27   13
                                                          Software Licensing
                          Enabling Grids for E-sciencE



   •  Generally we are an open source system
         –  Operating system and application software
   •  Common to see site based licensed software
         –  No issues about software such as LSF, ...
   •  Strong collaboration with Mathworks over MATLAB
         –  Very common application within many disciplines
         –  Support for cluster based computation...
         –  But challenges over ‘grid’ licensing
   •  MATLAB on EGEE
         –  Define problem locally, but execute remotely
         –  Use client license to release functionality on the cluster
                Still need to have a license



EGEE-III INFO-RI-222667                                  Lessons Learned - Steven Newhouse - OGF27   14
                                                                                 Summary
                          Enabling Grids for E-sciencE



   •  Operating a distributed computing infrastructure
         –  This requires a LOT of effort
   •  Moving to a federated model supported by strong
      national activities
         –  Small central coordinating body (EGI.eu)
   •  Move from individual users to communities
         –  Support strategic EU & national investments & collaborations




EGEE-III INFO-RI-222667                                  Lessons Learned - Steven Newhouse - OGF27   15

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:14
posted:11/28/2011
language:English
pages:15