Docstoc

Isabel-Granada

Document Sample
Isabel-Granada Powered By Docstoc
					             NGI-ES
  Design of the National Grid
    Infrastructure in Spain


            II Reunión Iniciativa e-CA
            Granada, 16-17 - X - 2008
   Isabel Campos Plasencia [isabel@campos-it.es]
 Científico Titular at Instituto de Física de Cantabria



Thanks to: Carlos Fernández, Jesus Marco, Jorge Gomes,...
OUTLINE



   Spanish NGI and Grid infrastructure
   Design of NGI-ES National Grid Infrastructure
   Operations and interoperability layer
   A relevant example: the initiative Grid-CSIC
   Summary




                                                   2
Spanish e-Science Network
Organization
Coordinator: Vicente Hernández    (Universidad Politécnica Valencia)‫‏‬




                       Coordinator: I Martín Llorente (UCM)




 Coordinator: I Blanquer (UPV)


                                               Coordinator: Isabel Campos (CSIC)




                                                                               3
                                                                                 3
                                           Coordinator: Mateo Valero (BSC-CNS)
National Grid Iniative (NGI)
and Grid Infrastructure
      NGI: Entity recognized at the National Level which is
      established as the Single Contact Point, and which operates a
      general purpose e-science infrastructure supporting the
      different user communities existing in the country. The NGI
      should be able to mobilize resources and contribute to
      standards and international policy boards
      The NGI should therefore establish a collaboration framework
      between all the participating institutions to foster a
      coordinated development of Grid Infrastructure in Spain.
      The area of Grid Infrastructure is in charge of deploying and
      operating such infrastructure
         Propose a sustainable design of the Grid e-Infrastructure in
         Spain for the EGI era
         Ecosystem of different Grid projects, computing centers, grid
         infrastructures, etc…
         Run central services to keep up the infrastructure
Grid Infrastructure
Necessities and Goals

     Coordinate all the common necessary elements

          Internationally acknowledged Certification Authority
          Interface with the National NREN (RedIRIS)
          Deploy a system of global resource monitorization
          Repositories of middleware and applications
          Contact with the developpers of both

     Start up a coordinated infrastructure among all the
     participating institutions

     Follow up the services that are being offered
         Rate cost/performance
         Evaluate Researchers satisfaction in order to propose a
         sustainability plan for the e-infrastructure
Components of the Grid Infrastructure

     Core: Spanish institutions participating in research and
     development projects about Grid research infrastructures
     being the attractor for many other smaller-scale centers
        EGEE, EUFORIA, DORII, EELA, i2g, … with common basic
        middleware (mainly gLite based)
        Infrastructure GRID - CSIC
        Universities and computing centers with middleware based on
        Globus Toolkit 4
        RedIris (Spanish NReN) support EUGRIDPMA certificates
     Relation with the Spanish Supercomputing Network (RES)
        This network comprises several Spanish research centers that
        operate a common infrastructure of supercomputing.
        Analyze possibility of mixed owrkflows between both
        infrastructures
     Relation with the Portuguese NGI in the framework of
     Ibergrid
                                                                       6
                                                                       6
Grid Infrastructures
Development in 2008

        April - June      July – Aug.    Sept.        Oct – Nov.      December



Interoperability
Middleware Requirements

                  Test Infrastructure
                    IFCA - CESGA
                                          First
                                        testbed

                                                   Integration of
                                                      all sites
                                                                      Positioning and
                                                  Integration layer   tender EGI.org
                                                  with Portuguese
                                                        NGI
     NGI-ES: Design of the NGI
          Infrastructure

               Coordinator: Isabel Campos-IFCA
               Deputy: Carlos Fernández-CESGA


Expert´s Pannel: Jose Salt (IFIC), Jesus Marco (IFCA), Antonio
Fuentes (RedIRIS), Kai Neuffer (PIC), Nicanor Colino (CIEMAT),
 Rubén Santiago (UCM), Jorge Ejarque (BSC), Ana Silva (CICA)
 NGI-ES Architecture:
 Oriented to support communities

         VO Oriented
The Architecture of the NGI-ES is    VIRTUAL ORGANIZATIONS
oriented to the support of Virtual
Organizations

Key Issues                             CENTRAL SERVICES
 Advanced VO services

     User support
     Monitoring & Accounting         RESOURCE CENTERS

 Application porting and support

 Middleware driven by application
  requirements
                                                             9
VO  Services and Resource
Centres relationship
                                                               RC B

       User Interface                    WMS Broker          LCG CE

                                                               CE
          SE              VOMS
                                               RC A          SE SRM

                        CrossBroker

VO
Applications                                     Cluster Manager
Users                                 LCG CE
                                                 WN WN WN
                                      CE

                                 SE SRM

                                                                      10
The need for interoperation at the NGI level:
important issues from different perspectives
  E-Science application users
    Common ways for accessing any e-infrastructure resources
    Sometimes it is the same user that wants to access one of another
    type of resources depending on the application
  The access method should be transparent for the user
    Potential access to a significantly larger set of resources
  E-infrastructure owners (Resource Centers)‫‏‬
    Reduced management overheads if only a single Grid middleware
    system needs deployment
    Potential for greater resource utilisation
  Grid Middleware and Applications Developers
    An interoperable set of services
    Applications portable across different Grid middleware systems


                                                                        11
NGI-ES added value: Brokerage services


                                                                     The Spanish NGI includes the experience in Scheduling and
                                                                     Job Management of the team from the University
                                                                     Complutense in Madrid which developed GRIDWAY

                                                                     More information on http://www.gridway.org
                                                                     Contact: Ignacio Martin Llorente (UCM)‫‏‬



                                 MPI Job
                                Submission
                    Migrating     Plugin        Roaming

                                                                                                       The Spanish NGI supports MPI and interactive Jobs
                                                                     Cross
                    Desktop                      Access                         MPI-start invocation
                                                                     Broker
                                                 Server
                                                                 Matchmaking
  SERVICES                                   Web server
                                                                                                       using the broker technology developed at the
                                                                                                       University of Autonoma of Barcelona,
      Information   Replica
         Index      Manager
                                                          Internet

                                                                                                        the CROSSBROKER
         gLite                                                                 gLite
              CE                                                                  CE
                                                               MPI-start
                                                                                                       More information under http://www.i2g.eu
                                                            Open -MPI

                                                          WN
                                                                                                       Contact: Enol Fernández (UAB)‫‏‬
 WN




                                                                                                                                                           12
Infrastructure and resources in NGI-ES
We made a call for resources
                                  Typology of the resource centers

 A total of 18 resource centers
 have answered the call              The Infrastructure of GRID – CSIC
 Each center has èpecified a         deployed to serve spanish researchers
 minimum amount of resources,        belonging to the Spanish National
 but also a maximum amount,          Research Council (CSIC)
 allowable only when the local       Regional Computing Centers
 ocupancy is low.
                                         CESGA (Galicia), CICA
 Minimum amount is ~ 1300                (Andalucia), CESCA (Cataluña)
 cores, the maximum is over
 4500.                               Research Centers involved in EU
                                     projects
 Online storage around 350 TB
                                         EGEE, i2G, EELA, Cytegrid,
                                         WLCG,..
                                     University Departments
                                         Física, Química, Medioambiente,
                                         ingeniería, biología,…
Results from the Resource Centers Poll

                                                                         Dedicacion                    Cores         Cores
                                                                             generic    Dedicación         Est           má
                                                     Almacenamiento          a a la         máxima         abl           xim
Centro          Responsable            Cores             Online (GB)         NGI            (picos)        es            o
CESGA           Ignacio López                  400                3000            25%           100%        100           400
UNICAN          Antonio Cofiño                  16                 745            25%           100%             4         16
IFCA            Rafael Marco                   200                5000            25%           100%         50           200
CICA            Juan A. Ortega                  75                9800            25%           100%         30            75
IFIC            Jose Salt                      120                2000            25%           100%         30           120
PIC             Manuel Delfino                 200              250000           100%           100%        200           200
UOC             Josep Corba                     32                1000            25%           100%             8         24
QCyCAR          Alfonso Niño                    20                 100           100%           100%         20            20
CIEMAT-Madrid   Fernando Blanco                 40                2000            50%           100%         20            40
GRyCAP (UPV)    Vicente Hernández               60                1000           100%           100%         60            60
CIEMAT-CETA     Raul Ramos                      56               30000            75%           100%         42            56
I2BASK          Charo Sánchez                   32                1000            25%           100%             8         32
GRID - CSIC     Jesús Marco de Lucas       2500                  30000            25%           100%        625          1875
BIFI            Alfonso Tarancón                45                 300            50%           100%         23            45
EHU             Txema Mercero                   16                4000           100%           100%         16            16
UAB             Miquel Senar                    21                  60           100%           100%         21            21
UAB-Campus      Miquel Senar                   250                 250
CESCA           Joan Cambras                   132                2000            12%           100%         16           132
ESAC            Rubén Alvarez                   92                1200            25%           100%         23            23
eCA             Jose Ruedas                     16                                25%           100%             4         16




TOTALES                                    4923                 343455                                     1300          3371
Resource Centers Map




                 Resource Centers
Requirements and “agreements”
RC  NGI
Requirements of the NGI to                 Requirements of Resource
  the Resource Centers                       Centers to the NGI
  NGI-ES is built as a production            Resource Centers have already
  infrastructure                             well defined access policies
                                                Mainly depending on the funding agency
      In terms of QoS
                                                How will those allocation policies will
       • Follow the indications of site         interact with NGI-ES is still to be defined
         maintenance
                                             Resource centers dedicated to
       • Sites need to guarantee             general user support need a
         continuity at the level of
         human and material support          mechanism to assign resources in a
            – Minimum support hours:         deterministic way
              Monday – Friday from 9-17.        Ex. The application of Bio Informatics
                                                from user A will use the resources of the
     Resource Accessibility                     Biomed VO for the next 6 months with a
       • Signature of a Resource                maximum of 300 cores and 2 TB / day
         Allocation Policy to                The NGI has to prove the
         guarantee users the access
         to the infrastructure               advantages of accessing the
       • Sites will be included in the       resources in Grid mode
         monitoring system of the NGI..
       • …and will have to respect the
         directives


                                                                                              16
         Interoperation Layer

               Coordinator: Isabel Campos-IFCA
               Deputy: Carlos Fernández-CESGA


Expert´s Pannel: Jose Salt (IFIC), Jesus Marco (IFCA), Antonio
Fuentes (RedIRIS), Kai Neuffer (PIC), Nicanor Colino (CIEMAT),
 Rubén Santiago (UCM), Jorge Ejarque (BSC), Ana Silva (CICA)
Interoperations Layer
Making users life easier

                User      Brokerage   Resource
Applications   Gateway    Services    Centers


                                           Worker
                         XBroker      CE   Nodes


                         GT4
                                           Worker
                         WMS          CE
                                           Nodes


                         Glite
                         WMS          CE   Worker
                                           Nodes
Global Services needed for the
interoperation layer

  Authentication mechanism based on EUGRIDPMA
    PKIRISGRID
    Under consideration using ID Card based Certificates for
    researchers without Registration Authority
     • Certificates are still such an obstacle for new users induction…

  VOMS deployment
    VOMS are deployed by application area at the NGI support centers
    Tied with the existence of a scientific community at the center
     • Proximity to users makes user support easier




                                                                          19
Global Services needed for the
interoperation layer

   Global Information System
      TOP-BDII based on standards:
       • OpenLDAP + GlueSchema implemented at CSIC
      Integration work going on with the Portuguese NGI TOP-BDII
       • IBERGRID initiative will model the union of the two NGI s to support
         common research projects between Spain and Portugal
   Portal for Monitoring and Accounting
      Accounting and monitoring portal based on the tools developed for EGEE
      by CESGA
   Dedicated Brokerage Services
      Glite WMS services
      Gridway to be used with glite and/or GT4
      CrossBroker to be used with glite and/or GT4
   Helpdesk for site administrators based on RT
      e-mail based ticketing system
                                                                                20
User oriented services needed for
interoperation




        Brokerage Services Interoperation Layer:
               Global service of the NGI




                    User Gateway:
               Global Service of the NGI
A relevant Example: GRID - CSIC
                            GRID – CSIC Infrastructure



    The Spanish National Research Council-CSIC is
   deploying the first stable distributed computing
infrastructure in Spain to facilitate CSIC researchers
 the accomplishment of scientific projects requiring
  computing resources beyond the capabilities of a
            single user or research group




   The GRID - CSIC infrastructure intends to foster
  multidisciplinary and joint projects between CSIC
 centers . Researchers from CSIC will have seamless
access to a distributed infrastructure consisting in the
    first phase (2008 - 2010) of over 8000 cores
     and a total storage of more than 1 Petabyte


                                                           23
             GRID-CSIC Initiative
Origin:
   experience at CSIC on GRID projects (CROSSGRID, i2g, EGEE, EUFORIA, DORII)‫‏‬
   Opportunity: participation in e-Science network, NGI, EGI
   Strategic objective
     • CSIC is also present in the Supercomputing Infrastructures in Spain: BSC & CESGA
   Area of collaboration with other European institutions (like CNRS)‫‏‬
Objective: deploy an advanced production Grid infrastructure to support
research projects where the distributed computing needs exceed what
available for a single user or group
Oriented to promote multidisciplinary or multi-center projects where
researchers need to simulate, analyze, process, distribute or access
large data volumes.
Examples (e-Science):
   Experimental Particle Physics (CDF, CMS, ATLAS, ILC…)‫‏‬
   Phenomenology (SUSY models) & Lattice
   Space Missions (XMM, Planck…)‫‏‬
   Astronomical Observations
   Climate modeling
   Computational Chemistry
   Bio computing
                                                                                     24
Description

  The project is based in the use on Grid technology based on glite,
  allowing to interoperate the infrastructure currently run for the EGEE
  project and the one of the project Interactive European Grid (this one
  coordinated by CSIC)
  It will also be possible to share the infrastructure with the IberGrid
  initiative under development with Portugal, and with the Institute des
  Grilles of CNRS in France.
  The project implies the deployment of a total computing capacity of
  about 8000 cores and 1000 Terabytes (1 Petabyte) of online storage
  There infrastructure will be deployed in three phases along (2008, 2009,
  2010):

     The first year will include centers with experience in Grid infrastructures (IFCA, IFIC,
     e IAA)
     The second phase will extent the infrastructure to Madrid (CTI) and Cataluña (ICMAB)
     The last phase will consolidate the infrastructure at the level of the whole country
Structure

  The project has three areas
         Infrastructure: Instalation, integration and operation of the Grid
         Aplications and development: Support to application porting
         Coordination: Dissemination, management,…
  Initial team

             Centro               Personal y Experiencia en el Área
  IFCA      Instituto de Física   Personal en plantilla
            de Cantabria          Jesús Marco, Isabel Campos, Rafael Marco, Celso Martínez Rivero
                                  Línea de Investigación en Computación Distribuida (GRID)
                                  Personal contratado: Iban Cabrillo, Pablo Orviz, Álvaro López, Irma
                                  Díaz
  IFIC      Instituto de Física   Personal en plantilla
            Corpuscular           José Salt, Santiago González ,Javier Sánchez, Alvaro Fernández
                                  Línea de Investigación en GRID
                                  Personal contratado: Gabriel Amorós
  IAA       Instituto de          Personal en plantilla: José Ruedas, Wilfredo More
            Astrofísica de        Personal contratado:
Current status

  Equipment in place already for
     IFCA:
       • Computación:
            –   IBM blades, 182 (dual quad: 1456 cores)
            –   70 + 14 con Infiniband
            –   Conexiones a red 3 x 10G
       • Almacenamiento:
            –   Cabinas Discos SATA (~175 Terabytes)
            –   4 servidores GPFS
     IFIC
       • Computación:
            –   HP + DELL
       • Almacenamiento:
            –   SUN
     IAA
       • Computación:
            –   servidores IBM x3850 M2, con tecnología de 4ª generación X-Architecture, que permite escalar desde 4 hasta 16 procesadores
                (Intel Quad Core Xeon X7350), y hasta 1TB de memoria RAM en la configuración de 16 procesadores
       • Almacenamiento:
            –   DELL
  Instalation finished in September - October
  We are hiring people (1 titulado superior + 1 doctor) at each institute.
Interoperability and relation with NGI

     La infraestructura GRID-CSIC mantendrá la interoperabilidad
     con otras infraestructuras existentes de computación Grid, en
     particular con la de los proyectos europeos Interactive
     European Grid (i2g), EGEE-II, DORII, EUFORIA, y la de los
     proyectos nacionales Tier-2 de las colaboraciones ATLAS y CMS
     Así como con la nueva iniciativa Grid nacional dentro de la Red
     Española de e-Ciencia, en la que el CSIC tiene un papel
     relevante (coordinando la infraestructura Grid)
        25% minimum resources dedicated to the NGI (2500 cores
     Se espera además que esta iniciativa permita al CSIC participar
     de modo directo en la futura Infraestructura Grid Europea
     (EGI).
Summary
     NGI Grid Infrastructure is well active!
        Added value: broker middleware, support to MPI, etc.
        Next meeting: 23-24 October in Sevilla
        Deployment of NGI infrastructure going on
     Relevant Grid Initiatives in Spain
        Existing RC in EGEE, i2g, EUFORIA, DORII, EELA…
        New large initiatives: GRID-CSIC
     Interest in European Grid Infrastructure
     GRID-CSIC project fostering collaboration at European level
        Join us at the e-Science & Grids Workshop, June 2009, Santander




                                                                          29

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:13
posted:9/23/2011
language:Galician
pages:29