HPC in the UK An Update

Document Sample
HPC in the UK An Update Powered By Docstoc
					HPC in the UK:
   An Update
  Alan Gray, EPCC, The University of
                        Edinburgh
HPC User Forum, EPFL, October 2009



                        a.gray@ed.ac.uk
Contents

• EPCC
• UK National Services: HECToR and HPCx
• Case Studies
  –   Environmental Modelling
  –   Computational Materials Chemistry
  –   Fractal Generated Turbulent Flows
  –   Interactive Biomolecular Modelling
  –   FireGrid: Next Generation Emergency Response Systems
Contents

• EPCC
• UK National Services: HECToR and HPCx
• Case Studies
  –   Environmental Modelling
  –   Computational Materials Chemistry
  –   Fractal Generated Turbulent Flows
  –   Interactive Biomolecular Modelling
  –   FireGrid: Next Generation Emergency Response Systems
EPCC

• The University of Edinburgh founded EPCC in 1990 to act as
  the focus for its interests in simulation
• Today, EPCC is a leading centre for computational science
  in Europe
   – ~70 permanent staff                                     HPC
                                               Facilities    Research
   – Working in academia and industry
• Managing both UK national HPC          Technology               Training
                                         Transfer
  facilities
   – HECToR: Cray XT5h                          European          Visitor
   – HPCx: IBM Power5 eServer                 Coordination   Programme




                                                                             4
Contents

• EPCC
• UK National Services: HECToR and HPCx
• Case Studies
  –   Environmental Modelling
  –   Computational Materials Chemistry
  –   Fractal Generated Turbulent Flows
  –   Interactive Biomolecular Modelling
  –   FireGrid: Next Generation Emergency Response Systems
HECToR
• HECToR: High End Computing Terascale Resource
  – 6 year service, funded by UK government. Commenced 2007
  – Used for wide variety of apps across academia and industry
  – Located at University of Edinburgh, managed & operated by EPCC
    (with help from Daresbury Laboratory staff)


                             Cray XT5h




 • XT4: 5664 quad-core Opterons          • X2: 112 Cray Vector Proce
    – peak performance 208 TFlops

                                                                     6
HECToR Upgrade path

• Currently in Phase 2a
• Q1 2010: Cray „Baker‟ (Phase 2b, stage 1)
   – 20 cabinets; 3612 AMD „Magny Cours‟ 12-core chips, 44,544 cores
     total
   – estimated peak performance of 338 TFlops.
   – ~30 cabinets of XT4 will be retained.
• Q4 2010: upgrade Baker to Gemini network (Phase 2b,
  stage 2)
• Q3 2011: Phase 3. ?????
   – No Hardware provision contract at this time




                                                                       7
HPCx


• UK policy is to have
    overlapping HPC services
•   HPCx: previously main
    service, now in operation as
    secondary service
• Located at Daresbury, managed and operated by EPCC
    and Daresbury
•   160 IBM e-Server p575 nodes
    – 16 Power5 1.5 GHz cores per node: 2560 cores total
    – IBM HPS interconnect (aka Federation)
    – 12.9 TFLOP/s Linpack




                                                           8
Complementarity

• Managing both HECToR and HPCx simultaneously provided
  unique opportunity to maximise benefits for UK research.
  Run as “Complementary services”:
   – HECToR is our leading HPC facility
   – HPCx is our "National Supercomputer", trading overall utilisation in
     favour of a more flexible service
       – Very long jobs
       – Interactive use (computational steering, visualisation, debugging,
         etc)
       – Advanced reservations
       –…
• Gain experience to advantage of future HPC services.


                                                                              9
Contents

• EPCC
• UK National Services: HECToR and HPCx
• Case Studies
  –   Environmental Modelling
  –   Computational Materials Chemistry
  –   Fractal Generated Turbulent Flows
  –   Interactive Biomolecular Modelling
  –   FireGrid: Next Generation Emergency Response Systems
Environmental modelling

Lois Steenman-Clark, University of Reading
• HIGEM: seven UK academic groups plus UK Met Office
   – Aim: achieve a major advance in developing an Earth System model
     of unprecedented resolution
      – capable of performing multi-century simulations.
• Increasing horizontal resolution of Earth System models
  allows capture of climate processes and weather systems in
  much greater detail.
   – but scientifically challenging
      – new model created, tested, analysed, assessed, tuned and
         optimised
   – The control experiment, 115 model years of HIGEM, run on HPCx



                                                                        11
Environmental modelling

                      • sea surface temperature
                          anomalies associated with El-
                          Nino events from
                          – a) an observational climatology
                          – b) the HIGEM control run
                          – c) standard climate resolution UM
                            experiments
                      • HIGEM model now regularly used
                          in current research projects
                          – will be used for some very high
                            resolution experiments as part of the
                            input to the next IPCC (International
                            Panel on Climate Change) report
                            due in 2013.
                                                                12
Computational Materials Chemistry

Richard Catlow and Scott Woodley, University College London
• Materials Chemistry Consortium, comprises over 25
  research groups
• extensive applications portfolio,
   –   energy and environmental materials,
   –   catalysis and surface science,
   –   quantum devices,
   –   nano-science
   –   biomaterials
• Highest users of HPCx by project over lifetime of service.
   – Currently heavily utilising HECToR



                                                               13
 Computational Materials Chemistry
Energy and Environmental Materials
• Modelling radiation damage in materials using DL_POLY MD code
    – assessment and design of materials for use in nuclear reactors.
• Implemented effects of electronic stopping and electron–ion interactions
   within radiation damage simulations of metals,
• investigated the evolution of the damage on annealing for SiO2, GeO2,
   TiO2, Al2O3, and MgO,




                                    Simulation of damage created by 50 keV recoil
                                       atom in quartz.




                                                                                    14
Computational Materials Chemistry
Biomaterials
• explored fundamental factors relating to the structure of
  bone, in particular the interface between apatite and
  collagen.




 MD simulation of the nucleation of hydroxyapatite in an aqueous environment at a
 collagen template, showing the clustering of the calcium and phosphate ions
 around the collagen functional groups.


                                                                                    15
Computational Materials Chemistry

Nano-Chemistry and Nucleation
• rapidly expanding area: exploiting computational tools to develop models
   for the structures, properties and reactivities of nano-particulate matter.
• explored possible structures and properties of such nanoparticles, as well
   as how particularly stable particles can be employed as building blocks




                                                       Stable octahedral clusters are
                                                       connected to create
                                                       microporous crystals




                                                                                   16
Fractal-Generated Turbulent Flows

Professor Christos Vassilicos, Imperial College London
• New industrial fluid flow solutions urgently needed to meet
   unprecedented requirements
    – Increase energy savings, reduce environmental impacts
    – Industries which want to create or minimise turbulence
        – Aerospace and automotive industries
            – Reduce noise, fuel consumption, pollutant emissions.
        – Chemical and process industries
            – use turbulence for mixing
• New flow concept originating from UK: turbulent flows generated by
   fractal grids
    – create intense turbulence with very little effort or power input
        – only need very small changes to the grid to have enormous effect.
    – Size of simulations is so large that they are impossible without HPC




                                                                              17
Fractal-Generated Turbulent Flows




• Fractal Square
  Grid
   – Fluids pass
     through grid,
     turbulence
     created


                                    18
Fractal-Generated Turbulent Flows




    Streamwise velocity in one of the planes normal to a turbulence-generating fractal
    square grid. (From Laizet & Vassilicos 2009.)


 • The first ever successful simulations of turbulence generated
   by fractal grids performed on HECToR in 2008 and 2009.
    – Used Incompact3d code



                                                                                         19
Interactive Biomolecular Modeling

Carmen Domene, University of Oxford
• Aim: further understanding of “ion channel“ proteins within
  nervous system
   – These regulate ion flow through the cellular membrane, exerting
     control on electrical signals in cells
   – dysfunction can cause diseases in muscles, kidney, heart or bones.
   – Improved understanding may lead to better drugs and treatment
• Use Computational Steering to manipulate simulation by
  hand, to create specific starting configurations of interest.
   – IMD: VMD (run on user PC) connects to NAMD (on back end of
     HPCx). Can manipulate molecules by hand.
   – Allows simulation of rare, but possible, mechanisms



                                                                          20
Interactive Biomolecular Modeling




 • Work has demonstrated that alternative pathways for ion conduction to
    the one already proposed in the literature are possible
 • Considering these results, it would be also interesting to revisit many of
    the models proposed in the literature which did not successfully agree
    with experimental data

                                                                                21
FireGrid: Next Generation Emergency Response systems


• Retrospective analysis of every emergency poses question:
     Was the response adequate?
•   Almost invariably the answer is:
    Better information would have led to more effective response.


• Brutally illustrated when
    emergency crews continued
    operations oblivious to
    impending collapse of
    WTC1 and WTC2.


                                                                22
FireGrid: Next Generation Emergency Response systems



                                          Large
                                        databases



                                                                  Emergency Response




   Dense Sensor network for
  early detection and monitoring

           Alerts                    Super Real Time Simulation
                                         of fire growth and
                                        structure response




                                                                       Incident Commander


      Building Command and Control
                                                                                            23
FireGrid: Next Generation Emergency Response systems

• Partners:
    – The University of Edinburgh: R&D for all areas of the project
         – EPCC, the Institute for Infrastructure and Environment, the Institute for Digital
           Communication, the National e-Science Centre, and the Artificial Intelligence
           Applications Institute
    – BRE (Building Research Establishment) project leader and also provided the state-of-
      the-art experimental facilities that housed the fire
    – ABAQUS UK Limited and ANSYS-CFX (structural mechanics and CFD software)
    – Xtralis expertise on active fire protection systems, as well as sensor equipment in
      support of experiments;
    – the London Fire Brigade: principal user and guided the development of the command
      and control interface.
• Initial project has completed
    – prototype Integrated Emergency Response System
    – Successful live demos (with real fires) utilised HPCx and local Edinburgh University
      cluster HPC resources.



                                                                                               24
References
•   EPCC: www.epcc.ed.ac.uk
•   HECToR: www.hector.ac.uk
•   HPCx: www.hpcx.ac.uk
•   Environmental Modelling
    “Moving the capability boundaries”, Lois Steenman-Clark, HPCx Capability
      Computing Issue 13, http://www.hpcx.ac.uk/about/newsletter/
• Computational Materials Chemistry
    “Computational materials Chemistry on HPCx”, Richard Catlow and Scott
      Woodley, HPCx Capability Computing Issue 13,
      http://www.hpcx.ac.uk/about/newsletter/
• Fractal Generated Turbulent Flows
    To appear under “Casestudies” section at www.hector.ac.uk
• Interactive Biomolecular Modelling
    “Interactive Biomolecular Modeling with VMD and NAMD at HPCx”, Carmen
       Domene, HPCx Capability Computing Issue 13,
       http://www.hpcx.ac.uk/about/newsletter/
• FireGrid: Next Generation Emergency Response Systems
    http://www.epcc.ed.ac.uk/research-collaborations/casestudies/firegrid ;
    http://www.firegrid.org/