Introduction to HPC by gregorio11

VIEWS: 28 PAGES: 16

									               The 17th Annual International Symposium on High
               Performance Computing Systems and Applications
               17ième symposium international sur les applications et
               systèmes de calcul de haute performance




           Introduction to HPC
                    Presented by

                   Carol Gauthier

           Centre de Calcul Scientifique (CCS)
               Université de Sherbrooke

Réseau québécois de calcul de haute performance (RQCHP)
Introduction to HPC
               Summary of the presentation




              1) Who needs HPC?

              2) HPC Architectures

              3) What goes on inside a computer?

              4) Overall Performance Strategy




                   Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
                        Who needs HPC?
What defines High Performance Computing?

     “We use HPC to solve problems that could not be solved in a
     reasonnable amount of time using a single desktop computer.”


     Problems solved using HPC :

                      Are Long to compute
                         and/or
                      Needs large quantity of RAM
                         and/or
                      Requires large quantity of runs
                         and/or
                      Are Time Critical



                       Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
                          Who needs HPC?
HPC Domains of Application
     “More and more scientific domains of application are now
     requiring HPC facilities. It is not any more the strong hold of
     Physics, Chemistry and Engineering...”
    HPC domains of applications
                      Fluid dynamics and heat transfer
                      Physics and Astrophysics
                      Nanoscience
                      Chemistry and Biochemistry
                      Biophysics and Bioinformatics
                      Geophysics and Earth imaging
                      Medical Physics and Medicine
                      Databases and Data Mining
                      Financial Modelling
                      Signal and Image Processing
                         And more ....
                         Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
                   HPC Architectures




                Vectorial
                Shared Memory
                Distributed Memory : Clusters
                Hybrid




                  Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
                   HPC Architectures
Vectorial Architecture
                                                           Memory


                                                CPU       CPU      CPU       CPU




    Xi
                                          OP
    Yi

                                          =
    Zi



                         Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
                   HPC Architectures
Shared Memory Architecture


   Single memory image

                                                                                  System
                Your program
                 in memory
                                                                         Another program


                                     Memory Switch Fabric
    CPU   CPU   CPU   CPU   CPU    CPU    CPU   CPU   CPU    CPU   CPU    CPU   CPU   CPU   CPU    CPU




                                  Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
                   HPC Architectures
Distributed Memory Architecture : Clusters


   Memory Node 1       Memory Node 2            Memory Node 3               Memory Node 4
    Your program       Your program              Your program                Your program
     in memory          in memory                 in memory                   in memory
        task 0             task 1                    task 2                      task 3
           System             System                      System                     System


     CPU     Network    CPU      Network           CPU       Network          CPU       Network
              Card                Card                        Card                       Card




                                Network Switches



                          Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
                   HPC Architectures
Hybrid Architectures


                  Memory Node 1                                   Memory Node 2

                                          Network                                          Network
            CPU    CPU    CPU    CPU                        CPU     CPU    CPU     CPU
                                           Card                                             Card




          Memory Node 3                                   Memory Node 4

                                Network                                          Network
    CPU    CPU    CPU    CPU                        CPU    CPU    CPU     CPU
                                 Card                                             Card




                                          Network Switches

                                Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
            What goes on inside a computer?
CPU Insights

           Instruction cache            Register (L1 cache)



              Pipeline                       Load/Store Load/Store
                                                Unit       Unit                          L2-L3
                                                                                         Cache

    FP         FX        FMA       FMA         Vector         Specialised
    Unit       Unit      Unit      Unit         Unit             Unit



                                           RAM

                            Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
            What goes on inside a computer?
The Process of Building and Delivering Tables!


                Inside the Factory

                                                   Factory           Truck
                                         Lift     Warehouse

                      Human
         Production line                                                          Sherbrooke
         in Sherbrooke                                                               Train
                                                                                    Station

                                                                                            Train

                                 Raw materials and
      Tables                        Clients in              Plates        Legs
                                     Montreal                                         Painting


                           Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
            What goes on inside a computer?
Memory Access vs Clock Cycles
                                                     TLB
  CPU                                 Translation Lookaside Buffer
  L1 Cache (register)                 List of most recently accessed
                                               memory pages
  1-4 cycles

     L2 – L3 Cache                                                               PFT
                              TLB miss: 30-60 cycles                    Page Frame Table
  8-20 cycles                                                            List of memory
                                                                          pages location
                        RAM
                                                    Memory Pages
  80000 - 120000 cycles


                                              Disk



                               Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
               Overall Performance Strategy
Scientific Application Developpment Steps

Tsa
            1-12 months                       Scientific Analysis
                                 Theory, Maths, Algorithms, Data Structures, etc.
            1-6 months
Tspec                                       Writing Specifications
            1-12 months
Tcode                                                 Coding
            1-6 weeks
Topt                                          Serial Optimization
            1-6 months
Tpar                                   Parallelization (if necessary)
         1 msec-60 min / run
Tprep                                            Prepare cases
        1 min – x months / run                                                         Cases     Modification
Trun                                                Run cases                           loop        loop

             1-12 months
Tmod                                           Code Modifications


                                     Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
               Overall Performance Strategy
Amdahl's Law
   Running time of the code (Trun)


    Portion of the code                      Portion of the code
      that cannot be                             that can be
    parallelize/optimize                     parallelize/optimize

   Even after an excellent parallelisation or optimisation...

   Running time of the code (Trun)




                .... this code will never run more than 3 times faster !!!


                           Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC
               Overall Performance Strategy
Overview of Development and Production Time


     Development
     Time


            Tsa     Tspec                  Tcoding              Topt         Tpar



     Production
     Time


            Tprep           Trun                              Tmodif



                     Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke
Introduction to HPC




                 Coffee Break ...

                     Next
               Code Optimization



                  Carol Gauthier, Centre de Calcul Scientifique (CCS), Université de Sherbrooke

								
To top