Supercomputing in Plain English An Overview of High Performance

Document Sample
Supercomputing in Plain English An Overview of High Performance Powered By Docstoc
					Supercomputing in
  Plain English
       An Overview of
 High Performance Computing
          Henry Neeman, Director
OU Supercomputing Center for Education & Research
                University of Oklahoma
          Bradley University, December 2 2005
                          People




Supercomputing in Plain English: Overview of HPC
   Bradley University, Friday December 2 2005      2
                          Things




Supercomputing in Plain English: Overview of HPC
   Bradley University, Friday December 2 2005      3
        What is Supercomputing?
Supercomputing is the biggest, fastest computing
  right this minute.
Likewise, a supercomputer is one of the biggest,
  fastest computers right this minute.
So, the definition of supercomputing is constantly
  changing.
Rule of Thumb: A supercomputer is typically at
  least 100 times as powerful as a PC.
Jargon: Supercomputing is also known as
  High Performance Computing (HPC).


         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      4
Fastest Supercomputer vs. Moore




             GFLOPs: billions of calculations per second
    Supercomputing in Plain English: Overview of HPC
       Bradley University, Friday December 2 2005          5
What is Supercomputing About?


     Size                                         Speed




   Supercomputing in Plain English: Overview of HPC
      Bradley University, Friday December 2 2005          6
     What is Supercomputing About?
   Size: Many problems that are interesting to
    scientists and engineers can’t fit on a PC – usually
    because they need more than a few GB of RAM, or
    more than a few 100 GB of disk.
   Speed: Many problems that are interesting to
    scientists and engineers would take a very very
    long time to run on a PC: months or even years.
    But a problem that would take a month on a PC
    might take only a few hours on a supercomputer.


            Supercomputing in Plain English: Overview of HPC
               Bradley University, Friday December 2 2005      7
                   What Is It Used For?
   Simulation of physical phenomena, such as
       Weather forecasting   [1]
       Galaxy formation
       Oil reservoir management
   Data mining: finding needles of
                                                                 Moore, OK
    information in a haystack of data,                            Tornadic
    such as                                                        Storm

       Gene sequencing
                                                   May 3 1999[2]
       Signal processing
       Detecting storms that could produce tornados
   Visualization: turning a vast sea of data into
    pictures that a scientist can understand    [3]


              Supercomputing in Plain English: Overview of HPC
                 Bradley University, Friday December 2 2005                  8
                       What is OSCER?
   Multidisciplinary center
   Division of OU Information Technology
   Provides:
       Supercomputing education
       Supercomputing expertise
       Supercomputing resources: hardware, storage, software
   For:
       Undergrad students
       Grad students
       Staff
       Faculty
       Their collaborators (including off campus)
             Supercomputing in Plain English: Overview of HPC
                Bradley University, Friday December 2 2005      9
       Who is OSCER? Academic Depts
   Aerospace & Mechanical Engr                        Health & Sport Sciences
   Anthropology                                       History of Science
   Biochemistry & Molecular Biology                   Industrial Engr
   Biological Survey                                  Geography
   Botany & Microbiology                              Geology & Geophysics
   Chemical, Biological & Materials Engr              Library & Information Studies
   Chemistry & Biochemistry                           Mathematics
   Civil Engr & Environmental Science                 Meteorology
   Computer Science                                   Petroleum & Geological Engr
   Economics                                          Physics & Astronomy
   Electrical & Computer Engr                         Radiological Sciences
   Finance                                            Surgery
                                                       Zoology
More than 150 faculty & staff in 25 depts in Colleges of Arts & Sciences,
Business, Engineering, Geosciences and Medicine – with more to come!
               Supercomputing in Plain English: Overview of HPC
                  Bradley University, Friday December 2 2005                     10
          Who is OSCER? Organizations
   Advanced Center for Genome                      Langston University Mathematics Dept
    Technology                                      Microarray Core Facility
   Center for Analysis & Prediction of
    Storms                                          National Severe Storms Laboratory
   Center for Aircraft & Systems/Support           NOAA Storm Prediction Center
    Infrastructure                                  OU Office of Information Technology
   Cooperative Institute for Mesoscale             OU Office of the VP for Research
    Meteorological Studies
                                                    Oklahoma Center for High Energy
   Center for Engineering Optimization              Physics
   Fears Structural Engineering                    Oklahoma Climatological Survey
    Laboratory
                                                    Oklahoma EPSCoR
   Geosciences Computing Network
   Great Plains Network                            Oklahoma Medical Research Foundation
   Human Technology Interaction Center             Oklahoma School of Science & Math
   Institute of Exploration & Development          St. Gregory’s University Physics Dept
    Geosciences                                     Sarkeys Energy Center
   Instructional Development Program               Sasaki Applied Meteorology Research
   Laboratory for Robotic Intelligence and          Institute
    Machine Learning                                YOU COULD BE HERE!
                  Supercomputing in Plain English: Overview of HPC
                     Bradley University, Friday December 2 2005                     11
                   Biggest Consumers
   Center for Analysis & Prediction of Storms:
    daily real time weather forecasting
   Oklahoma Center for High Energy Physics:
    simulation and data analysis of banging tiny
    particles together at unbelievably high speeds
   Advanced Center for Genome Technology:
    bioinformatics (e.g., Human Genome Project)



           Supercomputing in Plain English: Overview of HPC
              Bradley University, Friday December 2 2005      12
                          Who Are the Users?
Over 250 users so far:
 over 50 OU faculty

 over 50 OU staff

 over 100 students

 about 20 off campus users

 … more being added every month.

Comparison: National Center for Supercomputing
  Applications (NCSA), after 20 years of history and
  hundreds of millions in expenditures, has about
  2100 users.*
*   Unique usernames on cu.ncsa.uiuc.edu and tungsten.ncsa.uiuc.edu
                    Supercomputing in Plain English: Overview of HPC
                       Bradley University, Friday December 2 2005      13
 What Does OSCER Do? Teaching




       Science and engineering faculty from all over America learn
supercomputing at OU by playing with a jigsaw puzzle (NCSI @ OU 2004).
           Supercomputing in Plain English: Overview of HPC
              Bradley University, Friday December 2 2005                 14
What Does OSCER Do? Rounds




  OU undergrads, grad students, staff and faculty learn
  how to use supercomputing in their specific research.
   Supercomputing in Plain English: Overview of HPC
      Bradley University, Friday December 2 2005          15
            Current OSCER Hardware
   TOTAL: 1484 GFLOPs*, 368 CPUs, 434 GB RAM
   Aspen Systems Pentium4 Xeon 32-bit Linux Cluster
       270 Pentium4 Xeon CPUs, 270 GB RAM, 1.08 TFLOPs
   Aspen Systems Itanium2 cluster
       66 Itanium2 CPUs, 132 GB RAM, 264 GFLOPs
   IBM Regatta p690 Symmetric Multiprocessor
       32 POWER4 CPUs, 32 GB RAM, 140.8 GFLOPs
   IBM FAStT500 FiberChannel-1 Disk Server
   Qualstar TLS-412300 Tape Library
* GFLOPs: billions of calculations per second

               Supercomputing in Plain English: Overview of HPC
                  Bradley University, Friday December 2 2005      16
         Coming OSCER Hardware (2005)
   TOTAL: 11,445 GFLOPs*, 1856 CPUs, 2508 GB RAM
   NEW! Dell Pentium4 Xeon 64-bit Linux Cluster
       1024 Pentium4 Xeon CPUs, 2240 GB RAM, 6.55 TFLOPs
   Aspen Systems Itanium2 cluster
       66 Itanium2 CPUs, 132 GB RAM, 264 GFLOPs
   COMING! 2 x 16-way Opteron Cluster
       16 AMD Opteron CPUs, 96 GB RAM, 128 GFLOPs
   NEW! Condor Pool: 750 student lab PCs
   COMING! National Lambda Rail
   Qualstar TLS-412300 Tape Library
* GFLOPs: billions of calculations per second

                  Supercomputing in Plain English: Overview of HPC
                     Bradley University, Friday December 2 2005      17
       Hardware: IBM p690 Regatta
32 POWER4 CPUs (1.1 GHz)
32 GB RAM
218 GB internal disk
OS: AIX 5.1
Peak speed: 140.8 GFLOPs*
Programming model:
 shared memory
 multithreading (OpenMP)
 (also supports MPI)
*GFLOPs:  billions of calculations
  per second
                                               sooner.oscer.ou.edu
             Supercomputing in Plain English: Overview of HPC
                Bradley University, Friday December 2 2005      18
Hardware: Pentium4 Xeon Cluster
270 Pentium4 XeonDP CPUs
270 GB RAM
~10,000 GB disk
OS: Red Hat Linux
   Enterprise 3
Peak speed: 1,080 GFLOPs*
Programming model:
  distributed multiprocessing
  (MPI)
* GFLOPs: billions of
   calculations per second


                                          boomer.oscer.ou.edu
         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      19
      Hardware: Itanium2 Cluster
66 Itanium2 1.0 GHz CPUs
132 GB RAM
5,774 GB disk
OS: Red Hat Linux
   Enterprise 3
Peak speed: 264 GFLOPs*
Programming model:
  distributed multiprocessing
  (MPI)
*GFLOPs: billions of
   calculations per second

                                       schooner.oscer.ou.edu
         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      20
         New! Pentium4 Xeon Cluster
1,024 Pentium4 Xeon CPUs
2,180 GB RAM
14,000 GB disk
Infiniband & Gigabit Ethernet
OS: Red Hat Linux Enterp 3
Peak speed: 6,553 GFLOPs*
Programming model:
  distributed multiprocessing
  (MPI)
*GFLOPs:  billions of calculations
  per second
                                       topdawg.oscer.ou.edu
                              DEBUTED AT #54 WORLDWIDE,
       www.top500.org
                              #9 AMONG US UNIVERSITIES,
                              #4 EXCLUDING BIG 3 NSF CENTERS
             Supercomputing in Plain English: Overview of HPC
                Bradley University, Friday December 2 2005      21
   Coming! National Lambda Rail
The National Lambda Rail (NLR) is the next
  generation of high performance networking.




        Supercomputing in Plain English: Overview of HPC
           Bradley University, Friday December 2 2005      22
               Coming! Condor Pool
Condor is a software package that allows number
  crunching jobs to run on idle desktop PCs.
OU IT is deploying a large Condor pool (750 desktop
  PCs) over the course of the 2005.
When deployed, it’ll provide a huge amount of
  additional computing power – more than is
  currently available in all of OSCER today.
And, the cost is very very low.


         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      23
                       What is Condor?
Condor is grid computing technology:
 it steals compute cycles from existing desktop PCs;

 it runs in background when no one is logged in.

Condor is like SETI@home, but better:
 it’s general purpose and can work for any

  “loosely coupled” application;
 it can do all of its I/O over the network, not using

  the desktop PC’s disk.


          Supercomputing in Plain English: Overview of HPC
             Bradley University, Friday December 2 2005      24
                  Current Status at OU
   Pool of approx 100 test machines in PC labs
   Submit/management from Neeman’s desktop PC
   Rollout to multiple labs during fall
   Total rollout to 750 PCs by end of 2005
   COMING: 2 submit nodes with large RAID,
    2 management nodes




           Supercomputing in Plain English: Overview of HPC
              Bradley University, Friday December 2 2005      25
Supercomputing
              Supercomputing Issues
   The tyranny of the storage hierarchy
   Parallelism: doing many things at the same time
       Instruction-level parallelism: doing multiple
        operations at the same time within a single processor
        (e.g., add, multiply, load and store simultaneously)
       Multiprocessing: multiple CPUs working on different
        parts of a problem at the same time
           Shared Memory Multithreading

           Distributed Multiprocessing

   High performance compilers
   Scientific Libraries
   Visualization
            Supercomputing in Plain English: Overview of HPC
               Bradley University, Friday December 2 2005      27
A Quick Primer
 on Hardware
                     Henry’s Laptop

Gateway M275 Tablet[4]                 Pentium 4 1.5 GHz
                                        w/1 MB L2 Cache
                                       512 MB 400 MHz
                                        DDR SDRAM
                                       40 GB 4200 RPM Hard Drive
                                       Floppy Drive
                                       DVD/CD-RW Drive
                                       10/100 Mbps Ethernet
                                       56 Kbps Phone Modem

         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005          29
     Typical Computer Hardware
   Central Processing Unit
   Primary storage
   Secondary storage
   Input devices
   Output devices




         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      30
          Central Processing Unit
Also called CPU or processor: the “brain”
Parts:
 Control Unit: figures out what to do next --
  e.g., whether to load data from memory, or to
  add two values together, or to store data into
  memory, or to decide which of two possible
  actions to perform (branching)
 Arithmetic/Logic Unit: performs calculations –
  e.g., adding, multiplying, checking whether two
  values are equal
 Registers: where data reside that are being used
  right now
         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      31
                        Primary Storage
   Main Memory
       Also called RAM (“Random Access Memory”)
       Where data reside when they’re being used by a
        program that’s currently running
   Cache
       Small area of much faster memory
       Where data reside when they’re about to be used
        and/or have been used recently
   Primary storage is volatile: values in primary
    storage disappear when the power is turned off.

             Supercomputing in Plain English: Overview of HPC
                Bradley University, Friday December 2 2005      32
                   Secondary Storage
   Where data and programs reside that are going to
    be used in the future
   Secondary storage is non-volatile: values don’t
    disappear when power is turned off.
   Examples: hard disk, CD, DVD, magnetic tape,
    Zip, Jaz
   Many are portable: can pop out the
    CD/DVD/tape/Zip/floppy and take it with you


           Supercomputing in Plain English: Overview of HPC
              Bradley University, Friday December 2 2005      33
                           Input/Output
   Input devices – e.g., keyboard, mouse, touchpad,
    joystick, scanner
   Output devices – e.g., monitor, printer, speakers




           Supercomputing in Plain English: Overview of HPC
              Bradley University, Friday December 2 2005      34
   The Tyranny of
the Storage Hierarchy
        The Storage Hierarchy
                   [5]




Fast, expensive, few  Registers
                      Cache memory
                      Main memory (RAM)
                      Hard disk
                      Removable media (e.g., CDROM)

 Slow, cheap, a lot  Internet
                                       [6]




         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      36
                              RAM is Slow
The speed of data transfer
                                        CPU 67 GB/sec[7]
between Main Memory and the
CPU is much slower than the
speed of calculating, so the CPU               Bottleneck
spends most of its time waiting
for data to come in or go out.                               3.2 GB/sec[9] (5%)




              Supercomputing in Plain English: Overview of HPC
                 Bradley University, Friday December 2 2005                       37
                      Why Have Cache?
Cache is nearly the same speed
                                        CPU 67 GB/sec[7]
as the CPU, so the CPU doesn’t
have to wait nearly as long for
stuff that’s already in cache:                           48 GB/sec[8] (72%)
it can do more
operations per second!                                       3.2 GB/sec[9] (5%)




              Supercomputing in Plain English: Overview of HPC
                 Bradley University, Friday December 2 2005                       38
           Henry’s Laptop, Again

Gateway M275 Tablet[4]                 Pentium 4 1.5 GHz
                                        w/1 MB L2 Cache
                                       512 MB 400 MHz
                                        DDR SDRAM
                                       40 GB 4200 RPM Hard Drive
                                       Floppy Drive
                                       DVD/CD-RW Drive
                                       10/100 Mbps Ethernet
                                       56 Kbps Phone Modem

         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005          39
                Storage Speed, Size, Cost
              Registers    Cache         Main      Hard       Ethernet     CD-RW         Phone
             (Pentium 4   Memory        Memory     Drive    (100 Mbps)                  Modem
Henry’s                                                                                (56 Kbps)
              1.5 GHz)      (L2)       (400 MHz
Laptop                                   DDR
                                       SDRAM)
 Speed       68,664[7]    49,152 [8]    3,277       100            12          4         0.007
                                          [9]       [10]                      [11]
(MB/sec)      (3000
 [peak]     MFLOP/s*)

   Size     304 bytes**       1          512      40,000     unlimited     unlimited   unlimited
                [12]
  (MB)


  Cost                     $90 [13]     $0.09     $0.0004      charged     $0.0007       charged
                                         [13]       [13]                      [13]
 ($/MB)          –                                           per month                 per month
                                                             (typically)               (typically)

* MFLOP/s: millions of floating point operations per second
** 8 32-bit integer registers, 8 80-bit floating point registers, 8 64-bit MMX integer registers,
   8 128-bit floating point XMM registers
                Supercomputing in Plain English: Overview of HPC
                   Bradley University, Friday December 2 2005                                  40
              Storage Use Strategies
   Register reuse: do a lot of work on the same data
    before working on new data.
   Cache reuse: the program is much more
    efficient if all of the data and instructions fit in
    cache; if not, try to use what’s in cache a lot
    before using anything that isn’t in cache.
   Data locality: try to access data that are near
    each other in memory before data that are far.
   I/O efficiency: do a bunch of I/O all at once
    rather than a little bit at a time; don’t mix
    calculations and I/O.
           Supercomputing in Plain English: Overview of HPC
              Bradley University, Friday December 2 2005      41
Parallelism
                       Parallelism
Parallelism means
doing multiple things at
the same time: you can
get more work done in
the same time.
     Less fish …




                                                     More fish!
         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005            43
The Jigsaw Puzzle Analogy




 Supercomputing in Plain English: Overview of HPC
    Bradley University, Friday December 2 2005      44
          Serial Computing
             Suppose you want to do a jigsaw puzzle
             that has, say, a thousand pieces.

             We can imagine that it’ll take you a
             certain amount of time. Let’s say
             that you can put the puzzle together in
             an hour.




Supercomputing in Plain English: Overview of HPC
   Bradley University, Friday December 2 2005          45
Shared Memory Parallelism
                  If Julie sits across the table from you,
                  then she can work on her half of the
                  puzzle and you can work on yours.
                  Once in a while, you’ll both reach into
                  the pile of pieces at the same time
                  (you’ll contend for the same resource),
                  which will cause a little bit of
                  slowdown. And from time to time
                  you’ll have to work together
                  (communicate) at the interface
                  between her half and yours. The
                  speedup will be nearly 2-to-1: y’all
                  might take 35 minutes instead of 30.

 Supercomputing in Plain English: Overview of HPC
    Bradley University, Friday December 2 2005          46
  The More the Merrier?
                  Now let’s put Lloyd and Jerry on the
                  other two sides of the table. Each of
                  you can work on a part of the puzzle,
                  but there’ll be a lot more contention
                  for the shared resource (the pile of
                  puzzle pieces) and a lot more
                  communication at the interfaces. So
                  y’all will get noticeably less than a
                  4-to-1 speedup, but you’ll still have
                  an improvement, maybe something
                  like 3-to-1: the four of you can get it
                  done in 20 minutes instead of an hour.


Supercomputing in Plain English: Overview of HPC
   Bradley University, Friday December 2 2005           47
     Diminishing Returns
                  If we now put Dave and Paul and Tom
                  and Charlie on the corners of the
                  table, there’s going to be a whole lot
                  of contention for the shared resource,
                  and a lot of communication at the
                  many interfaces. So the speedup y’all
                  get will be much less than we’d like;
                  you’ll be lucky to get 5-to-1.

                  So we can see that adding more and
                  more workers onto a shared resource
                  is eventually going to have a
                  diminishing return.

Supercomputing in Plain English: Overview of HPC
   Bradley University, Friday December 2 2005           48
               Distributed Parallelism



Now let’s try something a little different. Let’s set up two
tables, and let’s put you at one of them and Julie at the other.
Let’s put half of the puzzle pieces on your table and the other
half of the pieces on Julie’s. Now y’all can work completely
independently, without any contention for a shared resource.
BUT, the cost of communicating is MUCH higher (you have
to scootch your tables together), and you need the ability to
split up (decompose) the puzzle pieces reasonably evenly,
which may be tricky to do for some puzzles.
             Supercomputing in Plain English: Overview of HPC
                Bradley University, Friday December 2 2005         49
More Distributed Processors
                                           It’s a lot easier to add
                                           more processors in
                                           distributed parallelism.
                                           But, you always have to
                                           be aware of the need to
                                           decompose the problem
                                           and to communicate
                                           between the processors.
                                           Also, as you add more
                                           processors, it may be
                                           harder to load balance
                                           the amount of work that
                                           each processor gets.

 Supercomputing in Plain English: Overview of HPC
    Bradley University, Friday December 2 2005                  50
                        Load Balancing




Load balancing means giving everyone roughly the same
amount of work to do.
For example, if the jigsaw puzzle is half grass and half sky,
then you can do the grass and Julie can do the sky, and then
y’all only have to communicate at the horizon – and the
amount of work that each of you does on your own is
roughly equal. So you’ll get pretty good speedup.
            Supercomputing in Plain English: Overview of HPC
               Bradley University, Friday December 2 2005       51
                        Load Balancing




Load balancing can be easy, if the problem splits up into
chunks of roughly equal size, with one chunk per
processor. Or load balancing can be very hard.
            Supercomputing in Plain English: Overview of HPC
               Bradley University, Friday December 2 2005      52
Moore’s Law
                          Moore’s Law
In 1965, Gordon Moore was an engineer at Fairchild
   Semiconductor.
He noticed that the number of transistors that could be
   squeezed onto a chip was doubling about every 18
   months.
It turns out that computer speed is roughly
   proportional to the number of transistors per unit
   area.
Moore wrote a paper about this concept, which
   became known as “Moore’s Law.”
          Supercomputing in Plain English: Overview of HPC
             Bradley University, Friday December 2 2005      54
Fastest Supercomputer vs. Moore




             GFLOPs: billions of calculations per second
    Supercomputing in Plain English: Overview of HPC
       Bradley University, Friday December 2 2005          55
Why Bother?
     Why Bother with HPC at All?
It’s clear that making effective use of HPC takes
   quite a bit of effort, both learning how and
   developing software.
That seems like a lot of trouble to go to just to get
   your code to run faster.
It’s nice to have a code that used to take a day run
   in an hour. But if you can afford to wait a day,
   what’s the point of HPC?
Why go to all that trouble just to get your code to
   run faster?
         Supercomputing in Plain English: Overview of HPC
            Bradley University, Friday December 2 2005      57
      Why HPC is Worth the Bother
   What HPC gives you that you won’t get
    elsewhere is the ability to do bigger, better,
    more exciting science. If your code can run
    faster, that means that you can tackle much
    bigger problems in the same amount of time that
    you used to need for smaller problems.
   HPC is important not only for its own sake, but
    also because what happens in HPC today will be
    on your desktop in about 15 years: it puts you
    ahead of the curve.
          Supercomputing in Plain English: Overview of HPC
             Bradley University, Friday December 2 2005      58
                  The Future is Now
Historically, this has always been true:
  Whatever happens in supercomputing today
  will be on your desktop in 10 – 15 years.
So, if you have experience with supercomputing,
  you’ll be ahead of the curve when things get to the
  desktop.




          Supercomputing in Plain English: Overview of HPC
             Bradley University, Friday December 2 2005      59
Weather Stuff
                  Traditional Forecasting/Simulation
                            Methodology



       OBSERVATIONS                          Analysis/Assimilation       Prediction/Detection     Product Generation,
                                                                                                        Display,
          Radar Data                             Quality Control        PCs to Teraflop Systems      Dissemination
        Mobile Mesonets                     Retrieval of Unobserved
     Surface Observations
                                                   Quantities
      Upper-Air Balloons
                                           Creation of Gridded Fields
      Commercial Aircraft
Geostationary and Polar Orbiting
               Satellite
         Wind Profilers
         GPS Satellites
                                      The Process is Entirely Serial
                                       and Static (Pre-Scheduled):
                                      No Response to the Weather!
                                                                                                      End Users

                                                                                                         NWS
                                   Supercomputing in Plain English: Overview of HPC
                                                                                                  Private Companies
                                      Bradley University, Friday December 2 2005                             61
                                                                                                       Students
                    The Future: Weather Observations
                   and Model Solutions Driving Models



       OBSERVATIONS                          Analysis/Assimilation       Prediction/Detection     Product Generation,
                                                                                                        Display,
          Radar Data                             Quality Control        PCs to Teraflop Systems      Dissemination
        Mobile Mesonets                     Retrieval of Unobserved
     Surface Observations
                                                   Quantities
      Upper-Air Balloons
                                           Creation of Gridded Fields
      Commercial Aircraft
Geostationary and Polar Orbiting
               Satellite
         Wind Profilers
         GPS Satellites




                                                                                                      End Users

                                                                                                         NWS
                                   Supercomputing in Plain English: Overview of HPC
                                                                                                  Private Companies
                                      Bradley University, Friday December 2 2005                             62
                                                                                                       Students
                   The Future: Models Driving Remote
                            Sensing Devices



       OBSERVATIONS                          Analysis/Assimilation       Prediction/Detection     Product Generation,
                                                                                                        Display,
          Radar Data                             Quality Control        PCs to Teraflop Systems      Dissemination
        Mobile Mesonets                     Retrieval of Unobserved
     Surface Observations
                                                   Quantities
      Upper-Air Balloons
                                           Creation of Gridded Fields
      Commercial Aircraft
Geostationary and Polar Orbiting
               Satellite
         Wind Profilers
         GPS Satellites




                                                                                                      End Users

                                                                                                         NWS
                                   Supercomputing in Plain English: Overview of HPC
                                                                                                  Private Companies
                                      Bradley University, Friday December 2 2005                             63
                                                                                                       Students
           What do We Need for a Truly
              Adaptive Capability?
   Adaptive tools (weather models, hazardous
    weather detection systems)
       In space
       In time
       In configuration
   Adaptive sensors
   Adaptive cyberinfrastructure




              Supercomputing in Plain English: Overview of HPC
                 Bradley University, Friday December 2 2005      64
     New NSF Engineering Research Center for
    Adaptive Sensing of the Atmosphere (CASA)
   UMass/Amherst, OU, CSU, UPRM
   Concept: inexpensive, phased array
    Doppler radars on cell towers and
    buildings




            Supercomputing in Plain English: Overview of HPC
               Bradley University, Friday December 2 2005      65
     New NSF Engineering Research Center for
    Adaptive Sensing of the Atmosphere (CASA)
   Dynamically adaptive sensing of
    multiple targets while simultaneously
    meeting multiple end-user needs
   Complementary to NEXRAD




            Supercomputing in Plain English: Overview of HPC
               Bradley University, Friday December 2 2005      66
   Limitations of Current NEXRAD
                     #1. Operates largely independently
                     of the prevailing weather conditions




                        #2. Earth’s curvature
                       prevents 72% of the
                     atmosphere below 1 km
                       from being observed
 #3. Operates entirely independently from
the models and algorithms that use its data

       Supercomputing in Plain English: Overview of HPC
          Bradley University, Friday December 2 2005        67
         Second Generation:
Fully Solid State Electronic Scanning




      Supercomputing in Plain English: Overview of HPC
         Bradley University, Friday December 2 2005      68
Supercomputing in Plain English: Overview of HPC
   Bradley University, Friday December 2 2005      69
  One-Way Static Data Flow Today



Distributed
radars

                                                                 Models




              Supercomputing in Plain English: Overview of HPC
                 Bradley University, Friday December 2 2005               70
     Two-Way Dynamic Data Flow



Distributed
radars

                                                                 Models




              Supercomputing in Plain English: Overview of HPC
                 Bradley University, Friday December 2 2005               71
LEAD: Linked Environments
 for Atmospheric Discovery




  Supercomputing in Plain English: Overview of HPC
     Bradley University, Friday December 2 2005      72
             The LEAD Goal
          Provide the IT necessary to allow
People (scientists, students, operational practitioners)
                          and
    Technologies (models, sensors, data mining)

        TO INTERACT WITH
             WEATHER

          Supercomputing in Plain English: Overview of HPC
             Bradley University, Friday December 2 2005      73
        The LEAD Goal Restated
To create an integrated, scalable framework that allows
  analysis tools, forecast models, and data repositories
  to be used as dynamically adaptive, on-demand
  systems that can
 change configuration rapidly and automatically
  in response to weather;
 continually be steered by new data (i.e., the weather);
 respond to decision-driven inputs from users;
 initiate other processes automatically; and
 steer remote observing technologies to optimize data
  collection for the problem at hand;
 operate independent of data formats and the physical
  location of data or computing resources
          Supercomputing in Plain English: Overview of HPC
             Bradley University, Friday December 2 2005      74
                 Sample Problem Scenario




                                                               Storms
                                                               Forming


                                                                         Forecast Model
 Streaming
Observations                          Data Mining




               Supercomputing in Plain English: Overview of HPC
                                                    Visualization         On-Demand
                  Bradley University, Friday December 2 2005                      75
                                                                         Grid Computing
       In LEAD, Most Everything
            is a Web Service
 Service A                   Service B                       Service C
  (ADAS)                      (WRF)                        (NEXRAD Stream)


 Service D                  Service E                         Service F
(MyLEAD)                  (VO Catalog)                         (IDV)

 Service G                   Service H                        Service I
(Monitoring)               (Scheduling)                       (ESML)

  Service J                  Service K                        Service L
(Repository)                (Ontology)                        (Decoder)

                         Many others…
        Supercomputing in Plain English: Overview of HPC
           Bradley University, Friday December 2 2005                        76
                                        References
[1] Image by Greg Bryan, MIT: http://zeus.ncsa.uiuc.edu:8080/chdm_script.html
[2] “Update on the Collaborative Radar Acquisition Field Test (CRAFT): Planning for the Next Steps.”
    Presented to NWS Headquarters August 30 2001.
[3] See http://scarecrow.caps.ou.edu/~hneeman/hamr.html for details.
[4] http://www.gateway.com/
[5] http://www.f1photo.com/
[6] http://www.vw.com/newbeetle/
[7] Richard Gerber, The Software Optimization Cookbook: High-performance Recipes for the Intel
Architecture. Intel Press, 2002, pp. 161-168.
[8] http://www.anandtech.com/showdoc.html?i=1460&p=2
[9] ftp://download.intel.com/design/Pentium4/papers/24943801.pdf
[10] http://www.seagate.com/cda/products/discsales/personal/family/0,1085,621,00.html
[11] http://www.toshiba.com/taecdpd/techdocs/sdr2002/2002spec.shtml
[12] ftp://download.intel.com/design/Pentium4/manuals/24896606.pdf
[13] http://www.pricewatch.com/
[14] Steve Behling et al, The POWER4 Processor Introduction and Tuning Guide, IBM, 2001, p. 8.
[15] Kevin Dowd and Charles Severance, High Performance Computing,
     2nd ed. O’Reilly, 1998, p. 16.
[16] http://emeagwali.biz/photos/stock/supercomputer/black-shirt/




                    Supercomputing in Plain English: Overview of HPC
                       Bradley University, Friday December 2 2005                                 77

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:11/22/2011
language:English
pages:77