Docstoc

Center for Adaptive Supercomputing Software

Document Sample
Center for Adaptive Supercomputing Software Powered By Docstoc
					Software and Hardware Requirements for
     Next-Generation Data Analytics

                      John Feo
     Center for Adaptive Supercomputing Software
        Pacific Northwest National Laboratory


                    October, 2010
Graphs are everywhere in science

       Astrophysics
Problem: Outlier detection.
Challenges: massive datasets,
temporal variations.                    Bioinformatics
Graph problems: clustering,     Problem: Identifying drug target
matching.                       proteins.
                                                                          Social Informatics
                                Challenges: Data heterogeneity,
                                                                   Problem: Discover emergent
                                quality.
                                                                   communities, model spread of
                                Graph problems: centrality,
                                                                   information.
                                clustering.
                                                                   Challenges: new analytics routines,
                                                                   uncertainty in data.
                                                                   Graph problems: clustering,
                                                                   shortest paths, flows.
… and in commerce
               has more than 300 million active users                              1000x
                                                                                  growth
                                                                                in 3 years!




  Sample queries:
     Allegiance switching: identify entities that switch communities.
     Community structure: identify the genesis and dissipation of communities
     Phase change: identify significant change in the network structure
     Thought leaders: identify influential individuals that drive events
  Graph features:
     Topology: Interaction graph is low-diameter and has no good separators
     Irregularity: Communities are not uniform in size
     Overlap: individuals are members of one or more communities
    Small-world and scale-free
                                                                               “Six degrees of separation”
                           Low diameter (small-world):
                                 work explodes
                                 difficult to partition/load-balance
                                 high % of nodes are visited quickly

                          Scale-free (power-law):
                                 difficult to partition/load-balance
                                 work concentrates in a few nodes
                          1.00
     Ratio of edges cut




                          0.50
                                                                   Block   RMAT
                                                                   k-way   graph with a
                                                                           million
                          0.25
                                                                           vertices
4                                           Number of partitions
    Grids, Erdős–Rényi, and Scale-Free Graphs
           USA Roadmap
                          Communication trace from execution
                            of ½-approx weighted matching
                             (data distributed using Metis)



                                        Scale-Free




           Erdős–Rényi




5
Challenges
  Problem size
     Ton of bytes, not ton of flops
  Little data locality
     Have only parallelism to tolerate latencies
  Low computation to communication ratio
     Single word access
     Threads limited by loads and stores
  Synchronization points are simple elements
     Node, edge, record
  Work tends to be dynamic and imbalanced
     Let any processor execute any thread
System requirements
  Global shared memory
     No simple data partitions
     Local storage for thread private data
  Network support for single word accesses              Cray XMT

     Transfer multiple words when locality exists
  Multi-threaded processors
     Hide latency with parallelism
     Single cycle context switching
     Multiple outstanding loads and stores per thread
  Full-and-empty bits
     Efficient synchronization
     Wait in memory
  Message driven operations
     Dynamic work queues
     Hardware support for thread migration
Center for Adaptive Supercomputer Software

                      Driving development of next-generation
                     multithreaded architectures and methods for
                                 irregular problems
                                                     Sponsored by DOD




                                    Data Analytics                   Commerce
   Scientific Simulations

               Internet

                              Knowledge Discovery          Science
        DATA
   Sensor Networks

                                   Trend Analysis                       Policy
            Databases
Partners
Analytic methods and applications
      FaceBook - 300 M users                                                               SmartGrid

                                                     National Security

                                                  Train
                                                              Anthrax
                                 Bus

                                                                  Money
                                                  Endo

                                        Hayashi
 Community Activities                                     Zaire            N-x contingency analysis
                                       Connect-the-dots
                                                                                          Security

                                                            Semantic Web
                 Blog Analysis




                                                                               Anomaly detection


                                       People, Places, & Actions
Community thought leaders
Research focus areas
 Applications




                 SmartGrid      Sensor Networks          BioInformatics          Computer Security
 Methods




                    Bayesian networks      Social networks        Mesh generation     MapReduce

                       Clustering       Semantic Databases          N-x contingency analysis
 Languages




                                        Chapel for hybrid systems


                  Compiler and runtime system                Performance analysis and tools
Runtime
System




                               Communication software for hybrid systems
  Architecture




                               Next generation multithreaded architectures
Methods for data analytics
 Paths                       Influential Factors
    Shortest path          Degree distribution
                                                     Load imbalance
    Betweenness                Normal                  Non-planar
    Min/max flow               Scale-free
                                                       Difficult to partition
 Structures                Planar or non-planar
    Spanning trees
    Connected components   Static or dynamic         Concurrent inserts
                                                       and deletions
    Graph isomorphism
                           Weighted or unweighted
 Groups                        Weight distribution
    Matching/Coloring
    Partitioning           Typed or untyped edges
    Equivalence
Systems for large-scale analytics



            Netezza TwinFin


 Cray XMT




                                Graph        RDBS
                               resides in   runs on
                              XMT memory     cluster
  Dynamic Bayesian Network Model for
  Atmospheric Sensor Network Validation
                      vap                                              vap                                              vap




        tbsk           sky ir                            tbsk           sky ir                            tbsk           sky ir
        y 31           temp     wsp                      y 31           temp     wsp                      y 31           temp     wsp
                                d_va                                             d_va                                             d_va




rada                                    precip   rada                                    precip   rada                                    precip
 r7                                      -tbrg    r7                                      -tbrg    r7                                      -tbrg




                                percent_op                                       percent_op                                       percent_op
       rada                        aque                 rada                        aque                 rada                        aque
        r13                                              r13                                              r13




               rada                                             rada                                             rada
                r19                                              r19                                              r19




        Replicate per time step
        Add dependencies across time steps (not shown)
DBN to Junction Tree Conversion
                    v                                                v                                                v
                    a                                                a                                                a
                    p                                                p                                                p




           t                                                t                                                 t
           b            sky   w                             b            sky   w                              b           sky   w
           s             ir   s                             s             ir   s                              s            ir   s
           k            tem   p                             k            tem   p                              k           tem   p
           y             p    d                             y             p    d                              y            p    d
           3                  _                             3                  _                              3                 _
           1                  v                             1                  v                              1                 v
                              a                                                a                                                a




  r                                                 r                                                 r
  a                                          prec   a                                         prec    a                                        prec
  d                                           ip-   d                                          ip-    d                                         ip-
  a                                          tbrg   a                                         tbrg    a                                        tbrg
  r                                                 r                                                 r
  7                                                 7                                                 7




       r                                                r                                                 r
       a                      percent_opaq              a                      percent_opaq               a                     percent_opaq
       d                           ue                   d                           ue                    d                          ue
       a                                                a                                                 a
       r                                                r                                                 r
       1                                                1                                                 1
       3                                                3                                                 3




                r                                               r                                                 r
                a                                               a                                                 a
                d                                               d                                                 d
                a                                               a                                                 a
                r                                               r                                                 r
                1                                               1                                                 1
                9                                               9                                                 9




  Convert dynamic Bayesian network to junction tree for inferencing
  Each node in the junction tree is a clique or super node containing several nodes
  from original Bayesian network
  Junction Tree based “Evidence Propagation” is an efficient method of propagating
  the effect of any variable’s state to every other variable in the BN


                                                                    Model as
Temp           Cloudy             Rain                              Variables                        Model as                                         Temp,    Cloudy,
                                                                                                     Cliques                                          Cloudy    Rain
Evidence Propagation is highly irregular
                 SMALL SYSTEMS HAVE   100S OF MILLIONS OF NODES


                             Compute per node is unbalanced
                             Degree per node is irregular
                             Data moves up and down




                               Loop parallelism intra-node
                               Task parallelism inter-node
                               (recursion, futures)
                               Data flow scheduling
                               Data synchronization
Atmospheric Sensor Network Validation Framework
 Semantic analysis                                                                  PNNL, SNL, Cray

          Understanding the relationships among data
               Data intensive science
               National security
               Commerce
          Data and relationships best expressed as triples and graphs
               <John owns Dog>
                                                                          Mary
                                                   Blue
                                                  bumps
                                                                                                          Pink
     Patient     Blue   Pink rash   High fever                               has symptom
                                                                                                          rash
                bumps
                                                      has symptom
     John        Yes        _          Yes                                                         has symptom
                                                   John
                                                                          High
      Alice       _        Yes          _                                 Fever
                                                                                           Alice
                                                            has symptom

      Mary        _         _          Yes


Mayo Clinic’s patient database has 650K columns



18
18
 XMT’s potential for semantic analysis
      RDFS closure
           Inferring new relationships and attributes
           Rule based

                                                 JOB 3: Delete Duplicates


          <John studied under Jim Browne>
                         +
         <Jim Browne teaches at UT Austin>
                         
             <John attended UT Austin>
                                                 JOB 0: Transitive Closure


                                                    Original Diagram from Urbani et al. "Scalable Distributed Reasoning using MapReduce" ISWC 2009

865 million triples
       Machine           Programming Model       Performance                          Author
                                             (inferences per sec)
  X86, 32 nodes, 128             MPI           ~ 600 K inf/sec               Weaver and Hendler
        cores                                                                  (ISWC 2009)
  X86, 64 nodes, 256           Hadoop           ~550K – 800K                       Urbani et al
        cores                                                                     (ESWC 2010)
   256 Treadstorm                C++         ~2.2M w/ read time
     processors                              ~13M w/o read time
Summary

 The new HPC is irregular and sparse
   Bad news: we need new architectures
   Good news: there are commercial and consumer applications

 Shared memory is necessary, but not sufficient
   Need processors that can fill the memory system with requests
   Need memory systems that support millions of simultaneous requests
   Need fine-grain hardware synchronization in memory

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:0
posted:4/17/2013
language:English
pages:20