A-Hierarchical-Environment-for-Virtual-Supercomputing

Document Sample
A-Hierarchical-Environment-for-Virtual-Supercomputing Powered By Docstoc
					 A Hierarchical Environment
 for Virtual Supercomputing

Scientific Computing Associates, Inc.
      New Haven, Connecticut
            (203) 777-7442
    Software@LindaSpaces.com
    http://www.LindaSpaces.com



                                        SCIENTIFIC
                                        COMPUTING ASSOCIATES
                                           Forward Thinking
A Hierarchical Environment for Virtual Supercomputing
                                                            Eval
                                                           Server
                                                                         Application
                                                                          Application
                                                                         Thread
                                                                           Application
                                                                          Thread
                                                                            Application
                                                                           Thread
                                                                              Application
                                                                            Thread
                                                                              Thread
                                                                                                                                                           New Ideas
                                                           Thread


                                                  Proxy
                                                 Manager
                                                 Thread             Process Memory
                                                                                                                                               Cost-effective HPC platforms will
                          Application
                           Application
                                                                                                                            Application
                                                                                                                             Application
                                                                                                                                               employ hierarchical architectures
                          Thread
                            Application                                                                                     Thread
                                                                                                                              Application
             Eval          Thread
                             Application                                                                    Eval             Thread
                                                                                                                               Application
            Server
            Thread
                            Thread
                               Application
                             Thread
                               Thread
                                                                                                           Server
                                                                                                           Thread
                                                                                                                              Thread
                                                                                                                                 Application
                                                                                                                               Thread
                                                                                                                                 Thread
                                                                                                                                               with heterogeneous components
  Proxy                                                                                           Proxy
 Manager                                                                                         Manager
 Thread                                                                                          Thread
                     Process Memory                                                                                 Process Memory

                                                                                                                                               Virtual shared memory systems
                                    Application
                                     Application
                                    Thread
                                      Application
                                                                                                                Application
                                                                                                                 Application
                                                                                                                Thread
                                                                                                                  Application
                                                                                                                                               can provide the conceptual
                      Eval           Thread
                                       Application                                                Eval           Thread
                                                                                                                   Application
                                      Thread                                                                      Thread
                                                                                                                     Application

                                                                                                                                               foundation for a uniform, high-
                     Server              Application
                                       Thread                                                    Server            Thread
                                         Thread                                                  Thread              Thread
                     Thread


            Proxy                                                                       Proxy
                                                                                       Manager

                                                                                                                                               performance software environment
           Manager
           Thread                                                                      Thread              Process Memory
                              Process Memory




                                                    Impact
                         Portable, high-performance software for parallel machines
                         Reduction in cost of software development and maintenance
                         Increased software lifetimes


                                                                                                                                                                     SCIENTIFIC
                                                                                                                                                                     COMPUTING ASSOCIATES
                                                                                                                                                                        Forward Thinking
Long-Range Hierarchical Target Architecture
Cluster of hierarchical
memory machines                                                                                                                                                                                                                            Eval
                                                                                                                                                                                                                                          Server
                                                                                                                                                                                                                                          Thread
                                                                                                                                                                                                                                                        Application
                                                                                                                                                                                                                                                         Application
                                                                                                                                                                                                                                                        Thread
                                                                                                                                                                                                                                                          Application
                                                                                                                                                                                                                                                         Thread
                                                                                                                                                                                                                                                           Application
                                                                                                                                                                                                                                                          Thread
                                                                                                                                                                                                                                                             Application
                                                                                                                                                                                                                                                           Thread
                                                                                                                                                                                                                                                             Thread




                                                                                                                                                                                                                                                                                                                                                                                                                                                 Multiple SMPs
                                                                                                                                                                                                                                 Proxy
                                                                                                                                                                                                                                Manager
                                                                                                                                                                                                                                Thread             Process Memory




                                                                                                                                                                                                       Application                                                                                         Application
                                                                                                                                                                                                        Application
                                                                                                                                                                                                       Thread                                                                                               Application
                                                                                                                                                                                                                                                                                                           Thread
                                                                                                                                                                                                         Application
                                                                                                                                                                                                        Thread                                                                                               Application
                                                                                                                                                                                                                                                                                                            Thread
                                                                                                                                                                                     Eval                 Application                                                                      Eval               Application




                                                                                                                                                                                                                                                                                                                                                                                                                                                 connected on high
                                                                                                                                                                                                         Thread
                                                                                                                                                                                                            Application                                                                                      Thread
                                                                                                                                                                                                                                                                                                                Application
                                                                                                                                                                                    Server                Thread                                                                          Server              Thread
                                                                                                                                                                                    Thread                  Thread                                                                        Thread                Thread



                                                                                                                                                                          Proxy                                                                                                  Proxy
                                                                                                                                                                         Manager                                                                                                Manager
                                                                                                                                                                         Thread               Process Memory                                                                    Thread             Process Memory




                                                                                                                                                                                               Eval
                                                                                                                                                                                              Server
                                                                                                                                                                                                                   Application
                                                                                                                                                                                                                    Application
                                                                                                                                                                                                                   Thread
                                                                                                                                                                                                                     Application
                                                                                                                                                                                                                    Thread
                                                                                                                                                                                                                      Application
                                                                                                                                                                                                                     Thread
                                                                                                                                                                                                                        Application
                                                                                                                                                                                                                      Thread
                                                                                                                                                                                                                                                                                 Eval
                                                                                                                                                                                                                                                                                Server
                                                                                                                                                                                                                                                                                               Application
                                                                                                                                                                                                                                                                                                Application
                                                                                                                                                                                                                                                                                               Thread
                                                                                                                                                                                                                                                                                                 Application
                                                                                                                                                                                                                                                                                                Thread
                                                                                                                                                                                                                                                                                                  Application
                                                                                                                                                                                                                                                                                                 Thread
                                                                                                                                                                                                                                                                                                    Application
                                                                                                                                                                                                                                                                                                  Thread
                                                                                                                                                                                                                                                                                                    Thread
                                                                                                                                                                                                                                                                                                                                                                                                                                                 performance
                                                                                                                                                                                              Thread                    Thread                                                  Thread


                                                                                                                                                                                    Proxy
                                                                                                                                                                                   Manager
                                                                                                                                                                                   Thread                 Process Memory
                                                                                                                                                                                                                                                                       Proxy
                                                                                                                                                                                                                                                                      Manager
                                                                                                                                                                                                                                                                      Thread              Process Memory                                                                                                                                         network fabrics

                                                                               Application                                                                                                                                                                                                                                                                                                                                          Application
                                                                                Application
                                                                               Thread                                                                                                                                                                                                                                                                                                                                                Application
                                                                                                                                                                                                                                                                                                                                                                                                                                    Thread
                                                                                 Application
                                                                                Thread                                                                                                                                                                                                                                                                                                                                                Application
                                                                                                                                                                                                                                                                                                                                                                                                                                     Thread
                                                                 Eval             Application
                                                                                 Thread                                                                                                                                                                                                                                                                                                                              Eval              Application
                                                                                                                                                                                                                                                                                                                                                                                                                                      Thread
                                                                Server              Application
                                                                                  Thread                                                                                                                                                                                                                                                                                                                            Server               Application
                                                                                                                                                                                                                                                                                                                                                                                                                                       Thread
                                                                Thread              Thread                                                                                                                                                                                                                                                                                                                          Thread               Thread



                                                       Proxy                                                                                                                                                                                                                                                                                                                                              Proxy
                                                      Manager                                                                                                                                                                                                                                                                                                                                            Manager
                                                      Thread              Process Memory                                                                                                                                                                                                                                                                                                                 Thread               Process Memory




                               Application                                                                                          Application                                                                                                                                                                                                                                 Application                                                                                            Application
                                Application
                               Thread                                                                                                Application
                                                                                                                                    Thread                                                                                                                                                                                                                                       Application
                                                                                                                                                                                                                                                                                                                                                                                Thread                                                                                                  Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Thread
                                 Application
                                Thread                                                                                                Application
                                                                                                                                     Thread                                                                                                                                                                                                                                       Application
                                                                                                                                                                                                                                                                                                                                                                                 Thread                                                                                                  Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Thread
                  Eval            Application
                                 Thread                                                                              Eval              Application
                                                                                                                                      Thread                                                                                                                                                                                                                      Eval             Application
                                                                                                                                                                                                                                                                                                                                                                                  Thread                                                                               Eval               Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         Thread
                 Server             Application
                                  Thread                                                                            Server               Application
                                                                                                                                       Thread                                                                                                                                                                                                                    Server              Application
                                                                                                                                                                                                                                                                                                                                                                                   Thread                                                                             Server                Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Thread
                 Thread             Thread                                                                          Thread               Thread                                                                                                                                                                                                                  Thread              Thread                                                                           Thread                Thread



       Proxy                                                                                             Proxy                                                                                                                                                                                                                                        Proxy                                                                                                  Proxy
      Manager                                                                                           Manager                                                                                                                                                                                                                                      Manager                                                                                                Manager
      Thread              Process Memory                                                                Thread               Process Memory                                                                                                                                                                                                          Thread               Process Memory                                                                    Thread             Process Memory




                                         Application                                                                    Application                                                                                                                                                                                                                                                         Application                                                                    Application
                                          Application                                                                    Application
                                                                                                                        Thread                                                                                                                                                                                                                                                               Application                                                                    Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                           Thread
                                         Thread
                                           Application                                                                    Application
                                                                                                                         Thread                                                                                                                                                                                                                                                             Thread
                                                                                                                                                                                                                                                                                                                                                                                              Application                                                                    Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                            Thread
                           Eval           Thread
                                            Application                                                   Eval             Application
                                                                                                                          Thread                                                                                                                                                                                                                                           Eval              Thread
                                                                                                                                                                                                                                                                                                                                                                                               Application                                                   Eval             Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                             Thread
                                           Thread
                                              Application                                                Server              Application
                                                                                                                           Thread                                                                                                                                                                                                                                                             Thread
                                                                                                                                                                                                                                                                                                                                                                                                 Application                                                Server              Application
                                                                                                                                                                                                                                                                                                                                                                                                                                                                              Thread
                          Server            Thread                                                                           Thread                                                                                                                                                                                                                                       Server               Thread                                                                           Thread
                          Thread              Thread                                                     Thread                                                                                                                                                                                                                                                           Thread                 Thread                                                     Thread


                 Proxy                                                                        Proxy                                                                                                                                                                                                                                                             Proxy                                                                              Proxy
                Manager                                                                      Manager                                                                                                                                                                                                                                                           Manager                                                                            Manager
                Thread                                                                       Thread                Process Memory                                                                                                                                                                                                                              Thread                                                                             Thread              Process Memory
                                   Process Memory                                                                                                                                                                                                                                                                                                                                      Process Memory




                                                                                                                                             Application                                                                                                                                                                                                            Application
                                                                                                                                              Application
                                                                                                                                             Thread                                                                                                                                                                                                                  Application
                                                                                                                                                                                                                                                                                                                                                                    Thread
                                                                                                                                               Application
                                                                                                                                              Thread                                                                                                                                                                                                                  Application
                                                                                                                                                                                                                                                                                                                                                                     Thread
                                                                                                                              Eval              Application
                                                                                                                                               Thread                                                                                                                                                                                                 Eval             Application
                                                                                                                                                                                                                                                                                                                                                                      Thread
                                                                                                                             Server               Application
                                                                                                                                                Thread                                                                                                                                                                                               Server              Application
                                                                                                                                                                                                                                                                                                                                                                       Thread
                                                                                                                             Thread               Thread                                                                                                                                                                                             Thread              Thread



                                                                                                                   Proxy
                                                                                                                  Manager
                                                                                                                  Thread               Process Memory
                                                                                                                                                                                                                                                                                                                                            Proxy
                                                                                                                                                                                                                                                                                                                                           Manager
                                                                                                                                                                                                                                                                                                                                           Thread              Process Memory
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        Individual SMPs
                                                                                           Application                                                                                          Application                                                                                                       Application                                                                                             Application
                                                                                            Application
                                                                                           Thread                                                                                                Application
                                                                                                                                                                                                Thread                                                                                                             Application
                                                                                                                                                                                                                                                                                                                  Thread                                                                                                   Application
                                                                                                                                                                                                                                                                                                                                                                                                                          Thread
                                                                                             Application
                                                                                            Thread                                                                                                Application
                                                                                                                                                                                                 Thread                                                                                                             Application
                                                                                                                                                                                                                                                                                                                   Thread                                                                                                   Application
                                                                                                                                                                                                                                                                                                                                                                                                                           Thread
                                                                            Eval              Application
                                                                                             Thread                                                                             Eval               Application
                                                                                                                                                                                                  Thread                                                                                        Eval                 Application
                                                                                                                                                                                                                                                                                                                    Thread                                                                                 Eval              Application
                                                                                                                                                                                                                                                                                                                                                                                                                            Thread
                                                                           Server               Application
                                                                                              Thread                                                                           Server                Application
                                                                                                                                                                                                   Thread                                                                                      Server                  Application
                                                                                                                                                                                                                                                                                                                     Thread                                                                               Server               Application
                                                                                                                                                                                                                                                                                                                                                                                                                             Thread
                                                                           Thread               Thread                                                                         Thread                Thread                                                                                    Thread                  Thread                                                                             Thread               Thread



                                                                 Proxy                                                                                                Proxy                                                                                                          Proxy                                                                                                     Proxy
                                                                Manager                                                                                              Manager                                                                                                        Manager                                                                                                   Manager
                                                                Thread              Process Memory                                                                   Thread             Process Memory                                                                              Thread               Process Memory                                                                       Thread               Process Memory




                                                                                                     Application                                                                    Application                                                                                                                               Application                                                                      Application
                                                                                                      Application                                                                    Application
                                                                                                                                                                                    Thread                                                                                                                                     Application                                                                      Application
                                                                                                                                                                                                                                                                                                                                                                                                               Thread
                                                                                                     Thread
                                                                                                       Application                                                                    Application
                                                                                                                                                                                     Thread                                                                                                                                   Thread
                                                                                                                                                                                                                                                                                                                                Application                                                                      Application
                                                                                                                                                                                                                                                                                                                                                                                                                Thread
                                                                                     Eval             Thread
                                                                                                        Application                                                   Eval             Application
                                                                                                                                                                                      Thread                                                                                                              Eval                 Thread
                                                                                                                                                                                                                                                                                                                                 Application                                                    Eval              Application
                                                                                                                                                                                                                                                                                                                                                                                                                 Thread




                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          SCIENTIFIC
                                                                                                       Thread
                                                                                                          Application                                                Server              Application
                                                                                                                                                                                       Thread                                                                                                                                   Thread
                                                                                                                                                                                                                                                                                                                                   Application                                                 Server               Application
                                                                                                                                                                                                                                                                                                                                                                                                                  Thread
                                                                                    Server              Thread                                                                           Thread                                                                                                          Server                  Thread                                                                             Thread
                                                                                    Thread                Thread                                                     Thread                                                                                                                              Thread                    Thread                                                      Thread


                                                                           Proxy                                                                            Proxy                                                                                                                              Proxy                                                                                Proxy
                                                                          Manager                                                                          Manager                                                                                                                            Manager                                                                              Manager
                                                                          Thread                                                                           Thread              Process Memory                                                                                                 Thread                                                                               Thread                Process Memory
                                                                                              Process Memory                                                                                                                                                                                                         Process Memory

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          COMPUTING ASSOCIATES
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             Forward Thinking
Objectives

 Investigate API design & implementation issues for
 parallel/distributed systems with hierarchical memory
 Provide code portability among parallel architectures
 including SMPs, distributed memory machines, & clusters
 Achieve high performance through
    Language-level processing and optimization
    Exploitation of native compilers and libraries
    Intelligent dynamic process/data mapping
 Achieve ease of use by providing a uniform logical
 software architecture across the entire hierarchy



                                                     SCIENTIFIC
                                                     COMPUTING ASSOCIATES
                                                        Forward Thinking
Initial Target Hardware Architecture
  Nodes: Shared Memory Multiprocessors (SMPs)
         (either PCs or workstations)
  Interconnect: Fast Ethernet or a switch fabric
          (e.g., Myrinet, Giganet, Quadrics)
  Operating Systems: Windows NT, Linux, Vendor UNIX




                                                   SCIENTIFIC
                                                   COMPUTING ASSOCIATES
                                                      Forward Thinking
Possible Approaches for Linda System Software
  Network Linda
    Treat each node of each SMP as a separate, full-fledged
    “machine” on the network
  Simple Hybrid System
    Paradise server for VSM
    Native SMP or Original Linda on each SMP
  Hierarchical System
    One VSM server per SMP, with all linked together to manage a
    single VSM
    Intelligent run-time system to exploit locality through redirection,
    caching and data relocation.



                                                             SCIENTIFIC
                                                             COMPUTING ASSOCIATES
                                                                Forward Thinking
Hierarchical Linda Software Architecture
    Linda Process
                                  Application
                                   Application
                                  Thread
                                    Application
                                   Thread
                                     Application
                                    Thread
               In Proxy                Application
                                     Thread
               Thread
           In Proxy                    Thread
           Thread
                                              Thread
      Thread                                  Create
      Create
                                            Eval
                     Process Memory        Server
        Proxy
       Manager                             Thread
       Thread


                                                 SCIENTIFIC
                                                 COMPUTING ASSOCIATES
                                                     Forward Thinking
System Design I             Proxy Manager
                                                        Receive
                                                        Message
Hierarchical Linda System       Store tuple in local   out     eval        Out local eval
                                 shared memory                             request tuple
    Threads of Control                                        in
                                                       Spawn local
                                                       service TOC Main thread continues
                                No
                                      Waiters
                                        ?                          New handler thread
Eval Server
                                          Yes

                                     Wake
                                   appropriate                Beginning
                                     waiters                 of new TOC
    in global eval                                                                   Wake up
     count array
                                                              Check for
                                                               match
                                                                                      Sleep

    Update global                                                          No        Register
   eval count array                                            Match
                                                                ?                    as waiter

                                                                     Yes
                                                             Send tuple to
                                                                                       Exit
   out global eval                                           remote TOC
    count array


     Spawn local
      user TOC                                                                   SCIENTIFIC
                                                                                 COMPUTING ASSOCIATES
                                                                                      Forward Thinking
System Design II

Linda Code Segments Embedded in User TOCs
                        in/rd Processing                    eval Processing            out Processing
    Part of current                                            Part of current        Part of current
      user TOC                                                   user TOC               user TOC


                                                                rd global eval                               Send out request
         Local     No                  Send in/rd request        count array               Local No
          Set                                                                               Set                and tuple to
                                       to remote manager                                                     remote manager
           ?                                                                                 ?
        Yes                Wake up                             Select processor               Yes
                                       Receive tuple from     for eval and update
       Check for                        remote handler         global count array   Store tuple in local
        match                                                                        shared memory
                            Sleep
                                                                   Out eval
                                                                request tuple
                   No      Register                              for selected                       No
         Match                                                                            Waiters
          ?                as waiter                              processor
                                                                                            ?
            Yes                                                                               Yes                Return
                                                                   Return
  Get tuple from local                                                                   Wake
   shared memory                                                                       appropriate
                                                                                         waiters
                                            Return




                                                                                                           SCIENTIFIC
                                                                                                           COMPUTING ASSOCIATES
                                                                                                               Forward Thinking

				
DOCUMENT INFO
Shared By:
Tags: A-Hie, rarch
Stats:
views:7
posted:11/30/2009
language:English
pages:9
Description: A-Hierarchical-Environment-for-Virtual-Supercomputing