Docstoc

User Interface

Document Sample
User Interface Powered By Docstoc
					 Infrastructure for
Data Fitting Service
                Distributed Computing Environment

User/Client              User   User   User    User   User




ServiceServer                                          Cluster
Management                        Service Server
                                   Master Node


WorkingServer
Software Infrastructure
ServiceServer Node
                     Service                  User
                     Controlle
                        r
                                             Input


                                            Interface
Working Nodes        Services    Services

                     Services    Services
                         Users
• The work should be done ( assume that the
  service exists in the cluster)
• May provide some information, such as the
  required resources, to perform the work


     User
               Request                    Reply



     Cluster
                         Service Server
                         Master Node(s)
       Client
Connect to service server



 Submit service request



    Get service reply




     Update the UI
                 Developers

 ServiceServer    Service            User



                    MessageQueueSy
                    s

JobQueueSys



            WorkingClient




WorkingServer
Working Nodes


Waiting for request




Accept the request



Perform the work




 Return the result
  Parallel Pattern 1: Fitting work parallelized



                          Server / Master




   Node1             Node2                        NodeN
      Multiplexors
      Fitting work       Multiplexors
                         Fitting work               Fitting work
                                                    Multiplexors
      1                  2                          n

Working Nodes
Parallel Pattern 2: model parallelized


                                 Request




                           Master Node
                                Fitting work
                                Multiplexors




     Node1              Node2                  NodeN
           Model
        Multiplexors          Model
                           Multiplexors             Model
                                                 Multiplexors


 Slave Nodes
               Implementation: Mapping
• Mapping (works for pattern 1 and 2)
   – Assign working unit [ i ] to working node [ j ]
       Master Node (reduce function)      Working Nodes (map function)


       Accept service request


                               Yes
           Finish Service                 Accept working unit request


    Request working unit service        Perform requested working unit

         Finish working unit
                                            Finish the working unit
                Implementation: Mapping
• Advantage
  –   Dynamical loading balance
  –   Efficiency for heterogeneous cluster
  –   Fault tolerance
  –   Easy to use. Working unit can be sequential or parallel
      programs
• Disadvantage
  – The efficiency depends on the working unit partition of the
    service
  – May be less efficient when compared with dedicated parallel
    software.
                 High level Mapping
                    A list of mappings
• Sequential Mapping (dependent mappings)
   – The mapping only can start when its previous mapping has
     finished.
• Parallel Mapping (independent mappings)
   – The mappings can start simultaneously
• Session Mapping
   – The list of sequential and/or parallel mappings for one
     session
          Job Queue
                 JobQueue




Waiting Queue        Scheduled Queues




Finished Queue

                      Running Queue
            Message Queue
                 Result Message




                 Message Queue



                                      Registered Listeners



Listener1
             Listener2                   Listenern
                          Listener…
            XML File for job request
<?xml version='1.0' encoding='UTF-8'?>
<session version='2.0.1' type='6' user='wwchen’
    email='wwchen@nist.gov' priority='0' >
    <group name='group1'>
        <job name='job1' priority='4' cnt=‘4' >
          <dataset>
              ……..
           </dataset>
          <task>
              ……..
         </task>
        </job>
    </group>
</session>
             XML file for dataset
<DataSet>
<D Name="Data1" LocationType="URL" FileName="gauss1.dat">
http://www.physics.utah.edu/~kevin/NIST/
</D>

<D Name="Data2" LocationType="URL" FileName="gauss2.dat">
http://www.physics.utah.edu/~kevin/NIST/
</D>

<D Name="Data3" LocationType="URL“ FileName="gauss3.dat">
http://www.physics.utah.edu/~kevin/NIST/
</D>
</DataSet>
                XML file for task
<DataSet>
  <D Name="Data1" LocationType="URL" FileName="gauss1.dat">
  http://www.physics.utah.edu/~kevin/NIST/
</D>

<D Name="Data2" LocationType="URL" FileName="gauss2.dat">
http://www.physics.utah.edu/~kevin/NIST/
</D>

<D Name="Data3" LocationType="URL“ FileName="gauss3.dat">
http://www.physics.utah.edu/~kevin/NIST/
</D>
</DataSet>
                   XML file for task
<task user='wwchen' type='0' cmd='longwork0.py' id='1801'
   gid='100' sid='2000' priority='10'>
   <bufsize value='3000'/>
   <env name='HOME' value='~/home'/>
   <cwd value='/home/wwchen/danse/'/>
   <stdin name='test1.in'/>
   <input> <! -- This includes information for the program! -->
        <Multiplexor > … </Multiplexor >
   </input>
   <stdout name='test1.out' back='0'/>
   <stderr name='test1.err' back='1'/>
   <prefunc name='prefunc'/>
   <postfunc name='prefunc'/>
</task>
               XML for multiplexor(1)
<Multiplexor>
 <Model Name="M1">
    <Theory class="Gauss"> This is a gaussian function with one peak
         <p Name="Background" Value="3.0" />
         <p Name="Intensity" Value="100" />
         <p Name="Center" Value="3.0" />
         <p Name="sigma" Value="5.0" />
    </Theory>

    <p Name="Background" Low="1.00" High="5.0" />
    <p Name="Intensity" Low="10" High="200" />
    <p Name="sigma" Low="1.00" High="10.0" />

   <Data Name="gauss1.dat“/>
 </Model>
……
               XML for multiplexor(2)
  <Model Name="M2">
       <Theory class="Gauss"> This is a gaussian function with one peak
          <p Name="Background" Value="3.0" />
          <p Name="Center" Value="6.0" />
          <p Name="sigma" Value="5.0" />
          <p Name="Intensity" Value="100" />
     </Theory>

      <p Name="Center" Low="1.00" High="10.0" />
      <p Name="sigma" Low="1.00" High="10.0" />
      <Data Name="gauss2.dat"> </Data> </Model>

   <Calculated>
          <C Target= "M2.Intensity"> 5*M1.Intensity </C>
          <C Target="M2.Background">
                  5*M1.Background + M1.Intensity/100</C>
          <C Target= "M3.Intensity"> 3*M1.Intensity </C>
   </Calculated>
</Multiplexor>
   Full Service Server
 ServiceManagement

 ClusterManagement
  JobQueueSystem
MessageQueueSystem    Controller
 DataFetchingSystem
   ArchiveSystem

   LoggingSystem
   Working Servers

ServiceManageSystem
 TaskManagement

  RestartSystem …     Controller
DataFetchingSystem
   LoggingSystem
           Fully Distributed Services ?
Client                           User



Services
                          Service Register


         Job Queue                                  Message Queue


     Cluster Management
                                                    Task Management

           Service Management                 Data Fetching

                     Archive              Logging
                               Shared Files
Security & Authentication ?

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:12/30/2011
language:
pages:24