Docstoc

job job job job

Document Sample
job job job job Powered By Docstoc
					             Keldysh Institute of Applied Mathematics
                  Russian Academy of Sciences




Resource Manager for Grid with global
    job queue and with planning
       based on local schedules
     V.N.Kovalenko, E.I.Kovalenko, D.A.Koryagin,
        E.Z.Ljubimskii, A.V.Orlov, E.V.Huhlaev
    {kvn,kei,koryagin,ljubimsk,ao,huh}@keldysh.ru
                                                        1
Job submitting   Job submitting
in Globus        by means of
system           Broker
                            Broker




                                     2
           Resource Brokers
   GRID Resource Broker (GRB) – HPC lab,
    University of Lecce, Italy and CACR,
    California Institute of Technology.
    http://sara.unile.It/grb/
   EZ-Grid - Department of Computer Science,
    University of Houston.
    http: //www.cs.uh.edu/~ ezgrid/
 MetaDispatcher – Keldysh Institute of
  Applied Mathematics, Moscow

                                                3
Job submitting   Job submitting
in Globus        by means of
system           Broker
                            Broker




                                     4
Architecture of MetaDispatcher




                                 5
 Problem of scheduling
  The problem of scheduling is decided
on two sets: 1) the set of jobs and 2)
the set of computing elements.
  Scheduling results:
- The dispatch time for each job
- The place, where the job should be
directed and executed


                                         6
Two management levels - local and global, each having
own objects: job, queue, and management system -
Local Resource Monitor (LRM) and MetaDispatcher.




 Global
 queue                                            Config.



  job     MetaDispatcher
  job
  job                                       Config. file
                                   Local
  job                              queue

                                            LRM


                                           Local level
                    Global level
                                                            7
Question 1: In What Order Should
   the Global Jobs Be Served?
                   The order, in which the scheduler serves the job
      new job
                    queue, should differ from FIFO.
                   User should have available the management
                    facilities for placing his job at any position in the
                    global queue.
job
job             To achieve that:
job
                   Limited budget is allocated to each user.
job
job                Within the budget limits user prices his jobs.
job
                   Function GP evaluates global priority of the job:
                      GP=GP(price, required resources, run time )
                                                                            8
  Question 2: When Forward a Job to a
                  Target Computing Element?
        new job
                          If destination point of a job is determined at
                    job
                          the moment, when it comes in to a global
 job
 job
                    job
                    job
                          queue, and the job is immediately routed to
 job                job   a local queue…
                          it may be delayed there because of the local
 job                      job arrival. At the same time resources of
 job
 job
                          other computing elements may become free
 job                      and idle.


The conclusion:
It is more reasonablly to store global jobs in global queue
as long as possible, best of all up to the moment of start.
                                                                           9
Question 3:       To Which Computing
     Elements a Job Should Be Passed?

The scheduling model of computing
installation:
  A set of resources


Resource description:
  Static attributes: (OS type, CPU time, memory volume)
  Dynamic attributes: free/busy, resource amount

                                                          10
     Resource Release Time
Resource                          Busy resources have an
                                  additional attribute – release
    Running job                   time estimated from the
                                  request of a running job.
                                  Being aware of the release
Running job                       time, the scheduler is able to
                                  plan the future usage of the
                                  busy resource.
 Running job           Time


However the scheduler must have a guarantee,
that the planned global job will really start and
will not stay waiting in a local queue.
                                                                   11
Question 4: How should the interaction of the
Question 4: How the Interaction of the Global
  global scheduler and local resource Should
Scheduler and Local Resource Monitormonitor Be
                  Organized?
                be organized?
   two
If + jobs, local and global, ask for free resources, which one
should be preferred?
   Autonomy of computing element:
   Each computing element of the Grid belongs to a certain owner that
   could be able to restrict access for external jobs completely or partly.

 If global and local jobs make demands for the same resources, their
 priorities are compared. For this purpose each computing element i
 determines the function LPi() that calculates the local priority of a
 global job. This function depends on job’s price, consumable
 resources and run time:
          LPi = LPi (price, consumable resources, run time)
                                                                              12
  Question 4:         How the Interaction of the Global
    Scheduler and Local Resource Monitor Should Be
                     Organized?
                                                       Global queue

+
                                                          jobG
      Resource
                                   PG= LP(jobG)
        Running job
                       PG
             PG<PL                       Local queue
                        PL

     Running job                            jobL
                                                         Time



The global scheduler should distribute its jobs so that the global jobs
would not withhold the start of any more "expensive” local jobs.
                                                                          13
                         Schedule
    Resource

        Running job
                                              priority3
   Running job
                                  priority2
                      priority1                     priority4
    Running job                                                  Future
                                                                 Time
The local schedule is the plan of resource occupation by local jobs
for some period of time in the future.
     Local schedule:
     For each local job
         {priority, assigned resources, occupation and release time}
                                                                          14
    The local schedule is drawn up by the special
agents of the global scheduler. Such agents,
working on each computing installation, arrange the
schedule in precise conformity with scheduling
strategy and configuration parameters of the local
monitor.
    The actual state of all local schedules is
delivered to the information base of the global
scheduler, and, thus, it has available the
information about the usage plan of all virtual
organization resources.
    On the basis of this aggregate schedule the
scheduler can make up the layout of global jobs
allocation to resources.
                                                      15
         Program architecture of scheduling

                 Data Base

                                                Agent

Global
queue
                                                LRM
                                             Agent

 job          Scheduler
 job                                         LRM
 job                                 Agent
 job

                             Queue

                                      LRM


                                                        16
 The global scheduler implementing certain
  scheduling strategy make up the global schedule.

 The information base resides adjacently with the
  scheduler and stores aggregate schedule. For data
  management the distributed system like Spitfire of
  Datagrid project with relational data base as a core
  is considered.

 The local agents of the scheduler works on each
  computing element. Interacting with the local
  resource monitor, the agent arranges a local
  schedule of this computing element and transfers
  updates to the global scheduler. Proposed
  implementation is based on Maui scheduler.
                                                         17
Future directions:

   Backfill algorithm implementation at the
    global level to avoid blocking of the
    jobs.

   Advanced resource reservation for
    distributed multiprocessor jobs.

   Economical model of virtual
    organization as applied to scheduling.

                                               18

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:4
posted:10/3/2012
language:Unknown
pages:18