LinSched The Linux Scheduler Simulator∗ by nni49141


									                            LinSched: The Linux Scheduler Simulator∗
        John M. Calandrino                                Dan P. Baumberger, Tong Li, Jessica C. Young, and Scott Hahn
  Department of Computer Science                                           Systems Technology Lab
  The University of North Carolina                                             Intel Corporation
      Chapel Hill, NC 27599                                                  Hillsboro, OR 97124

Abstract                                                                  Linux. For these researchers and other Linux developers, a
                                                                          good understanding of this new scheduler is crucial.
The Linux kernel 2.6.23 scheduler includes substantial
                                                                             While the new Linux scheduler is well-documented, it
changes that may entice researchers with no prior interest
                                                                          is typically difficult to fully understand the behavior of
in Linux to attempt to understand or modify its behavior.
                                                                          a Linux subsystem without direct experimentation, espe-
Often, this is no easy task, particularly for someone
                                                                          cially if planning to modify it in further development or re-
new to Linux kernel development. Virtual machines and
                                                                          search. This would typically involve observing the behav-
hardware simulators can help make the task easier;
                                                                          ior of the scheduler in a variety of synthetic test cases, fol-
however, they introduce their own problems. Motivated
                                                                          lowed by an attempt to change the scheduler to implement
by these observations, we present the LinSched tool to
                                                                          a new scheduling policy, where it is determined how those
host a Linux scheduler at a high simulation speed in an
                                                                          changes alter its behavior. These tasks can be tedious or
isolated environment, within which its behavior can be
                                                                          frustrating when the scheduler is not yet well understood,
observed on a variety of different platforms independently
                                                                          and even sometimes when it is, for several reasons.
of other Linux subsystems. The tool runs as a user-space
process, so bugs within the scheduling code crash a single                   First, tracing the scheduler behavior can be difficult since
process instead of the entire system, reducing development                the presence of the tracing mechanism itself may introduce
time substantially. Experiments show that scheduling                      bugs or change the behavior of the scheduler. In addition,
behavior within LinSched and the Linux kernel are nearly                  test cases that stress the system (e.g., four FIFO real-time
identical—this, combined with our personal experiences,                   tasks running on a four-processor machine, which could
suggest that the tool can be highly useful both for gaining               prevent all other tasks from executing indefinitely) may
an understanding of the scheduler and for the rapid                       cause the system to become unresponsive or crash, mak-
prototyping of new Linux scheduling policies.                             ing it difficult to determine the types of workloads that
                                                                          the scheduler can support. Second, bugs that are typi-
                                                                          cally straightforward to diagnose and fix in user-space pro-
1 Introduction                                                            cesses can be highly problematic when they occur within
A recent surge of interest in the scheduler of the Linux                  the scheduler. For example, an invalid memory reference
kernel led to significant changes in kernel version 2.6.23.                in the scheduler will almost certainly crash the system and
The new scheduler introduces a scheduling policy (the                     require a reboot. Debugging information is limited, and
Completely Fair Scheduler) that attempts to provide fair-                 not easily obtained or understood by a new developer. This
ness properties that may be of interest to some re-                       combination of long crash-reboot cycles and limited debug-
searchers. Additionally, the scheduler has been redesigned                ging information can result in a time-consuming develop-
as a framework that organizes scheduling policies into or-                ment process. A bug that might take ten minutes to diag-
dered classes—when a scheduling decision needs to be                      nose and fix in a user-space program could take hours to fix
made, policies are consulted in turn based on the order-                  when it occurs within the Linux kernel. These issues are
ing. This redesign can make it considerably easier to im-                 further complicated by the fact that changes made to the
plement scheduling policies that subsume the default poli-                scheduler may violate implicit assumptions that are made
cies (classes) of the Linux scheduler, by creating a new                  within other Linux subsystems, which can result in dead-
scheduling class that is consulted prior to any of the de-                lock or a system crash. While it is important to ultimately
fault classes. These changes may encourage researchers                    handle these issues, it would be preferable to debug the
who typically do not attempt kernel development to imple-                 scheduler itself first, rather than diagnosing scheduler bugs
ment and empirically evaluate scheduling policies within                  and integration-related issues concurrently. Clearly, these
   ∗ The first author was supported by grants from Intel and IBM Corps.,   issues can be problematic, especially to new Linux devel-
by NSF grants CNS 0408996, CCF 0541056, and CNS 0615197 and by            opers, and it would be helpful to have tools that could ease
ARO grant W911NF-06-1-0425.                                               their burden. The presence of such tools may also increase
interest in Linux scheduler development, which could help         oping schedulers for large-scale platforms that do not yet
to improve the overall quality of the scheduler itself.           exist. Multicore platforms are now the standard in desktop
   Virtual machines and hardware simulators may help to           and server systems, and the core counts of such platforms
ease the burdens outlined above; however, they introduce          are increasing—indeed, Intel has built test chips with 80
their own problems. Both tools remove the need to reboot          cores and is planning on increasing the number of on-die
Linux on physical hardware, but booting Linux within a            cores in released chips over the next few years [4]. A tool
virtual machine can take just as long, and booting within an      such as LinSched may provide us with initial insight into
architecture simulator can take several orders of magnitude       the scalability issues that will arise on these platforms, with
longer. Some architecture simulators allow checkpoints to         respect to developing effective scheduling policies. Virtual
be taken so that the entire boot process does not have to         machines are lacking in this area, as they typically cannot
occur every time a simulation is run. While this would as-        support a large number of CPUs, and architecture simula-
sist the debugging process, a full reboot would be required       tors are painfully slow for large core counts.
whenever the scheduling code is changed (assumedly fre-
                                                                  Related work. User Mode Linux (UML) [8] is a port of
quently). Further, tracing and debugging facilities within
                                                                  the Linux kernel that runs in user space on top of another
these environments are better than real hardware, but still
                                                                  Linux kernel. A similar project, Plex86 [6], creates a vir-
limited—this is especially true for multiprocessors.
                                                                  tual machine that is optimized to host a Linux guest OS. Fi-
   In this paper, we present a tool called LinSched, or
                                                                  nally, Bochs [1] is a portable open source IA-32 emulator
the Linux Scheduler Simulator. LinSched hosts a Linux
                                                                  capable of running many operating systems. All of these
scheduler in an isolated environment, within which its be-
                                                                  tools simulate the entire operating system, and are limited
havior can be observed for a variety of different platforms       to a small number of CPUs. Full-system architecture sim-
and workloads independently of other Linux subsystems.
                                                                  ulators such as Simics [5] exist that can simulate platforms
The tool runs as a user-space process, so bugs within the
                                                                  with large CPU counts, but as stated earlier, they are often
scheduling code simply result in the termination of the Lin-      slow even in their fastest run modes.
Sched process, immediately after which the scheduling
code can be modified and LinSched restarted. LinSched              Contributions. In this paper, we present LinSched,
can also be attached to the GNU Debugger (GDB) [7]                which has the following benefits over existing tools.
and debugged with the same ease as any other user-space              • Hosts the scheduler as an isolated subsystem in user
process. Since LinSched only simulates the scheduling                   space, for easier debugging of scheduler code. This
subsystem, it can produce scheduler traces for a variety                allows for the rapid prototyping and initial evaluation
of workloads in considerably less time than would be re-                of new scheduling policies. Porting code back into
quired within the Linux kernel. For example, a trace of the             the Linux kernel for further testing and evaluation is
scheduling decisions made for a large workload running on               relatively straightforward due to a high degree of code
a four-processor machine over one minute of execution can               sharing between LinSched and Linux.
be produced in about one second. Additionally, LinSched              • Allows scheduler behavior to be easily and quickly
has negligible startup time as compared with tools such as              observed for a variety of workloads and platforms.
virtual machines and architecture simulators.                           This includes platforms that substantially differ from
   The primary goal of LinSched is to provide a tool for                the platform on which LinSched is run, and work-
observing and modifying the behavior of the Linux sched-                loads that severely stress the platform being simulated
uler, and prototyping new Linux scheduling policies, in a               (and might cause a real system to freeze).
way that may be easier or less tedious to many developers            • Eliminates the need for a separate tracing mechanism
than using Linux itself. This is especially relevant consid-            that may alter scheduler behavior or introduce bugs.
ering the significant recent changes to the Linux scheduler        Experiments presented in Sec. 3.2 show that the behaviors
discussed earlier. Scheduling behavior can be observed,           of LinSched and the Linux scheduler are nearly identical.
and ideas for policies initially tested, without the need for a   An early version of LinSched has already been used to add
full implementation within Linux. This allows researchers         soft real-time support to a Linux scheduler running on an
to test many policies and select only the most promis-            asymmetric multicore platform [3]—we believe that using
ing ones to implement and test on real hardware. Since            LinSched cut overall development time substantially, and
the Linux scheduler is hosted in LinSched with minimal            porting the resulting code from LinSched into the Linux
changes, and macros are used to “ignore” or replace code          kernel was relatively painless. LinSched is also currently
rather than delete it, porting changes back into the Linux        being used in an internal Intel cache-simulation project.
kernel is a relatively straightforward process. LinSched is          The rest of this paper is organized as follows. Sec. 2
general enough so that a tool such as diff can be used to         discusses the design and architecture of LinSched. Sec. 3
update LinSched when new kernel versions are released,            presents an evaluation of LinSched that demonstrates its
after which it can be propagated to developers for their use.     ease of use and the strong correlation of its behavior with
   Additionally, LinSched may be of some use in devel-            the Linux scheduler. We conclude in Sec. 4.
2 LinSched Overview                                            Automation. To facilitate rapid evaluation of multiple
                                                               scheduling policies, LinSched programs can be parameter-
We discuss the design of LinSched in Sec-
                                                               ized so that many different workloads and platforms can be
tion 2.1 and present its high-level architecture in
                                                               tested in rapid succession through the use of batch scripts.
Section 2.2.   The most recent version of Lin-
Sched will soon be available for download at                   Scalability. Future platforms will likely consist of many∼jmc/linsched.                           more processing cores than the platforms of today. Thus,
                                                               LinSched is able to run larger simulations involving tens
2.1 Design and Features                                        or possibly hundreds of cores relatively quickly.
LinSched has one primary design goal: to provide a tool        Topology specification. LinSched currently supports a
for easily observing and modifying the behavior of the         “flat” topology where all CPUs are in the same scheduling
Linux scheduler, and for prototyping new Linux schedul-        domain, and can be easily extended to simulate hierarchical
ing policies. To achieve this goal, detailed modeling of the   topologies as they continue to become more commonplace.
platform and other Linux subsystems is not required. Thus,
LinSched models only the Linux scheduler and the plat-         2.2 Architecture
form topology that it requires. We now list the features of                                                   Stimuli        Simulation
LinSched that support our main design goal.                    LinSched consists of                                            Script

                                                               three main components,
                                                                                                     Other Simulation
Scheduling policy support. LinSched is able to ac-             as illustrated in Fig. 1.                   Tool
                                                                                                                           Script Interpreter

commodate many Linux scheduling policies.         In           The simulation engine
fact, it can model virtually any policy that calls a           (Sec. 2.2.1) presents an                              Engine

scheduler tick() function at every (periodic) timer            API that can be used

interrupt and a schedule() function whenever a                 to initialize and control                            Module

scheduling decision needs to be made.                          a simulation, and calls
                                                                                                  Figure 1: The LinSched
                                                               the appropriate sched-
Task specification. Linux allows tasks to be classified          uler functions to simu-
into categories, such as SCHED FIFO (a FIFO real-time          late the Linux scheduling policies. The environment mod-
task) or SCHED BATCH (a compute-intensive batch task).         ule (Sec. 2.2.2) provides an abstraction of the Linux ker-
These categories, along with some notion of priority, de-      nel in terms of code dependencies such as functions and
termine both the policy (class) that is used to schedule       macros (most of which are supplied directly by includ-
each task, and when each task is scheduled by that policy.     ing Linux source and header files), and is responsible for
Creating different types of tasks with varying priorities is   presenting an appropriate platform topology to the simula-
straightforward in LinSched. Additionally, we can specify      tion engine. Finally, stimuli (Sec. 2.2.3) are provided by
other task characteristics, such as how long a task runs, or   a scripting interpreter or another tool that uses the API of
when it suspends (e.g., to simulate I/O).                      the simulation engine to create tasks and start a simulation.
                                                               This component can be parameterized, so that batch scripts
Debugging. LinSched provides a friendly debugging en-          can be used to run simulations where many different types
vironment, which we achieve by hosting the Linux sched-        of workloads and platforms are investigated.
uler code in a user-space process. Thus, bugs in the code      2.2.1 Simulation Engine
will cause only the LinSched process to terminate, and it      The simulation engine provides an API that can be used by
can be easily attached to GDB for diagnosis.                   scripting tools to run simulations, and calls the appropriate
                                                               functions to simulate the Linux scheduling policies. Ad-
Source compatibility. The scheduling code within Lin-          ditionally, the API methods call functions that create tasks
Sched is taken directly from the Linux scheduler with min-     and initialize the kernel environment, many of which are
imal modifications to remove or replace code for handling       supplied by the environment module.
interactions with other subsystems such as memory man-            Much of the code within both the simulation engine and
agement. Macros are used so that the resulting code looks      the environment module is simply a small subset of the
identical to the Linux scheduler when a simulator-specific      Linux kernel source files, with modifications to provide
flag within the code is cleared. Thus, porting scheduling       support for certain features when necessary. A simulation-
code between Linux and LinSched is relatively straight-        specific flag is used liberally throughout the code, and
forward, and translation errors are minimized.                 when set during compilation, the source code changes re-
                                                               quired by LinSched take place—when the flag is cleared,
Data collection. Data and trace information is collected       all source files are identical to the original Linux source
by adding code to perform bookkeeping tasks. LinSched          code. Such a flag should make porting code between Lin-
only simulates the scheduler, instead of actually executing    Sched and Linux considerably easier. Thus, the task- and
tasks, so this code does not alter scheduler behavior.
runqueue-related structures that are used within the Linux    of the current CPU) and current (a pointer to the task
scheduler are used almost verbatim within LinSched.           currently scheduled on a CPU) work correctly.
   The API provided by the simulation engine includes the        Since LinSched is a user process, the scheduler can be
following functions, among others.                            traced by outputting data to a file or the screen. Such data
                                                              output within task callbacks can easily provide us with a
  • linsched init(): Initializes the scheduling sub-
                                                              clear view of LinSched scheduling behavior.
    system and environment.
  • linsched default callback(): An empty
    callback function. Task callbacks are performed           2.2.2 Environment Module
    whenever a task is scheduled, to determine task be-       The environment module provides an abstraction of the
    havior or perform bookkeeping. (An empty callback         Linux kernel by satisfying the code dependencies of the
    simulates a task executing an infinite loop or compute-    simulation engine and presenting an appropriate platform
    intensive code for the duration of the simulation.)       topology. As stated earlier, code dependencies are typi-
  • linsched create normal task(): Creates a                  cally satisfied by including a subset of the Linux kernel
    “normal” task, and requires a nice value (indicating      source files, augmented with a simulation-specific flag so
    priority) and a task callback to be provided.             that the files remain unchanged when that flag is cleared.
                                                              This is particularly important when code must be changed
  • linsched create RTfifo task(): Creates a
                                                              to emulate functionality that would be difficult or impossi-
    FIFO real-time task, and also requires both a task pri-
                                                              ble to support directly, e.g., the per cpu areas provided by
    ority and callback.
                                                              Linux. In Linux, the platform topology would be acquired
  • linsched change cpu(): Changes the CPU cur-               and provided to the scheduler during the boot process—
    rently being simulated.                                   in our case, this topology is generated during LinSched
  • linsched run sim():             Runs the simulation       initialization based on the number of CPUs that are speci-
    for a specified number of timer ticks.               The   fied in a configuration file (though this could be parameter-
    scheduler tick() function of the Linux sched-             ized for batch scripts). Currently, we support a “flat” topol-
    uler is called for every CPU at each tick, and            ogy where all CPUs are in the same scheduling domain—
    the schedule() function may be called to make             however, the environment module can be extended, and
    scheduling decisions according to the currently imple-    configuration file modified, to simulate other topologies.
    mented scheduling policies.                               While the simulated CPUs are symmetric, it is possible
                                                              to use task callbacks to simulate certain types of proces-
An example showing how these functions can be used to
                                                              sor asymmetry, so that tasks perform differently on differ-
run a simulation is provided in Sec. 3.1.
                                                              ent CPUs. Such callbacks can easily reference a globally-
   The core of the simulation engine is arguably the
                                                              visible data structure containing platform information.
linsched run sim() function. This function em-
ulates the periodic timer interrupts that are gen-
erated for every CPU. These timer interrupts call             2.2.3 Stimuli
the scheduler tick() function, which determines               Stimuli for the simulation are provided by a scripting inter-
if a scheduling decision needs to be made—if so,              preter or another tool that uses the API of the simulation
schedule() is called for that CPU. In LinSched,               engine to run a simulation. As LinSched does not actu-
timer interrupts are emulated by a loop in which              ally execute tasks, workloads are specified in the form of
scheduler tick() is called for each CPU in a ran-             task sets using the API. These tasks execute callback func-
dom order. This models real systems more closely because      tions when scheduled, which determine how the tasks be-
the arrival order of timer interrupts is system-dependent,    have within the scheduler and environment. For example, a
and in many cases, is essentially random for our pur-         task callback may be implemented so that the task behaves
poses (e.g., if interrupts depend on the order in which       differently on CPUs with different capabilities or cache lay-
CPUs initialize their local (APIC) timers). Note, how-        outs, or on a CPU on which it has been scheduled before
ever, that if a more deterministic ordering is desired,       (as it might be cache “warm” or ”hot” on that CPU). An
then the linsched run sim() function can be eas-              empty callback function would be equivalent to a task that
ily modified to support this—for example, CPU 0 could          executes an infinite loop or some other compute-intensive
be responsible for distributing an interprocessor inter-      code that does not complete during the simulation. An ex-
rupt (IPI) to all other CPUs, and thus may always call        ample showing how to create a simple, non-parameterized
scheduler tick() before any other CPU. We simulate            program that provides the stimuli for a LinSched simula-
the call to scheduler tick() from a particular CPU            tion is provided in Sec. 3.1. Note that stimuli could also
by changing the processor “context” in which the function     be provided through the use of the simulation engine API
is called (with linsched change cpu()), so that calls         within a larger simulation tool that simulates other Linux
to macros such as smp processor id() (to get the ID           subsystems in addition to the scheduler.
3 Evaluation                                                                 /* Include necessary header file to run LinSched simulations. */
                                                                           1 #include "linsched.h"
In this section, we present an evaluation of LinSched, in-
cluding both an example to demonstrate its ease of use, and                2 int main(int argc, char **argv)
                                                                           3 {
experiments to show how well its behavior correlates with                        /* Initialize simulator. */
the Linux scheduler. Sec. 3.1 presents an example program                  4     linsched_init();

that uses the simulation engine API to run a LinSched sim-                          /* Create 3 normal tasks with nice value 0. */
ulation, and Sec. 3.2 presents experimental results.                       5        linsched_create_normal_task(&ls_annc_cb, 0);
                                                                           6        linsched_create_normal_task(&ls_annc_cb, 0);
                                                                           7        linsched_create_normal_task(&ls_annc_cb, 0);
3.1 Running a LinSched Simulation                                                   /* Create one normal task (nice value -5), batch task, and FIFO task. */
                                                                           8        linsched_create_normal_task(&ls_annc_cb, -5);
Fig. 2 shows a sample C program that uses the simulation                   9        linsched_create_batch_task(&ls_annc_cb, 1);
engine API to create a workload and run a LinSched sim-                   10        linsched_create_RTfifo_task(&ls_annc_cb, 90);
ulation. The workload consists of seven tasks: three “nor-                          /* Run simulation for LINSCHED TICKS ticks (in linsched.h). */
mal” tasks with a nice value of zero, one normal task with                11        linsched_run_sim(LINSCHED_TICKS);
a nice value of -5, one batch task with a nice value of 1,                          /* Create round-robin task. */
one real-time FIFO task with priority 90, and one round-                  12        linsched_create_RTrr_task(&ls_annc_cb, 80);
robin real-time task with priority 80.∗ Each task is cre-                           /* Run simulation for LINSCHED TICKS additional ticks. */
ated by a function with two arguments: a pointer to a call-               13        linsched_run_sim(LINSCHED_TICKS);
back function and the task nice value or priority. The func-              14        return 0;
tion ls annc cb() is a callback that displays a message                   15 }
whenever the task is scheduled on a CPU. This callback                           Figure 2: A sample LinSched simulation program.
allows us to generate a trace of scheduling behavior that
can be parsed and analyzed—in experiments discussed in                    representing the following task sets, each of which was run
Sec. 3.2, we do exactly that when determining how well                    for one minute (simulated in LinSched).
LinSched behavior correlates with the Linux scheduler.                    (a) All normal tasks with a nice value of zero.
   After task creation, we run the simulation for 2 ∗                     (b) All normal tasks—tasks 1 through 40 have nice val-
LINSCHED TICKS ticks. Note that it is easy to create                           ues from -20 to 19, respectively. (Lower nice values
new tasks that begin execution at some time after the sim-                     indicate higher priority.)
ulation has begun. In this program, the round-robin task is               (c) Half batch tasks and half normal tasks. Tasks 1-10 and
not introduced into the system until the simulation is 50%                     11-20 are batch tasks with nice values of 0 and 10,
complete, since we run the simulation for equal numbers of                     respectively. Tasks 21-30 and 31-40 are normal tasks
ticks before and after the task is created. (We could also in-                 with nice values of 0 and 10, respectively.
troduce new tasks by modifying linsched run sim()                         (d) Three real-time tasks, and 37 normal tasks: one FIFO
directly.) Note that actual scheduling behavior is deter-                      real-time task with priority 90, two round-robin real-
mined by the Linux functions scheduler tick() and                              time tasks with priority 80, and 37 normal tasks with
schedule(), among others—in this case, these func-                             nice value 0. (Lower values indicate higher priority,
tions implement the normal 2.6.23 scheduling policies.                         and real-time tasks are prioritized over other tasks.)
                                                                          All tasks used the ls annc cb() callback described in
3.2 Experimental Results                                                  Sec. 3.1, thus a trace of scheduling decisions was gener-
We now present the results of experiments that demonstrate                ated. Tasks executed indefinitely (e.g., in an infinite loop).
how well scheduling decisions within LinSched correlate                   Results. The results of the first set of experiments are
with the behavior of the Linux scheduler, and how well Lin-               shown in Fig. 3, where the processor time received by each
Sched models scheduling behavior on larger platforms.                     task is presented within both the Linux kernel and Lin-
   In our first set of experiments, we compared the schedul-               Sched. In Fig. 3, insets (a) through (d) correspond to task
ing behavior of Linux 2.6.23 running on an SMP machine                    sets (a) through (d) above. Tasks were “mapped” to iden-
containing four Intel Xeon 2.7 GHz processors to similarly                tical identifiers (1 through 40) so that their behavior within
configured LinSched simulations. The kernel was modi-                      Linux and LinSched can be directly compared easily.
fied to allow a trace of scheduling activities to be obtained                 Overall, LinSched produces results that are highly cor-
in a relatively light-weight manner (first used in [2]), and a             related with the scheduler behavior of Linux—external in-
user-space program was created to launch tasks.                           terference within the Linux kernel (e.g., due to interrupts
   We considered four different workloads in this first set of             or the tracing infrastructure) accounts for the largest differ-
experiments. These workloads consisted of 40 tasks each,                  ences. Where the behavior of LinSched deviates from the
   ∗ More information on the types of tasks described here is available   Linux scheduler, it tends to achieve results that better ap-
by typing man sched setscheduler at any Linux terminal running            proximate intended scheduler behavior. For example, in in-
kernel version 2.6.16 or later.                                           set (a), all tasks have identical priorities, and should achieve
                         x 10
                                         Identical Nice Values
                                                                                                            x 10
                                                                                                                  10              Varying Nice Values
                                                                                                                                                                          4 Conclusion
                                                                  Linux                                                                                   Linux

                                                                                                  4                                                                       Understanding and experimenting with the Linux scheduler
                                                                                                                                                                          can often be a frustrating or time-consuming process, espe-
Run Time (ns)

                                                                                  Run Time (ns)
                    6                                                                                                                                                     cially for those new to Linux development. We present the
                                                                                                                                                                          LinSched tool, which hosts the Linux scheduler as an iso-
                                                                                                                                                                          lated subsystem in user space, for easier debugging of and
                                5   10     15    20    25    30     35       40
                                                                                                                   5         10     15    20    25   30     35       40
                                                                                                                                                                          experimentation with new and existing scheduling policies.
                                                Task #                                                                                   Task #
                                                                                                                                                                          This tool can be extremely useful in the early stages of
                                                (a)                                                                                      (b)
                                                                                                                                                                          scheduling policy development, as it allows for the rapid
                         x 10
                                     Normal and Batch Tasks                                                 x 10
                                                                                                                  10        Normal and Real−Time Tasks                    prototyping and evaluation of many different policies, from
                   15                                                                             6
                                                                                                                                                                          which the most promising ones can be selected for further
                                                                                                                                                                          detailed experimentation. Porting code back into Linux is
                   10                                                                             4
   Run Time (ns)

                                                                                  Run Time (ns)

                                                                                                                                                                          made less difficult since LinSched shares most of its code
                                                                                                                                                                          with the Linux scheduler and related subsystems. In our
                    5                                                                             2
                                                                                                                                                                          evaluation, we demonstrated that LinSched is relatively
                                                                                                                                                                          easy to use and that it generates results that are strongly cor-
                                5   10     15    20
                                                Task #
                                                       25    30     35       40
                                                                                                                   5         10     15    20    25   30     35       40   related with actual Linux scheduler behavior. Our personal
                                                                                                                                         Task #

                                                (c)                                                                                      (d)                              experiences [3] using LinSched support these claims.
                                                                                                                                                                             We want to expand upon this work in several ways. First,
Figure 3: Processor time received by the tasks in four different                                                                                                          we would like to strengthen the automation features of Lin-
workloads, in LinSched and Linux.
                                                                                                                                                                          Sched to easily support experimentation with many dif-
                                                                                                                                                                          ferent types of platforms and workloads. Second, we wish
virtually the same execution times. In inset (c), batch tasks                                                                                                             to improve support for hierarchical topologies, particularly
are supposed to receive a slight penalty as compared to nor-                                                                                                              those arising in NUMA, multicore, and multithreaded plat-
mal tasks. In both cases, the expected behavior is better                                                                                                                 forms. Finally, we are in the process of making LinSched
demonstrated in LinSched. Insets (b) and (d) indicate that                                                                                                                publicly available, and will release it once that process is
LinSched behavior for these workloads is nearly identical                                                                                                                 complete. We are very interested to see how other re-
to Linux. Note that these results are especially interesting                                                                                                              searchers might extend LinSched to suit their needs.
since the time granularity within LinSched is coarser than
that of Linux, as we only make scheduling decisions at each                                                                                                               References
tick—in the absence of interrupts and external interference,
this approach appears to be adequate for our workload.                                                                                                                    [1] Bochs Project.       bochs: The IA-32 Emulator Project.
                                                                                                                                                                          [2] B. Brandenburg, J. Calandrino, A. Block, H. Leontyev, and
Large-scale          sim-            6
                                       x 10
                                             Varying Nice Values, 32 CPUs                                              10
                                                                                                                                                                              J. Anderson. Real-time synchronization on multiprocessors:
ulation. We          next            5
                                                                                                                                                                              To block or not to block, to suspend or spin? Proc. of the 14th
demonstrate          how                                                                                                                                                      IEEE Real-Time and Embedded Technology and Applications
                                                                                                  Run Time (ns)

LinSched          models                                                                                                                                                      Symposium. IEEE, 2008.
scheduler behavior on                                                                                                                                                     [3] John M. Calandrino, Dan Baumberger, Tong Li, Scott Hahn,

larger platforms by                                                                                                                                                           and James H. Anderson. Soft real-time scheduling on per-
                                                                                                                                                                              formance asymmetric multicore platforms. Proc. of the 13th
simulating a scaled-up               0                                                                                                                                        IEEE Real-Time and Embedded Technology and Applications
                                            50    100   150     200   250 300
version of workload                                      Task #
                                                                                                                                                                              Symposium. IEEE, 2007.
(b) on a 32-processor               Figure 4: Results for the                                                                                                             [4] C. Farivar. Intel Developers Forum roundup: four cores now,
platform, again for one             32-processor workload.                                                                                                                    80 cores later.
minute of simulated run                                                                                                                                                       developers-forum-roundup-four-cores-now-80-cores-later/,
time. This workload contains 320 tasks with nice values                                                                                                                       2006.
uniformly distributed between -20 and 19—groups of eight                                                                                                                  [5] Peter S. Magnusson, Magnus Christensson, Jesper Eskilson,
                                                                                                                                                                                                            a               o
                                                                                                                                                                              Daniel Forsgren, Gustav H˚ llberg, Johan H¨ gberg, Fredrik
tasks share the same nice value. Results are shown in                                                                                                                         Larsson, Andreas Moestedt, and Bengt Werner. Simics: A
Fig. 4. Note the similarity between these results and those                                                                                                                   full system simulation platform. IEEE Computer, 35(2):50–
in Fig. 3(b)—such results closely approximate expected                                                                                                                        58. IEEE, 2002.
scheduler behavior. This suggests that LinSched can be                                                                                                                    [6] Plex86 Project. The new Plex86 x86 Virtual Machine Project.
used to provide initial insight into how scheduling policies                                                                                                        
will perform on larger platforms, and the scalability issues                                                                                                              [7] Free Software Foundation. GDB: The GNU Project Debug-
that may arise. While these results are preliminary, we                                                                                                                       ger.
believe from personal experience that LinSched is a very                                                                                                                  [8] User Mode Linux Project. User Mode Linux. http://user-
useful tool for exploring policies on platforms of all sizes.

To top