Project 3 -- Kernel Message System by yTLGz9

VIEWS: 17 PAGES: 39

									                       Programming Project #3
                       Message Passing System

                       CS 3013 Operating Systems
                             C-Term 2008


CS-3013, C-Term 2008            Project 3      1
                       Project

• To build and test a message-passing system
  for Inter-process Communication among
  separate address spaces
      – Implement API for message-passing functions
      – Kernel system calls to handle messages
      – Test program to exercise the message passing
        system



CS-3013, C-Term 2008     Project 3        2
                       Objective

• To gain experience programming and
  testing synchronization and IPC operations

• To gain experience with synchronization &
  memory management in the Linux Kernel




CS-3013, C-Term 2008     Project 3   3
                         Overview

• Add a mailbox to each process
      – Abstract object capable of holding messages
      – Messages are of bounded length, undefined structure
      – Mailboxes may be of bounded size
• All threads of a process share a mailbox
• Any Linux task can send a message to any
  mailbox
      – Including own mailbox
      – Addressed by pid_t

CS-3013, C-Term 2008        Project 3           4
                       Overview (continued)

• Task may receive message only from “own”
  mailbox
      – Any thread of task may receive!
• Mailbox created during fork
• Mailbox deleted during process termination
• Mailbox may be “stopped”
      – I.e., no more message are accepted
      – Optionally, mailbox may be flushed
CS-3013, C-Term 2008          Project 3       5
                       This Project

• Predefined API (Application Program Interface) at
      – mailbox.h in this directory


• All students must implement same API
      – User space interface program
      – Kernel implementation
      – Test program


CS-3013, C-Term 2008      Project 3      6
                                    API
int SendMsg(pid_t dest, void *msg, int
  len, bool block)
            •   Sends a message body at *msg to process dest
            •   Length = len; not more than MAX_MSG_SIZE
            •   Blocks if mailbox full and block = TRUE
            •   Returns zero if successful, error code if not

int RcvMsg(pid_t *sender, void *msg,
  int *len, bool block)
            •   Gets a message from own mailbox, puts in *msg
            •   Sender process ID returned in *sender
            •   Blocks if mailbox empty and block = TRUE
            •   Returns zero if successful, error code if not
            • Messages in FIFO order

CS-3013, C-Term 2008                Project 3               7
                             API (continued)
int ManageMailbox(bool stop, int *count)
     • Gets number of message currently queued in
       mailbox
     • If stop = TRUE, prevents mailbox from receiving
       more messages
                   – Unblocks all waiting SendMsg and RcvMsg
                   – Future RcvMsg calls can still retrieve remaining queued
                     messages
            • Returns zero if successful, error code if not




CS-3013, C-Term 2008                 Project 3                8
         API — Documented Error Codes
• MAILBOX_FULL                                • MSG_TOO_LONG
            • Non-blocking send                     • On send
• MAILBOX_EMPTY                               • MSG_ARG_ERROR
            • Non-blocking receive                  • Invalid argument or
• MAILBOX_STOPPED                                     pointer
                                                    • copy_to_user or
            • On any send
                                                      copy_from_user fails
            • Also after blocked send
              or receive                      • MAILBOX_ERROR
• MAILBOX_INVALID                                   • Any other kind of error
            • On any call                     • You may add other error
                                                codes as needed

CS-3013, C-Term 2008              Project 3                  9
                       Kernel Implementation

• Start with prePatch-Project3
            • I.e., so we all have same task_struct and header files
• Three system calls
               .long sys_mailbox_send
               .long sys_mailbox_rcv
               .long sys_mailbox_manage

• Create and delete mailboxes
• Memory allocation within kernel
            • All messages of fixed size
• Synchronization within kernel
            • Suggest simulating a monitor per mailbox
CS-3013, C-Term 2008               Project 3             10
                       Kernel Implementation

• Start with prePatch-Project3
            • I.e., so we all have same task_struct
• Three system calls
               .long sys_mailbox_send
               .long sys_mailbox_rcv
               .long sys_mailbox_manage

• Create and delete mailboxes
• Memory allocation within kernel
            • All messages of fixed size
• Synchronization within kernel
CS-3013, C-Term 2008           Project 3          11
                       Pre-Patch your Kernel

• Apply prePatch-Project3 to a clean kernel tree
• Adds to task_struct
            • struct *mailbox mailbox

• Defines syscalls in syscall.S and unistd.h
               .long sys_mailbox_send   /* 318 */
               .long sys_mailbox_rcv    /* 319 */
               .long sys_mailbox_manage /* 320 */

• Reason
            • Graders’ time in grading
            • Don’t modify other commonly used header files!


CS-3013, C-Term 2008              Project 3              12
                       Kernel Implementation

• Start with prePatch-Project3
            • I.e., so we all have same task_struct
• Three system calls
               .long sys_mailbox_send
               .long sys_mailbox_rcv
               .long sys_mailbox_manage

• Create and delete mailboxes
• Memory allocation within kernel
            • All messages of fixed size
• Synchronization within kernel
CS-3013, C-Term 2008           Project 3          13
                       Creating a Mailbox

• During do_fork()
            • In kernel/fork.c
• If this is a new process
            • Use kmalloc() to allocate a new data structure for a
              new mailbox
            • Initialize
            • Set pointer in task_struct->mailbox
• If this is just a new thread
            • Keep existing mailbox of process
            • Indicated by CLONE_THREAD argument to do_fork()
CS-3013, C-Term 2008             Project 3         14
              Creating a Mailbox (continued)

• No mailbox for kernel threads, etc.
            • Be sure task_struct->mailbox is null




CS-3013, C-Term 2008         Project 3          15
Kernel Implementation of Mailbox Operations

• Monitor model per mailbox
      – Three system calls for user-visible functions
               sys_mailbox_send
               sys_mailbox_rcv
               sys_mailbox_manage
      – Two other functions for kernel support
               mailbox_create
               mailbox_destroy
• Contents of monitor data structure
      – Monitor lock
            • spinlock_t initialized SPIN_LOCK_UNLOCKED
      – Semaphore and count to simulate condition variable
            • Both initialized to zero
      – List head for linked list of messages; flag for stopped mailbox
      – Other fields as needed
CS-3013, C-Term 2008                     Project 3         16
         Kernel Implementation (continued)
• RcvMsg
            • Grab monitor lock
            • While linked list is empty
                   – Simulate wait on condition variable
            • Unlink first message from mailbox linked list
            • Release monitor lock
            • copy_to_user to copy message body and other
              information to caller
            • Free kernel space for unlinked message (see below)
• Non-blocking receive
            • Exercise for student
CS-3013, C-Term 2008                 Project 3             17
         Kernel Implementation (continued)
• RcvMsg
            • Grab monitor lock
            • While linked list is empty
                   – Simulate wait on condition variable
            • Unlink first message from mailbox linked list
            • Release monitor lock
            • copy_to_user to copy message body and other
              information to caller
            • Free kernel space for unlinked message (see below)
• Non-blocking receive
            • Exercise for student
CS-3013, C-Term 2008                 Project 3             18
         Kernel Implementation (continued)
• RcvMsg
            • Grab monitor lock
            • While linked list is empty
                   – Simulate wait on condition variable
            • Unlink first message from mailbox linked list
            • Release monitor lock
            • copy_to_user to copy message body and other
              information to caller
            • Free kernel space for unlinked message (see below)
• Non-blocking receive
            • Exercise for student
CS-3013, C-Term 2008                 Project 3             19
         Kernel Implementation (continued)
• RcvMsg
            • Grab monitor lock
            • While linked list is empty
                   – Simulate wait on condition variable
            • Unlink first message from mailbox linked list
            • Release monitor lock
            • copy_to_user to copy message body and other
              information to caller
            • Free kernel space for unlinked message (see below)
• Non-blocking receive
            • Exercise for student
CS-3013, C-Term 2008                 Project 3             20
         Kernel Implementation (continued)
• SendMsg
            • Allocate kernel space for new message (see below)
            • copy_from_user to copy message body into kernel
            • Fill in other details (sender, length, etc.)
            • Grab monitor lock
            • Link new message to end of mailbox linked list
            • Simulate signal to condition variable if any receivers waiting
            • Release monitor lock
• Blocking send
            • Exercise for student
            • Must simulate wait on condition variable if mailbox is full

CS-3013, C-Term 2008                Project 3                21
         Kernel Implementation (continued)

• ManageMailbox
            • Grab monitor lock
            • Counter number of messages
            • If stop = TRUE, determine if any tasks waiting
            • If so, trick them into “unblocking” and returning
              error
            • Release monitor lock
• Revisit blocking SendMsg, RcvMsg
            • When unblocked, be sure mailbox has not been
              stopped in meantime.
CS-3013, C-Term 2008            Project 3           22
                       Deleting a Mailbox

• In do_exit()
• Only if the task group is dead!
• Stop the mailbox to be sure that blocked
  send operations can complete!
      – Flush messages (to free their space)
• Free the mailbox data structure
      – kfree()
• Zero out task_struct->mailbox
CS-3013, C-Term 2008          Project 3        23
            Memory Allocation Resources

• kmalloc(), kfree()
            • linux/slab.h
            • Similar to malloc() & free(), but with
              flags
• Slab allocator
            • kmem_cache_t* kmem_cache_create()
            • void* kmem_cache_alloc()
            • void kmem_cache_free()
            • int kmem_cache_destroy()
CS-3013, C-Term 2008       Project 3        24
Memory Allocation Resources (continued)

• Use slab allocator
      – All message bodies the same size in kernel
      – Highly optimized for rapid allocation and free
      – Low fragmentation


• Initialization
      – Establish cache and any static variables during
        fork_init() (in kernel/fork.c)

CS-3013, C-Term 2008      Project 3         25
                Locking Tools in the Kernel

• Mailbox
            • Needs to be locked to link and unlink messages
            • Also to change state (START, STOP)
• Remember – the Linux Kernel is fully
  preemptive!
            •   System call may be preempted before completion
            •   Interrupt may schedule another process at any time
            •   Interrupt may manage shared resources
            •   But not while holding a spinlock
      – Due to support for symmetric multi-processing
CS-3013, C-Term 2008             Project 3           26
                       Robert Love says …

• It is a major bug if …
      – An interrupt occurs to access a resource while
        kernel code is also manipulating that resource
      – Kernel code is preempted while accessing a
        shared resource
      – Kernel code sleeps while in the middle of a
        critical section
      – Two processors access same data at same time
• Implementing locking is not hard
• Tricky part is identifying what to lock.
CS-3013, C-Term 2008          Project 3     27
            Tools for Simulating Monitors

• Spin locks (for monitor locking)
            • spin_lock_init()
            • spin_lock(), spin_unlock()
            • See linux/spinlock.h
            • Must be used to provide mutual exclusion for all
              monitor functions
            • Also to release and acquire lock around condition
              variable waits




CS-3013, C-Term 2008           Project 3            28
                 Condition Variable (simulated)
• Condition variable representation
         • Struct semaphore, int wait_count, both initialized to zero
         • See asm/semaphore.h
• Wait on condition variable
         /* while holding the monitor lock */
           wait_count++;     /* incr # of waiting tasks */
           spin_unlock(&mr_lock);           /*release lock */
           down_interruptible(&sem);        /* wait */
           spin_lock(&mr_lock);      /* reacquire lock */
• Signal condition variable
         /* while holding the monitor lock */
           if (wait_count > 0)
                up(&sem);    /* unblock a task */
           wait_count--;     /* decr # of waiting tasks */

  CS-3013, C-Term 2008           Project 3             29
                              Monitor Example*
monitor FIFOMessageQueue {                               /* function implementations */
    struct qItem {                                       FIFOMessageQueue(void) {
      struct qItem *next,*prev;
      msg_t msg;                                          /* constructor*/
    };                                                    head = tail = NULL;

    /* internal data of queue*/                          };

    struct qItem *head, *tail;                           void addMsg(msg_t newMsg) {
    condition nonEmpty;                                   qItem *new = malloc(qItem);
    /* function prototypes */                             newprev = tail;
                                                          newnext = NULL;
    void addMsg(msg_t newMsg);
    msg_t removeMsg(void);                                if (tail==NULL) head = new;
                                                          else tailnext = new;
    /* constructor/destructor */
                                                          tail = new;
    FIFOMessageQueue(void);
    ~FIFOMessageQueue(void);                              signal nonEmpty;

};                                                       };

*    Adapted from Kleiman, Shah, and Smaalders
      CS-3013, C-Term 2008                   Project 3                  30
                            Monitor Example
/* function implementations                  /* function implementations
       continued*/                               concluded*/
msg_t removeMsg(void) {                      ~FIFOMessageQueue(void) {
 while (head == NULL)                         /* destructor*/
   wait(nonEmpty);                            while (head <> NULL) {
 struct qItem *old = head;                        struct qItem * top = head;
                                                  head = topnext;
 if (oldnext == NULL)                            free(top);
   tail = NULL; /*last element*/
 else                                        };
   oldnextprev = NULL;
 head = oldnext;
                                             /* what is missing here? */
 msg_t msg = oldmsg;
 free(old);
 return(msg);
};


     CS-3013, C-Term 2008        Project 3                  31
                            Monitor Example
/* function implementations                  /* function implementations
       continued*/                               concluded*/
msg_t removeMsg(void) {                      ~FIFOMessageQueue(void) {
 while (head == NULL)                         /* destructor*/
   wait(nonEmpty);                            while (head <> NULL) {
 struct qItem *old = head;                        struct qItem * top = head;
                                                  head = topnext;
 if (oldnext == NULL)                            free(top);
   tail = NULL; /*last element*/
 else                                        };
   oldnextprev = NULL;
 head = oldnext;
                                             /* what is missing here? */
 msg_t msg = oldmsg;                        /* Answer:- need to unblock
                                             waiting threads in destructor! */
 free(old);
 return(msg);
};


     CS-3013, C-Term 2008        Project 3                  32
                       Deleting mailbox

• Must stop mailbox first
• In loop, unblock all waiting tasks
• Be sure any waiting task has had time to
  complete operation
            • Potential race condition
• Flush remaining messages
• Delete mailbox data structure

CS-3013, C-Term 2008            Project 3   33
                              Testing

• Fork multiple processes
            • Create mailboxes, exchange mailbox IDs
• Randomly send messages to each other
            • Payload must be self identifying
• Acknowledge received messages
• Test extreme conditions
            • E.g., fill up mailbox



CS-3013, C-Term 2008            Project 3        34
                             Due Dates
• Project due at Friday, February 22, 6:00 PM
• Interim submissions:–
      – February 8:–
            • Creation & deletion of mailboxes in do_fork(), do_exit()
            • Design of mailbox data structure for producer-consumer
      – February 15:–
            • Sending and receiving messages
            • User space support and simple test program
      – February 22 (final submission):–
            • Stopping mailbox with blocked calls on SendMsg or RcvMsg
            • Flushing messages
            • Comprehensive test program
CS-3013, C-Term 2008              Project 3                35
                       Final Submission
• Submit using web-based turnin program
      – http://turnin.cs.wpi.edu:8088/servlets/turnin.ss
• Include
      – One patch file for kernel implementation
            • Difference from prePatch-Project3 and your implementation
      – User space implementation of mailbox.h interface
      – Test program(s) and results
      – Makefile
      – Write up explaining implementation and testing
• Put your name on all documents and at top of
  every edited file!
CS-3013, C-Term 2008              Project 3                36
                       Interim Submissions

• Patch file representing difference from
  prePatch-Project3

• Simple test program to show partial
  working




CS-3013, C-Term 2008          Project 3      37
                       Individual vs. Teams

• This may be an individual project or a 2-
  person team project
• If two people wish to be a team, register
  with TAs by 5:00 PM on February 5
• Teams will be graded more rigorously than
  individuals
            • Especially the test program



CS-3013, C-Term 2008           Project 3      38
                       Questions?




CS-3013, C-Term 2008     Project 3   39

								
To top