Docstoc

ecosystem_new

Document Sample
ecosystem_new Powered By Docstoc
					 Ideas for Cooperative Disk
Management with ECOSystem


            Emily Tennant
           Mentor: Carla Ellis
            Duke University
                  Outline
• Introduction
• Building blocks for cooperative file
  operations in ECOSystem
• Read Requests
• Write Requests
• Unresolved Issues
• Experimental Plan
                Vision and Goals
• The next step for ECOSystem
   – Energy efficient policy space
   – Energy aware applications
• Vision
   – Create a set of API extensions permitting applications to pass
     application-specific information to the OS.
   – Use this information to manage disk accesses more efficiently.
• Specific Goals
   – Simple interface
   – Minimal changes – backward compatibility
   – Energy savings in terms of currentcy
                    Related Work
• Recent submission to OSDI conference:
   – Andreas Weissel, Björn Beutel, Frank Bellosa. Cooperative I/O—
     A Novel I/O Semantics for Energy-Aware Applications.
• Goal: ―demonstrate benefits of application involvement in
  operating system power management.‖
• Coop-I/O – an approach to reduce power consumption of
  devices (disk)
• Encompasses all levels of the computer system
   – Hardware
   – Operating system
   – I/O interface for energy-efficient applications
     Main Elements – Coop-I/O
• New cooperative file operations
   – read_coop(), write_coop(), open_coop()
   – New parameters: time-out and cancel flag
   – Delay activating disk for length of time-out parameter
   – Possible to abort accesses after time-out period
• Energy-efficient update mechanism
   – Update of cached disk blocks is batched to maximize the time hard
     disk can spend in standby mode.
• OS controls hard disk modes
   – Disk drive is switched into low-power mode according to an
     adaptive algorithm.
   – ―device-dependent time-out with early shutdown (DDT/ES)‖
 Cooperative I/O in ECOSystem
• New cooperative file operations
   – Extend system calls (read, write) with extra parameters.
   – Deferrable disk accesses
   – Options for abortable file operations?

• Energy-efficient update mechanism
   – Updates deferred to create bursty disk access
   – Energy-efficient update strategies integrated into write process

• OS control of disk drive
   – Motivation of adaptive algorithm for powering down disk drive?
                  Outline
• Introduction
• Building blocks for cooperative file
  operations in ECOSystem
• Read Requests
• Write Requests
• Unresolved Issues
• Experimental Plan
                          Bidding
• How do we couch the cooperative, deferrable file
  operations of Coop-I/O in terms of ECOSystem’s
  currentcy?
• ―Bidding‖ process
   – Inflated entry price delays disk                  Entry Price

     spinup to ensure that processes
     have enough currentcy to execute
                                                                                        P1
     and generate more disk requests.



                                           Total Bid
                                                                                        P2
   – Each process ―bids‖ the amount                                                     P3

     of currentcy it is willing to
                                                                            Disk spins up
     contribute towards the entry price.
   – Is bidded currentcy considered                                  Time
     available for use?
                             Priorities
• Motivating Question: How can
  we implement the bidding
  process in a useful and intuitive
  interface (similar to Coop-I/O)?          Time-out           Priority          Currentcy
    – Currentcy is dynamic – difficult
      for application to assign directly.
    – Create static priorities.
    – Map priorities to a currentcy
      amount for bid.
• What type of priorities could be
  created?
    – Integer sets (1-10, 1-100)
    – Real-numbered intervals
    – Dynamic priorities
                                                       Priority (and currentcy bid)
                Dynamic Priorities
• Allow priority to change over time.
   – As a process waits for disk access, its priority can increase, causing
     an increase in the amount of currentcy allocated to its bid.
• A Simple Case:
   –   No feedback required from application
   –   Integer priority levels 1-10
   –   Include time-out parameter in system call
   –   If disk access not scheduled within time-out, jump up one priority
       level.
• Other possibilities
   – Priority function (priority vs. time) given by application
   – Upcalls to application allow on-the-fly changes in priority
• Issues
   – Default priorities, time-out, etc.
           Mapping Priority to Bid
                                       Resource Container
• Takes place within OS
• Involves resource container
• May require resource                 Available_currentcy


  container alteration                          Ticket

   – Bid, priority, time-out, etc.

                 Percentage of
                 available currentcy                                                   Priority?

priority                                                 Available_currentcy

                                                             Bid    Ticket
                  Percentage
                  of entry price
                                                                               Time-out?
       A Simple Mapping Model
• Assume numeric priority.
• Priority corresponds directly to percentage of available
  currentcy that is allocated to bid.

      BID = available_currentcy * (priority/10)


• Example: priority = 5, available currentcy = 1000mJ
• BID = 500mJ
• Remainder of available currentcy can be used (CPU, NIC)
  while process waits to access disk.
• Overhead issues
         Tackling the Exception of
               Abortability
• The ability to save energy/currentcy by aborting a disk access is a
  desirable concept.
• Can abortability be considered in terms of currentcy?

Coop-I/O:                           ECOSystem Possibilities:
• Cancel flag as system call      • Time-out must be a specified, non-
  parameter                         infinite value for all possibilities.
• Abort disk access after waiting • Boolean system call parameter
  through time-out period.
                                        – Abortable accesses can have
                                          decreasing priority over time.
                                    • Unique priority level (i.e. 0)
                                        – Does not require extra parameter
                                        – Can automatically allocate zero
                                          currentcy (no bid)
                  Outline
• Introduction
• Building blocks for cooperative file
  operations in ECOSystem
• Read Requests
• Write Requests
• Unresolved Issues
• Experimental Plan
     Sample Read Disk Accesses
1. System call is generated.              7. If waiting function returns at
   read(……...., [priority],[time-out]);      time-out:
2. Verify that data is uncached               1. If abortable, cancel access.
   and disk is inactive.                      2. Else, increase priority and ―re-
                                                 issue‖ bid.
3. Transform priority       bid.
                                          8. When bids  entry price, disk
4. ―Issue bid‖ – store bid amount
                                              spins up.
   in a data structure.
                                          9. Disk access is scheduled.
5. For dynamic priorities, enter
   timed waiting function/loop.           10. Data is read into buffer cache.
6. Update daemon waits until              11. Resource container is debited
   bids  entry price (reads &               for cost of access.
   writes).
       Write Disk Access Issues
• More complex!
• What does ―deferrable write‖ mean? Do we defer writing
  to the buffer cache or flushing the buffers to the disk?
• How do we guarantee consistency of file system when
  write requests are aborted?
• Coop-I/O: early commit/abort strategy
   – Delays writing data to buffer cache until it can be committed.
       • Drive is active at time of write request
       • Drive will be activated by another committed write (dirty buffers exist).
       • Time-out has been reached and write request is non-abortable.
   – Forces processes to wait before writing to buffers.
   – Defers both writing to buffer cache and flushing buffers to disk.
        Write Disk Access, cont.
• ECOSystem: Unified write/update policy defers disk access
• Simplify non-abortable writes
   – Data can be written to buffer immediately – no need to defer!
   – Process bids to flush buffer to disk drive (only requires resource
     container).
   – What happens to the bidding process when buffers are overwritten?
• Implement abortable writes with early commit/abort
  strategy.
   – Must immediately identify abortable writes.
   – Do not have to issue a bid.
   – Bidding process must take place before writing to buffer.
• All dirty buffers are always flushed to disk.
              Update Mechanism
Coop-I/O                       ECOSystem
• Separate update policy.      • Update strategy
• Four Drive specific-            – Bidding process creates
  cooperative update                bursty disk access.
  strategies:                     – If disk is active, updates
                                    are scheduled immediately.
   – Write back all buffers.
                                  – Disk spinup/spindown
   – Update cooperatively.          controlled by existence of
   – Update each drive              scheduled disk accesses.
     separately.                  – Decreasing entry price
   – Update on shutdown.            guarantees updates.
                                  – FlushStart vs. entry price
Sample Write Disk Accesses
                                       Generate system call

                                             Abortable?
                                 Yes                         No

                  Is buffer already                            Is buffer already
                  cached?                                      cached?
  Yes                                                                                      No
                              No                               Yes

Is disk active?          Is disk active?                  Write to buffer.         Is disk active?
           No                           No                                                           No
     Bid/Wait.                        Bid/Wait            Is disk active?                        Bid (for read).
                                      (for read).                       No
 Abort.                                                                                               Read
            Write to                                                  Bid.                            buffer.
                           Abort. Read
            buffer.               buffer.
                                                                  Flush buffer.                       Write to
            Flush                      Write to                                                       buffer.
            buffer.                    buffer.
                                                                                                          Flush
                                        Flush                                                             buffer.
                                        buffer.
                  Outline
• Introduction
• Building blocks for cooperative file
  operations in ECOSystem
• Read Requests
• Write Requests
• Unresolved Issues
• Experimental Plan
                  Unresolved Issues
                         Same Bytes         Different Bytes           Different
                                           Same Buffer Block        Buffer Blocks
                                                                     Add bids. One
             Same
             Task
                              ?                  Add bids?          total bid per task
                                                                     for disk access.
 Reads
                          One access            Add bids?
           Multiple                                                 Normal bidding
                         Replace bid?         Charge all tasks?
            Tasks                                                      process.
                                                  HOW?
                       Overwriting = one
             Same
                          disk access            Add bids?             Add bids.
             Task        Replace bid?
 Writes
                         Overwriting             Unlikely!
           Multiple                                                 Normal bidding
                         Replace bid?            Add bids?
            Tasks                                                      process.
                        Who is charged?    Charge all tasks? How?

1. How do we accumulate bids?
2. How do we charge for access? Do we weight charge by number of bytes read?
                Experimental Plan
•   Design – 6/14/02
•   Implementation of deferrable reads – 6/26/02
•   Implementation of deferrable writes – 7/5/02
•   Synthetic benchmarks/testing – 7/12/02
•   Rewrite cooperative application – 7/19/02
•   ―Real‖ application testing – 7/24/02
•   Implementation of abortable reads/writes
•   Testing of abortable reads/writes – as time allows (7/29/02)
                                 Testing
                          What do we want to test?
• Synthetic Benchmarks                       • ―Real‖ Application
   – Simple read/write programs.                 – Audio/video player, image
                                                   viewer.
   – Preliminary results.
                                                 – Read file from hard disk.
   – Create scenarios where energy-
                                                 – Test performance in non-
     savings are most obvious.                     optimized, ―real-life‖
   – Simulate different workloads:                 situations.
       • Multiple simultaneous cooperative   • Compare results on:
         tasks
       • Mix cooperative and non-                – ECOSystem unthrottled
         cooperative tasks.                        (Coop-I/O)
       • Test same buffer access issues.         – Current ECOSystem
                                                   implementation
                                                 – ECOSystem with cooperative
                                                   implementation
                     Summary
• Extended system calls for file operations.
• Priority can be specified by application.
• Priority determines amount of currentcy in bid.
• Total bids for disk access must exceed entry price.
• Abortable reads/writes may be cancelled after specific
  period of time.
• Interaction between decreasing entry price and bids for
  disk access works toward efficiently batching disk
  accesses while guaranteeing that non-abortable accesses
  occur.