自動化負載平衡子計畫

							自動化負載平衡子計畫
        第一季報告
    (2009.07~2009.09)
          董信煌
    樹德科技大學資管系
  tungsh@mail.stu.edu.tw
                  Outline
•   Objectives
•   Resource Management in Xen
•   CPU Schedulers of Xen
•   Monitoring Performance of Xen
•   Methodology
•   Summary
•   Outlook
          Objectives
• 系統規劃與需求分析. 了解及熟悉 Xen 環境
  中與資源分配及效能分析有關之功能, 例如
  Xen 對於多個執行於同一系統之虛擬機器
  (VM)的工作排程機制(bvt, sedf, credit等
  scheduler)
                     Xen
• An open source server virtualization
  software package
• This virtual machine monitor (hypervisor)
  offers near-native VM performance
  through
  – Paravirtualization
  – Hardware assisted virtualization
 Resource Management in Xen
• Each domain including host and guests
  consume three types of resource:
  – CPU
  – Memory
  – I/O
  CPU Resource Management
• The main target of this project
• Each guest domain can be assigned one
  or more VCPUs (Virtual CPUs) to be
  realized by physical CPU (or CPUs in
  SMP systems)
• This realization process is called CPU
  scheduling
• Xen has built-in schedulers
Memory Resource Management
• Each guest domain can be assigned a
  predetermined amount of memory
• With proper monitoring, memory utilization can
  be determined
• The goal of memory resource management is to
  warrant each VM has enough memory without
  paging mechanism, and no VM wastes memory
• Domain memory can be dynamically adjusted at
  run time with ‘xm mem-set’
   I/O Resource Management
• Current I/O model in Xen is done through
  drivers in Dom0
• Dom0’s memory assignment is critical for
  I/O performance


                     (Source: Cherkasova et al., Comparison
                     Of the Three CPU Schedulers in Xen,
                     Sigmetrics Perf Eval Review)
 Terminology for CPU Scheduling
• Terms have been extended from the
  domain of process scheduling to the
  domain of CPU scheduling
  – Proportional Share/Fair Share
  – Work Conserving/Non Work Conserving
  – Preemptive/Non Preemptive
     Proportional Share (PS)
• PS scheduling allocates CPU to VMs
  according to their shares (weights)
• PS scheduler provides instantaneous form
  of sharing
• Fair Share (FS) schedulers tend to provide
  a time-averaged form of proportional share
  based on the actual use over long time
  periods
      Work Conserving (WC)
• In the WC-mode, the shares are merely
  guarantees
  – In a case of two VMs with equal shares, if one
    is blocked, the other one can consume the
    entire CPU
• In a Non-WC mode, the shares are caps
  – In a case of two VMs with equal shares, if one
    has consumed 50% of a CPU, it can not
    consume any more of the CPU
  – Bad-behaved domains can be isolated
              Preemptive
• Preemptive schedulers rerun the
  scheduling decision when a new guest
  becomes ready
  – The new guest may preempt the running
    guest if it has a higher priority
• Non-Preemptive schedulers make
  decisions when the running guest gives up
  CPU
      CPU Schedulers of Xen
• Three built-in schedulers
  – BVT (Borrowed Virtual Time)
  – SEDF (Simple Earliest Deadline First)
  – Credit
• The scheduler is determined when Xen
  boots up and can not be changed after
  that moment
• Each scheduler takes input parameters to
  fine tune the performance at run time
                   BVT
• Based on the concept of virtual time (VT)
  consumed by a guest domain (virtual
  machine)
• The domain with a lower VT has a higher
  priority
• PS scheduling
• Work conserving
• Low latency support for real time apps
  through warping VT
                SEDF
• Support for both WC and Non-WC mode,
  bad-behaved VM can be isolated when the
  cap is reached (Non-WC mode)
• Preemptive
• Real time apps support
• Queue is per CPU
• No global load balancing among multiple
  CPUs in SMP systems
                   Credit
•   PS schedulers
•   Support for WC and Non-WC mode
•   Global load balancing in SMP systems
•   Non-preemptive
•   The default scheduler in Xen 3.0
Monitoring Performance of Xen
• Schedulers manage CPU resources for
  guest domains
• Performance of Xen can be monitored by
  – Xen-based tools
  – Apps-based tools
• Performance becomes a feedback to tune
  schedulers
       Xen-based Monitoring
• Xen offers several utilities to monitor the
  performance of a Xen system including
  Dom0 and guest domains
  – xenmon.py, a Python-based utility, was
    designed to compare Xen performance under
    different CPU weightings of the domains
  – xenstore stores configuration, event and
    status information about Xen domains
      Apps-based Monitoring
• VM performance can also be monitored
  through specific apps designated to run on
  the VM, e.g.
  – Web server
  – Database server
  – Computation server
           Adaptive Scheduling
•   Idea
    1. Use Credit scheduler to take advantages of
       global load balancing in SMP systems
    2. Credit accepts two input parameters (weight
       and cap) for each domain to fine tune its
       behavior
    3. Use proper monitoring tools to get domain
       performance
    4. Performance is fed back to adjust weight
       and cap
               Methodology

                                       Workload:
Weight                                 web, db, etc.


                                                           Monitored
                Credit Scheduler
                                                           Performance



Cap




         Performance feedback is used to tune parameters
              Summary (1)
• In this quarter, we have done
  – Study on the resource management of Xen
    domains, particularly from the perspective of
    CPU resource management
  – Study on Xen schedulers for managing CPU
    resources
  – Study on Xen performance monitoring tools
  – Study on fine-tuning Xen schedulers at run
    time
           Summary (2)
• 系統規劃
 – 使用open source軟體為開發基礎
 – 使用Xen平台為server virtualization基礎
 – 使用commodity level多核(multi-core)硬體為平台
 – 假設雲端系統為提供高效率之計算平台，例如使用
   Hadoop之map-reduce，設計一個以CPU資源應用為
   主的自動化負載平衡機制
 – 暫不考慮I/O、memory負載平衡
 – 先以單機多核的負載平衡為標的再拓展到多機群集
             Summary (3)
• 需求分析
 – 以CPU資源應用為主，如何在Xen的環境下管理CPU之資源應用
 – Xen的內建scheduler分配實體CPU的資源給予VM之虛擬VCPU，
   Xen 3.0預設使用的Credit scheduler能使用WC與non-WC mode
   (改善了BVT)及多核的負載平衡(改善了SEDF)
 – Credit scheduler接受參數輸入來設定每一個domain的weight及
   cap值
 – 預設之weight及cap值不一定對所有的domain來說都是最佳的，
   domain實際執行的應用會影響到效能
 – 必須能偵測domain之效能以回饋到Credit scheduler來動態調整每
   一個domain的weight及cap值
 – 使用Xen-based或Apps-based方式來偵測domain之效能
                Outlook
• Looking forward, we plan to
  – Xen 系統實做及資料收集. 實做 Xen 環境及上
    述之VM scheduler於多處理器平台, 收集效能
    分析資料並參考服務品質(QoS)專案之研究, 設
    定效能指標及環境變數.