Virtual Hierarchies to Support Server Consolidation

Virtual Hierarchies to Support Server Consolidation Mike Marty Mark Hill University of Wisconsin-Madison ISCA 2007 Motivation: Server Consolidation www server 64-core CMP L2 Cache Core L1 database server #2 database server #1 middleware server #1 middleware server #1 Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Motivation: Server Consolidation www server 64-core CMP database server #2 database server #1 middleware server #1 middleware server #1 Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Motivation: Server Consolidation www server 64-core CMP data database server #2 database server #1 data middleware server #1 middleware server #1 Optimize Performance Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Motivation: Server Consolidation www server 64-core CMP database server #2 database server #1 middleware server #1 middleware server #1 Isolate Performance Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Motivation: Server Consolidation www server 64-core CMP database server #2 database server #1 middleware server #1 middleware server #1 Dynamic Partitioning Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Motivation: Server Consolidation www server 64-core CMP database server #2 data database server #1 VMWare’s Content-based Page Sharing  Up to 60% reduced memory middleware server #1 middleware server #1 Inter-VM Sharing Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Executive Summary Motivation: Server Consolidation • Many-core CMPs increase opportunities Goals of Memory System: • • • • • Performance Performance Isolation between VMs Dynamic Partitioning (VM Reassignment) Support Inter-VM Sharing Hypervisor/OS Simplicity Proposed Solution: Virtual Hierarchy • • Overlay 2-level hierarchy on physically-flat CMP Harmonize with VM Assignment Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Outline Motivation • Server consolidation • Memory system goals • Non-hierarchical approaches Virtual Hierarchies Evaluation Conclusion Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin TAG-DIRECTORY A Read A duplicate tag directory Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin TAG-DIRECTORY A fwd Read A 3 data 1 2 duplicate tag directory getM A Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin STATIC-BANK-DIRECTORY A Read A 3 fwd data 1 2 A getM A Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin STATIC-BANK-DIRECTORY with hypervisor-managed cache 2 fwd A getM A Read A A 1 3 data Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Goals STATIC-BANK-DIRECTORY w/ hypervisor-managed cache STATIC-BANK-DIRECTORY TAG-DIRECTORY Optimize Performance Isolate Performance Allow Dynamic Partitioning Support Inter-VM Sharing No No Yes Yes No No Yes Yes ? Yes Yes Yes Yes Hypervisor/OS Simplicity Virtual Hierarchies to Support Server Consolidation Yes No Mike Marty, University of Wisconsin Outline Motivation Virtual Hierarchies Evaluation Conclusion Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Virtual Hierarchies Key Idea: Overlay 2-level Coherence Hierarchy on CMP - First level harmonizes with VM/Workload - Second level allows inter-VM sharing, migration, reconfig Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin VH: First-Level Protocol Intra-VM Directory Protocol w/ interleaved directories Questions: • How to name directories? • How to name sharers? getM INV Dynamic home tile selected by VM Config Table • Hardware VM Config Table at each tile • Set by hypervisor during scheduling Full bit-vector to track any possible sharer • Intra-VM broadcast also possible Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin VH: First-Level Protocol Example: p12 p13 p14 L2 Cache Core L1 per-Tile VM Config Table Address ……000101 offset 6 0 1 2 3 4 5 p12 p13 p14 p12 p13 p14 Dynamic Home Tile: p14 63 p12 Hypervisor/OS can freely change VM Config Table • • • No cache flushes No atomic updates No explicit movement of directory state Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Virtual Hierarchies Two Solutions for Global Coherence: VHA and VHB memory controller(s) Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Protocol VHA Directory as Second-level Protocol • Any tile can act as first-level directory • How to track and name first-level directories? Full bit-vector of sharers to name any tile • State stored in DRAM • Possibly cache on-chip + Maximum flexibility - DRAM State - Complexity Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin VHA Example 2 A getM A 6 getM A directory/memory controller data 1 3 data 5 Fwd Fwd A 4 data A Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Protocol VHB Broadcast as Second-level Protocol Attach token count for each block [token coherence] • T tokens for each block. One token to read, all to write • Allows 1-bit at memory per block • Eliminates system-wide ACK + Minimal DRAM State + Enables easier optimizations - Global coherence requires more activity Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin VHB Example 2 getM A A getM A 1 memory controller global getM A 3 Data+tokens 4 A Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Memory System Goals VHA and VHB Optimize Performance Isolate Performance Allow Dynamic Partitioning Support Inter-VM Sharing Yes Yes Yes Yes Hypervisor/OS Simplicity Yes Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Outline Motivation Virtual Hierarchies Evaluation Conclusion Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Evaluation: Methods Wisconsin GEMS • Full-system, execution-driven simulation • Based on Virtutech Simics • http://www.cs.wisc.edu/gems 64-core tiled CMP • • • • In-order SPARC cores 512 KB, 16-way L2 cache per tile 2D mesh interconnect, 16-byte links, 5-cycle link latency Four on-chip memory controllers, 275-cycle DRAM latency Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Evaluation: Methods Workloads: OLTP, SpecJBB, Apache, Zeus • Separate instance of Solaris for each VM Approximating Virtualization • Multiple Simics checkpoints interleaved onto CMP • Assume workloads map to adjacent cores • Bottomline: No hypervisor simulated Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Evaluation: Protocols TAG-DIRECTORY: • 3-cycle central tag directory (1024 ways!) STATIC-BANK-DIRECTORY • Home tiles interleaved by frame address VHA • All data allocates in L2 bank of dynamic home tile VHB • Unshared data always allocates in local L2 All Protocols: one L2 copy of block on CMP Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Result: Runtime Eight VMs x Eight Cores Each = 64 Cores (e.g. eight instances of Apache) 1.4 1.2 Normalized Runtime 1 0.8 0.6 0.4 0.2 0 OLTP Apache Zeus SpecJBB Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Result: Memory Stall Cycles Eight VMs x Eight Cores Each = 64 cores (e.g. eight instances of Apache) Off-chip 1.4 Local L2 Remote L1 Remote L2 Normalized Memory Stall Cycles 1.2 1 0.8 0.6 0.4 0.2 0 OLTP Apache Zeus SpecJBB Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Executive Summary Server Consolidation an Emerging Workload Goals of Memory System: • • • • • Performance Performance Isolation between VMs Dynamic Partitioning (VM Reassignment) Support Inter-VM Sharing Hypervisor/OS Simplicity Proposed Solution: Virtual Hierarchy • • Overlay 2-level hierarchy on physically-flat CMP Harmonize with Workload Assignment Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Backup Slides Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Omitting 2nd-Level Coherence: Protocol VH0 Impacts: • • • Dynamic Partitioning Inter-VM Sharing (VMWare’s Content-based Page Sharing) Hypervisor/OS complexity Example: Steps for VM Migration from Tiles {M} to {N} 1. 2. 3. 4. Stop all threads on {M} Flush {M} caches Update {N} VM Config Tables Start threads on {N} Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Omitting 2nd-Level Coherence: Protocol VH0 Example: Inter-VM Content-based Page Sharing • Up to 60% reduced memory demand Is read-only sharing possible with VH0? VMWare’s Implementation: • • • Global hash table to store hashes of pages Guest pages scanned by VMM, hashes computed Full comparison of pages on hash match Potential VH0 Implementation: • • How does hypervisor scan guest pages? Are they modified in cache? Even read-only pages must initially be written at some point Mike Marty, University of Wisconsin Virtual Hierarchies to Support Server Consolidation Physical Hierarchy / Clusters P P P P L1 $ L1 $ L1 $ L1 $ P P P P L1 $ L1 $ L1 $ L1 $ Shared L2 Shared L2 Shared L2 L1 $ L1 $ L1 $ L1 $ P P P P Shared L2 L1 $ L1 $ L1 $ L1 $ P P P P Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin Physical Hierarchy / Clusters middleware server #1 www server P P P P L1 $ L1 $ L1 $ L1 $ P P P P L1 $ L1 $ L1 $ L1 $ Shared L2 database server #1 Shared L2 Shared L2 L1 $ L1 $ L1 $ L1 $ P P P P Shared L2 L1 $ L1 $ L1 $ L1 $ P P P P Virtual Hierarchies to Support Server Consolidation Mike Marty, University of Wisconsin