Best Practices for implementing a virtualized SharePoint Production Farm
Viral Tarpara @tarpara viralta@microsoft.com http://blogs.technet.com/tarpara
Attribution of Content
Brian
Wilson, Microsoft Services UK
Senior Consultant Email Brian.Wilson@microsoft.com Twitter @brianwilson1 Team Blog http://blogs.msdn.com/uksharepoint Blog http://blogs.msdn.com/brianwilson
Tweet Your Questions
Send
a tweet on twitter with „#MSVT‟ hash tag Send me replies or direct messages to „@tarpara‟ or „d tarpara‟
Assumptions
Aimed at the SharePoint audience
Architects, infrastructure specialists, developers, testers and consultants
Pre-requisite knowledge
Understand the server roles in a SharePoint farm Understand common SharePoint deployment scenarios Understand what virtualization is.
Best Practice – “If you fight SharePoint, you will lose.”
Microsoft Confidential
Agenda
Why virtualize? Virtualization software SharePoint server role recommendations Real world examples 4 key physical resources in depth Server deployment best practices High availability and disaster recovery Common mistakes Key takeaways
Why virtualize?
Understand why organizations are moving to virtual server infrastructure
Credit crunch –reduce hardware, power consumption and cooling costs Reduce environmental impact Increase server utilization Improved development and testing life cycle
Why virtualize?
Consolidate
/ Dedicate servers
Reduce number of physical servers Make it easier to provision more dedicated specialty servers (i.e. Separate SharePoint roles onto multiple dedicated virtual servers instead of one „all-in-one‟ server Dedicated servers tend to have less issues as they run „without surprises‟ that can be caused by bundling services Your
salary increase and/or bonus!
Virtualization Software
Virtualization Software
Types of virtualization software
“Hosted” hypervisor “Bare metal” hypervisor
Best practice: use “bare metal” virtualization technologies. Examples: Windows Hyper V, VMware ESX infrastructure, Citrix XenServer
Virtualization Software
Understand the difference between physical versus virtual resources
CPU Memory Network Disk
Virtualization Software
Understand the physical and virtual hardware boundaries of your virtualization vendor technology
Windows Hyper V Architecture CPU Memory
Host: x64 Guest: x86 or x64 x64 CPU with hardware assisted virtualization 4 x virtual CPUS per Guest VM. (Numbers of CPUs is dependant on guest operating system) Symmetric MultiProcessing (SMP) supported Host: up to 1 TB Guest: up to 64 GB RAM per VM 10GB Ethernet adaptors 12 x network adaptors (8 synthetic ; 4 emulated) per virtual machine unlimited virtual switches and unlimited virtual machines per switch Direct Attach Storage (DAS): SATA, eSATA, PATA, SAS, SCSI, USB, Firewire Storage Area Networks (SANs): iSCSI, Fiber Channel, SAS Network Attached Storage (NAS) Pass through disk support (4 x virtual IDE and 4 x virtual SCSI) 512 TB of storage per VM
Networking
Disk
Licensing
Hyper V Server (host free only) ; Windows Standard Server (host + 1 VMs) ; Windows Enterprise Server (host + 4 VMs) Windows Datacenter Server (host + unlimited VMs)
Best Practices
Use x64 host for greater CPU and memory availability Use latest operating system for guest OS Review and understand management tools Understand support and ensure software is certified via Windows Server Virtualization Validation Program (SVVP) Understand licensing (Standard, Enterprise, Datacenter)… Applies to all virtualization software!
SharePoint server role recommendations
SharePoint server role recommendations
Good candidates include the following front-end roles
Web server role Query role Index role (“It depends…” ) Other application roles
Although it is supported, database servers are not good candidates.
SharePoint Farm – Web Role
Responsible for rendering of content with low amount of disk activity Multiple web role servers are common for redundancy and scalability Best Practices
Be sure to keep all components, applications, and patch levels the same
Network Load Balancing (NLB)
– Hardware -> Offload NLB to dedicated resources – Software -> CPU and Network usage on WFE
CPU
RAM
DISK
NETWORK
For minimum availability split your load balanced virtual web servers over two physical hosts
SharePoint Farm – Query Role
search queries Requires propagated copy of the index
10%- 30% of total size of documents indexed
Process
CPU
RAM
DISK
NETWORK
Best
Practice
Large Indexes – Prefer dedicated physical LUN on SAN over dynamic expanding virtual hard disk Don‟t put your query and index servers on the same underlying physical disk
Combine or split Web/Query role?
It depends on your environment. Web and Query performance requirements
SharePoint Farm – Index Role
Memory,
CPU, Disk I/O and network
CPU
intensive Best Practices
Give most amount of RAM out of front ends Potentially keep as physical machine Use Index server to be dedicated crawl server. Avoids hop. Prefer physical LUN on SAN to virtual hard disk
RAM
DISK
NETWORK
SharePoint Farm – Other roles
Services, document conversions services are good candidates for virtualization Additional servers can simply be added into the farm No additional hardware investment required
Excel
CPU RAM
DISK
NETWORK
SharePoint Farm – Database role
SQL Server 2005/ 2008 virtualization fully supported Memory, CPU, Disk I/O and network intensive Assess first using Microsoft Assessment and Planning Toolkit (www.microsoft.com/map). SQL Alias flexibility Argument for Physical:
SQL Server is already a consolidation layer Disk I/O activity Performance, performance, performance! Longer response times impacts ALL downstream roles in a SharePoint farm
CPU
RAM
DISK
NETWORK
SharePoint Farm – Database role
If
you decide to virtualize database layer:
Assign as much RAM and CPU as possible Offload the Disk I/O from the virtual machines – Prefer pass through disks over virtual disks SQL Clustering : Either virtualize the entire database layer or keep all physical. Do not virtualize the passive node. Not recommended.
CPU
RAM
DISK
NETWORK
Samples of real-world virtualized SharePoint environments
Courtesy of Michael Noel Michael@cco.com, Convergent Computing, http://www.cco.com, WhitePaper: http://tinyurl.com/virtualsp
Single Host Virtual Environment
Allows organizations that wouldn‟t normally be able to have a test environment to run one Allows for separation of the database role onto a dedicated server Can be more easily scaled out in the future
Slide courtesy of Michael Noel Michael@cco.com
Dual Host Virtual Environment
HighAvailability across Hosts All components virtualised Uses only two Windows Ent Edition Licenses
Slide courtesy of Michael Noel Michael@cco.com
Dual Host - Mixed Physical/Virtual
Highest transaction servers are physical Multiple farm support, with DBs for all farms on the SQL cluster Only five physical servers total, but high performance
Slide courtesy of Michael Noel Michael@cco.com
Multiple Hosts – Scale Out
Slide courtesy of Michael Noel Michael@cco.com
4 hardware resources in depth
CPU Best Practices
PHYSICAL
VIRTUAL
Performance is governed by processor efficiency, power draw and heat output Faster versus efficient processor – hidden power consumption cost Beware of built in processor software such as performance throttle for thermal thresholds Prefer higher number of processors and multi core Prefer PCI Express to limit bus contention & cpu utilization Prefer 64 bit to 32 bit as it effects how much memory can be addressed and supports larger number of CPUS.
Configure a 1-to-1 mapping of virtual CPU to physical CPU for best performance Be aware of the virtual processor limit for different guest operating systems and plan accordingly Beware of “CPU bound” issues, the ability of processors to process information for virtual devices will determine the maximum throughput of such a virtual device. Example: Virtual NICS
Memory Best Practices
PHYSICAL
VIRTUAL
Ensure there is sufficient memory installed on the physical computer that hosts the Hyper-V virtual machines Factor in Hypervisor memory overhead over and above standard virtual machine memory requirements. Use a 64-bit guest operating system when possible
Configure the correct amount of memory for guests. (memory configuration is hardware specific). NUMA Memory Considerations (see next slide) Watch out for page file / swap file. Disk is always slower than RAM. Ensure enough memory is allocated to each virtual machine. If using VMware, avoid over committing memory as it may cause virtual machine to swap to disk which is slower than RAM.
NUMA Memory Limitations
Non-Uniform Memory Access (NUMA) Boundaries exist at the hardware level. Virtual guests that are allocated more memory than exist within a single NUMA memory boundary have significantly impacted performance
Example: NUMA boundaries vary by processor and motherboard vendor, but good rule of thumb to calculate boundaries is to divide the amount of memory in the system by the total number of cores. i.e. Dual Quad-core host (2x4 cores = 8 cores) with 64GB of RAM on the host would mean NUMA boundary is 64/8 or 8GB. In this example, allocating more than 8GB for a single guest session would result in performance drops.
NUMA Memory Guidelines applied to SharePoint
Keeping NUMA boundaries in mind, this means that you will get significantly better performance provisioning two SharePoint frontends with half the amount of RAM as a single front-end with twice as much RAM. This applies to any virtualisation platform, as the limitation is hardware specific!
Disk Best Practices
PHYSICAL
VIRTUAL
Ensure you are using the fastest Fibre Channels SAN infrastructure: Attempt to provide each virtual machine with its own IO channel to shared storage using dual or quad ported HBAs and Gigabit Ethernet adapters. Ensure your disk infrastructure is as fast as it can be. (RAID 10; 15000 RPM) – Slow disk causes CPU contention as Disk I/O takes longer to return data. Put virtual hard disks on different physical disks than the hard disk that the host operating system uses
Prefer SCSI controller to IDE controller. Prefer fixed size to dynamically expanding (more info here) Prefer the pass-through disk feature in Hyper-V to either of the above. Beware of underlying disk read write contention between different virtual machines to their virtual hard disks Ensure SAN is configured and optimized for virtual disk storage. Understand that a number of LUNs can be provisioned on the same underlying physical disks
Network Best Practices
PHYSICAL
VIRTUAL
Use Gigabit Ethernet adaptors and Gigabit switches Increasing network capacity – Add a number of NICs to host. Use IPv4 as the network protocol for Hyper-V host and disable IPv6 (More info here)
Ensure that integration components (“enlightenments”) are installed on the virtual machine Use the Network Adapter instead of the Legacy Network Adapter when configuring networking for a virtual machine Prefer synthetic to emulated drivers as they are more efficient, use a dedicated VMBus to communicate to the Virtual NIC and result in lower CPU and network latency. Use virtual switches and VLAN tagging for security and performance improvement and create and internal network between virtual machines in your SharePoint farm. Associate SharePoint VMs to the same virtual switch. Use IPv4 as the network protocol for HyperV guests and disable IPv6 (More info here)
High Availability and Disaster Recovery
High Availability Start off with TechNet guidance – “Plan for availability” Implement failover clustering Test impact on your farm
– Quick/ Live Migration (Windows Hyper V) – VMotion (VMWare) – ZenMotion (Citrix Zen Servers)
Disaster Recovery Easy to setup virtual DR farm! Data : Log Shipping or SQL Mirroring Use your management tools to setup a disaster recovery farm
Server deployment best practices
Use SQL Alias for greater flexibility Use index as your dedicated crawl server No query propagation result in fewer disk requirements Network, network, network Database maintenance
Common mistakes
Understand the impact of your virtualization vendor feature set! Don‟t let governance slip in your virtualized SharePoint environment Restoring snapshots is not supported Beware of over subscribing host servers Host is a single point of failure
Key Takeaways
Don‟t fight SharePoint, you‟ll lose! Virtualisation allows you to cut costs, consolidate equipment, and take greatest advantage of hardware resources. Understand your hardware and use the correct processor architecture and operating system Plan for availability Continually measure and optimize configuration to achieve optimal performance of your SharePoint farm
And one final one......
Resources
MCS UK SharePoint Team Blog: Virtualizing SharePoint Series - Introduction Virtualizing SharePoint Series - Recommendations for optimizing the performance of a virtualized SharePoint environment Virtualizing SharePoint Series - Recommendations for each server role in the virtualized SharePoint environment Virtualizing SharePoint Series - Recommendations for monitoring and managing your virtualized SharePoint environment Virtualizing SharePoint Series - High availability and disaster recovery, deployment best practices, common mistakes and summary Michael Noel (Convergent Computing) - Michael@cco.com Whitepaper : http://tinyurl.com/virtualsp ,