Best practices for HP BladeSystem and Microsoft Exchange Server 2007 Executive summary..............................................................................................................................3 HP BladeSystem c-Class – An enabler for an Adaptive Infrastructure..........................................................3 HP BladeSystem c-Class....................................................................................................................4 Cost-savvy infrastructure................................................................................................................4 Energy-thrifty infrastructure.............................................................................................................4 Time-smart infrastructure................................................................................................................4 Change-ready infrastructure...........................................................................................................5 HP BladeSystem c-Class architecture overview.................................................................................5 Why HP BladeSystem for Microsoft Exchange Server 2007?....................................................................8 Deployment..................................................................................................................................11 Exchange Server 2007 deployment..............................................................................................13 Management................................................................................................................................14 HP Onboard Administrator..........................................................................................................15 HP Insight Control.......................................................................................................................16 HP ProLiant Essentials..................................................................................................................18 Bringing it all together with Exchange Server 2007........................................................................19 Enterprise management integration...............................................................................................20 Power and cooling........................................................................................................................21 I/O connectivity............................................................................................................................24 HP Virtual Connect.....................................................................................................................27 Network configuration................................................................................................................28 Storage configuration.................................................................................................................30 Performance.................................................................................................................................31 Server recovery.............................................................................................................................32 Rip and replace........................................................................................................................33 Automated node recovery...........................................................................................................33 Best practices for choosing an HP server blade for Microsoft Exchange Server 2007.................................34 Exchange Server 2007 roles-based architecture................................................................................34 Edge Server..............................................................................................................................36 Hub Transport Server..................................................................................................................37 Client Access Server...................................................................................................................38 Unified Messaging Server...........................................................................................................38 Mailbox Server.........................................................................................................................39 Active Directory.........................................................................................................................40 Multi-role servers........................................................................................................................40 Exchange Server 2007 example configurations.....................................................................................41 Appendix A – 5,000 user example configuration BOM..........................................................................46 Appendix B – 10,000 user example configuration BOM........................................................................49 Appendix C – 15,000 user example configuration BOM.......................................................................52 For more information.........................................................................................................................54 Executive summary Most customers consider Microsoft® Exchange Server 2007 to be an application that is critical to the organization. That any prolonged outages or performance issues with Exchange can have a serious impact on the business. However, to support this level of application service, and deploy a high performance and highly available solution to support Exchange can often result in a costly solution. To keep the costs and complexity of managing the Exchange deployment in control, customers are demanding solutions that simplify management and reduce the burden on their IT (Information Technology) staff. The HP BladeSystem c-Class infrastructure provides a compelling data center platform to achieve the goal of a lower total cost of ownership. The HP BladeSystem is a modular architecture with technology to help customers reduce their IT costs, both in terms of initial acquisition and ongoing operational costs1, and build an Adaptive Infrastructure for Exchange Server 2007. This white paper provides an overview on how deploying Exchange Server 2007 on the HP BladeSystem c-Class can save organizations both time and money and reduce the overall total cost of ownership of their Exchange deployment. This includes an overview of the HP BladeSystem architecture and the key technologies focused on improving infrastructure manageability, reducing power and cooling requirements, and building a change-ready architecture. The document then addresses why the HP BladeSystem c-Class architecture is a compelling platform for Microsoft Exchange Server 2007 including a look at how the various Exchange Server 2007 roles correlate to the different HP server blades. The final portion of the white paper covers three example configurations for Exchange on the HP BladeSystem. This includes 5,000, 10,000, and 15,000 Exchange user configurations. Target audience: This best practices guide is intended for a wide range of audiences from Exchange architects and administrators to business decision makers. Previous knowledge of Exchange Server 2007 and the HP BladeSystem architecture are beneficial. Additional background information is available at www.hp.com/go/bladesolutions/exchange, www.hp.com/go/bladesystem, and www.microsoft.com/exchange. HP BladeSystem c-Class – An enabler for an Adaptive Infrastructure Over the past few years, an industry shift has begun that is redefining the way customers evaluate IT. No longer is IT being viewed as a static entity, disassociated with other business functions. IT has become a core component of the company and must remain dynamic and agile to meet the demands of today’s most complex business environments. However, many IT data centers are not optimized to be adaptive and flexible to shifting business requirements today. Many data centers are composed of isolated pools of resources, with too many application silos, requiring too many resources and too much customization. This results in increased costs and unnecessary complexity in the data center. Customers are demanding a standardized approach to meet these IT challenges, an evolution to a light outs, 24x7 data center that can respond dynamically to change, and is always running, just like the business. This is the next-generation data center (NGDC). The NGDC is a simplified, automated data center based on standard, repeatable building blocks to ensure the highest levels of interoperability and economies of scale. This is the HP Adaptive Infrastructure.2HP meets your need to move to a next-generation data center with the Adaptive Infrastructure, a portfolio based on standard building blocks, automated using modular software and delivered through comprehensive services. Adaptive Infrastructure enablers lay the foundation for an Adaptive 1 Please read the “IDC whitepaper: TCO of blades” available at www.hp.com/go/bladesystem for more information. 2 For more information on the HP Adaptive Infrastructure, please visit www.hp.com/go/adaptiveinfrastructure 3 Enterprise by allowing you to move from high-cost IT islands to low-cost shared IT assets, helping you gain: • Lower cost of IT operations • Speed in introducing IT change • Higher quality of service The HP Adaptive Infrastructure provides a set of key enablers and integrated offerings that drive gains across broad areas of your data center—including IT systems, power and cooling, management, security, virtualization, and automation. The HP BladeSystem is a core component of the Adaptive Infrastructure, helping to address a number of these solution areas in the data center. HP BladeSystem c-Class The HP BladeSystem is a key enabler and a core building block of the Adaptive Infrastructure. The modular architecture of the HP BladeSystem provides a flexible, scalable, standards-based approach to meet the IT systems hardware requirements for the data center. But beyond addressing the hardware requirements of the next-generation data center, the HP BladeSystem is also designed to address a number of other key customer requirements for an Adaptive Infrastructure including: power and cooling, management, and virtualization. Customer concerns around server costs, power and cooling costs, management costs, and data center inflexibility are major driving factors in the data center. To address these IT pain points, the HP BladeSystem c-Class is designed to be a cost-savvy, energy-thrifty, time-smart, and change-ready platform. Cost-savvy infrastructure One of the core value propositions of the HP Adaptive Infrastructure is to lower the cost of IT operations, both in terms of initial hardware acquisition costs as well as on-going operational costs. Accomplishing this requires an intelligent hardware infrastructure that goes beyond individual resources (such as server, storage, and network components) and takes a holistic approach to the data center. The HP BladeSystem c-Class lowers the hardware capital and operational costs by taking advantage of a shared architecture design and centralized management and control. Energy-thrifty infrastructure The HP BladeSystem c-Class was designed towards improving power and cooling efficiency to enable IT organizations to realize maximum density benefits and a reduced data center footprint. The HP BladeSystem has the ability to monitor power usage and dynamically throttle power consumption. With the HP Thermal Logic technology, and the new graphical thermal dashboard, organizations can easily monitor the actual power being consumed by the HP BladeSystem and determine if the power draw is nearing capacity limits. The HP BladeSystem has also been designed to maximize the cooling capabilities and efficiency of the infrastructure with HP Active Cool fan technology and an innovative enclosure and fan architecture design. Redundant, scalable HP PARSEC (parallel and redundant scalable enclosure cooling) quickly dissipates heat in even the densest configurations, so you can pack in more compute capacity per square inch. Time-smart infrastructure Between wasted time, unproductive time, and down time, a lot of time can be lost managing your data center. By consolidating all the computing resources in the data center – server, storage, power, and networking, into a single, common infrastructure management console, the HP BladeSystem c-Class helps eliminate lost time in your data center. As a core component of the Adaptive 4 Infrastructure, the HP BladeSystem was designed to provide out-of-the-box 24/7 real-time monitoring and alerting and remote Lights-Out access. Change-ready infrastructure A core design requirement for the HP BladeSystem c-Class is to provide a modular architecture that can adapt and change with changing business requirements. This not only enables a more dynamic IT infrastructure, it also helps lower operational costs and improve the return on investment (ROI) of the data center by enabling more efficient resource utilization. By building on this design requirement and leveraging a centralized management environment, the HP BladeSystem c-Class provides a modular and dynamic infrastructure. And with the HP Virtual Connect (VC) architecture, the HP BladeSystem is always change-ready. Virtual Connect allows server blade LAN (local area network) and SAN (storage area network) connections to be virtualized. This means your IT staff can make behind-the-scenes adjustments without impacting any physical LAN and SAN networks: This means upgrading, migrating, redeploying, or failing over can be accomplished quickly, easily, and invisibly to the outside world. HP BladeSystem c-Class architecture overview The HP BladeSystem c7000 is a 10U enclosure and can support a combination of HP half-height and full-height server blades, along with a direct attached storage blade for additional storage expansion. At the time of publishing, the following server and storage blades are available: • ProLiant BL460c server blade • ProLiant BL465c server blade • ProLiant BL480c server blade • ProLiant BL685c server blade • Integrity BL860c server blade • StorageWorks SB40c storage blade The naming convention is such that models ending in a zero are based on Intel® Xeon® or Itanium® CPUs, while model numbers ending in 5 are based on AMD Opteron™ CPUs. The BL460c and BL465c are half-height server blades and up to 16 can be installed in a single enclosure. An example of a fully populated c7000 enclosure with 16 BL460c servers is pictured in Figure 1. 5 Figure 1. Fully populated c7000 enclosure with 16 ProLiant BL460c server blades The BL480c, BL685c and BL860c are full-height server blades and up to 8 can be installed in a single enclosure. Note: The HP BL860c3 server is based on the Intel Itanium processor family and is not supported with Exchange Server 2007. This server blade is designed to support high-end database applications (for example Microsoft SQL Server). The BL460c and BL480c are two processor servers with support for either dual-core or quad-core processors. This provides scalability of up to 8 processor cores for these blades. The BL465c is also a two processor server with dual-core processor support. The BL685c is a four processor server with dual-core processor support. The BL685c also provides scalability up to 8 cores, but with more physical processor sockets, four rather than two in the other server blades. The other blade option is the SB40c. The SB40c supports up to six small form factor (SFF), serial attached SCSI (SAS) or serial ATA (SATA), drives. This provides up to 876 GB of direct attached storage (DAS) to an adjacent server blade. There are two additional features on the c-Class enclosure worth noting from Figure 1 above. First, up to six power modules can be installed in the enclosure. This provides either single or three phase N+N or N+1 redundant power, self-contained within the enclosure. The second thing to observe is the integrated HP Insight Display panel. The HP Insight Display provides administrators an easy way 3 For more information of the HP BL860c please visit http://h18004.www1.hp.com/products/servers/integrity-bl/c-class/860c/index.html 6 to perform initial enclosure setup and a diagnostic display of the status of the HP BladeSystem environment. Figure 2 below shows an example of a fully populated c7000 enclosure from the rear view. Figure 2. Rear view of c7000 enclosure The c7000 enclosure supports up to four, fully redundant, or eight non-redundant, I/O modules supporting Fibre Channel (FC), Ethernet, and InfiniBand (IB) protocols. In total, the HP BladeSystem c-Class provides up to 5 TB of available bandwidth to the 16 server blades in the chassis. The bottom slot in the enclosure (directly above the bottom set of fans) is where the HP Onboard Administrator management modules are installed4. The Onboard Administrator serves as the brains of the enclosure. The c7000 enclosure also utilizes a new fan architecture design. The HP Active Cool fans are an integral part of the c7000, providing a unique mix of optimized cooling, acoustics, airflow and performance to make thermal regulation more efficient than ever. The HP BladeSystem c-Class supports up to ten fans for complete redundancy across the enclosure. Additional information on the components and architecture of the HP BladeSystem will be discussed throughout this white paper. However, for a thorough look at the HP BladeSystem architecture, please visit www.hp.com/go/bladesystem. 4 Only one module is required, with the second module available for redundancy. 7 Why HP BladeSystem for Microsoft Exchange Server 2007? Microsoft Exchange Server 2007 is a business critical application for many organizations. Given the importance of this type of application, any unplanned downtime can be costly and have a significant impact on the business. This requires companies to deploy Exchange on a highly available, high performing and secure messaging infrastructure that can reduce the likelihood of downtime. However, highly available, high performance solutions will come at a cost for customers to implement. For example, providing high availability for the new Exchange Server 2007 roles will require customers to scale-out the infrastructure and add additional servers for redundancy. To achieve the highest levels of performance for the Exchange solution, customers may need to scale-up to a four socket, or quad core processor server. Either scaling-up or scaling-out the infrastructure will add additional hardware and operational costs that companies must take into account as they are designing their Exchange deployment. The HP BladeSystem, as a key enabler of the Adaptive Infrastructure, and can help customers build a lights-out, 24x7 Exchange Server 2007 deployment that reduces their IT expenses, both in terms of initial acquisition and ongoing operational costs. The initial hardware acquisition costs for an Exchange deployment are only a fraction of the costs associated over the lifetime of a server. However, in a large, global Exchange organization, this cost may still be significant. The modular, shared architecture design of the HP BladeSystem helps businesses realize an incremental cost benefit as the number of server blades increases. A common misconception around HP blades is that because of the shared infrastructure components, server blades will be more expensive then comparable rack or tower mount servers. This is correct in small business scenarios where the customer is only looking to deploy 1 or 2 servers for the entire organization. However, when moving beyond a couple of servers, HP server blades quickly become compelling from a cost standpoint. The specific number of servers needed to transition from rack mount to server blades will vary depending upon the specific customer configuration; whether the HP BladeSystem is connected to a SAN, the type of network interconnects, etc. For example, with as few as eight server blades connected to a SAN, the HP BladeSystem is approximately 35-40 percent less expensive than comparable rack-mounted infrastructures. Figure 3 below shows the output from the HP BladeSystem TCO tool for an eight server configuration comparing the HP BL685c server blades against a comparable HP ProLiant DL585 server. 8 Figure 3. BL685c versus DL585 TCO (total cost of ownership) comparison In non-fibre channel (FC) SAN environments, the up-front costs are in the range of 10-15 percent less expensive, depending on networking choices. These costs benefits can significantly add up when looking at deploying tens to hundreds of server blades. While the acquisition costs of the hardware may not be insignificant, the operational expenses make up the greatest percentage of the total cost of ownership for an Exchange deployment. The major operational costs for an Exchange deployment are much the same as they would be for any application deployment and include the following expenses: • Real estate • Power and cooling • Management For organizations looking to improve the density of their Exchange IT environment, the HP BladeSystem provides the architecture to considerably reduce a data center’s infrastructure footprint. By consolidating to the HP BladeSystem, companies can realize a significant reduction in the real 9 estate footprint of the Exchange deployment. Organizations can deploy up to 64 server blades in a single 42U rack. This compares to a total of only 42, 1U rack mount, or 21, 2U rack-mount servers in the same space. And this can be accomplished without having to sacrifice server features and performance for density. The BladeSystem environment also allows customers to integrate the necessary network and storage area network switches or HP Virtual Connect5 modules within the enclosure. In a traditional non-blade environment, these components would consume valuable rack space in the data center and generate a significant cable management problem in dense server environments. With the HP BladeSystem environment, cable and port reduction can save money and help reduce unplanned downtime attributed to human error. Another common misconception is that a server blade consumes more power and/or requires more cooling then comparable rack mount models. This is also not true. In fact, in a study directly comparing rack mount and server blades, the rack mount servers actually consumed more power6. However, because of the density of a server blade environment, data center layouts need to be designed and validated to meet the power and cooling requirements of increasingly dense server environments. Technologies such as the HP Modular Cooling System7 and HP Dynamic Smart Cooling8 can help customers increase cooling resources in dense server environments. However, one of the major focuses during the design phase of the HP BladeSystem was around improving the intelligence and efficiency of the infrastructure. The HP BladeSystem will treat power and cooling requirements as a resource; dynamically adjusting the power supply utilizations and fan speeds to provide a more efficient utilization of thermal resources then rack or tower mount servers. More discussion around the power and cooling benefits of the HP BladeSystem in an Exchange environment is discussed in the Power and cooling section below. The cost benefits of the HP BladeSystem in terms of hardware, real estate, power and cooling are tangible benefits that provide real dollar returns for a company’s Exchange deployment. Beyond these costs, server management makes up the other major chunk of operational costs that is incurred9. To maximize the benefits of the integrated HP BladeSystem environment and reduce operation costs, HP provides a centralized management and deployment environment with HP Insight Control. With HP Insight Control, management of server, storage and network resources is controlled from a single console interface and deployment tasks are automated and repeatable to cut down on the amount of time required to provision an Exchange server blade. The HP BladeSystem also provides location awareness and system intelligence integrated into the management environment. Combining this awareness and intelligence, with the flexibility of the HP BladeSystem architecture also allows customers to react more dynamically to shifting business requirements. The flexibility of the HP BladeSystem is fundamental, both in terms of initial hardware rollout and in the ability to rapidly adjust to changing business requirements. Business agility is a critical component of any IT organization and is often overlooked in the decision making process. Exchange designs are often analyzed as a static entity at the time of deployment. However, most Exchange implementations frequently experience a percentage of growth (either internally or through external mergers and acquisitions) that require additional system resources. With the HP BladeSystem, the time that is required to deploy additional server blades to the environment can be significantly lower than comparable rack-mount infrastructures. This reduces the overall cost of managing the Exchange environment by leveraging the agility of the HP Adaptive Infrastructure. The following sections in this portion of the white paper go into further detail and analysis on the considerations for deploying Exchange Server 2007 on the HP BladeSystem including the following topics: • Deployment 5 www.hp.com/go/bladesystem/virtualconnect 6 Please read the report published at www.hp.com/go/bladepowerreport for more information on the study. 7 http://h18004.www1.hp.com/products/servers/proliantstorage/racks/mcs/index.html?jumpid=reg_R1002_USEN8 http://h71028.www7.hp.com/enterprise/cache/434556-0-0-000-121.html?jumpid=reg_R1002_USEN9 Please read the “IDC whitepaper: TCO of blades” available at http://www.hp.com/go/bladesystem for more information. 10 • Management • Power and cooling • I/O connectivity • Performance • Server recovery Deployment Independent of the application being provisioned, server deployment is a multi-stage process for IT departments. The first stage of server deployment is installing the hardware into the data center. This includes physically racking the servers and connecting the network and potentially SAN (storage area network) cables to the core infrastructure switches. Once the server has been physically installed and cabled, the next stage in the process is to install the software on the system. This would include the Microsoft Windows® operating system, service packs, hotfixes, ProLiant Support Packs, 3rd-party software, and Exchange Server 2007. Let’s first look at the first stage of server deployment, the physical server installation. With traditional rack or tower servers, the process of physically racking and cabling a large number of Exchange servers can be time consuming. Each server requires a considerable number of cables for power, networking, fibre, keyboard, video, and mouse (KVM) connections. For example, a typical rack mount server might utilize two power cords, three to five network cables (one for remote Integrated Lights-Out 2 (iLO 2) management), one KVM cable, and two fibre cables. This is a total of eight to ten cables per server, which when multiplied over the entire Exchange infrastructure, can be a significant cabling burden. Administrators must also make certain that proper networking wire schematics are followed to ensure that the correct network cables are plugged into the right network interfaces, and if a SAN is deployed, the same care must be taken for the fibre cable connections. Not only is this a time consuming and tedious process that is prone to administrator error, it often requires the involvement of multiple IT groups within an organization. The issues of physical server deployment are only further magnified when deploying servers into an operational data center. With rack-mount or tower servers, adding additional servers into the data center to meet increasing business demands is not a simple process. Extreme care must be taken so as not to disrupt servers that are currently in production and cause unplanned downtime in your Exchange environment. Looking at the worst case example, if a customer has standardized on 42U racks with 1U rack-mount servers, adding additional 1U servers into an extremely dense server environment can be a challenging task to complete without shutting down the production servers that are running in the rack. The HP BladeSystem c-Class architecture helps address these common server deployment concerns. With the HP BladeSystem, customers benefit from the modular architecture design and the rack-and-wire once deployment model. The modular, shared architecture design of the HP BladeSystem provides customers the flexibility to consolidate power, networking, and SAN FC connections within an enclosure. Remote management is provided through iLO 2 technology included with every server blade. Administrators can connect to the iLO 2 interface to perform a number of administrative tasks such as remote startup and shutdown, connect to virtual media (floppy, CD-ROM, serial) and remote console support. Each server blade contains a dedicated iLO 2 network connection, and multiple administrators can connect to the different server blades in the enclosure simultaneously. In addition, the shared design of the HP BladeSystem aggregates the required iLO 2 connections through the Onboard Administrator network port. And with complete control of power and management built into each server blade, a KVM switch is an unnecessary managed device and investment for a customer’s network. By eliminating the KVM switch and bundling the iLO 2 connections, cable requirements and equipment costs are further 11 reduced. Looking at the aggregate benefit just in terms of this cabling reduction and elimination of the KVM infrastructure, the HP BladeSystem architecture can help customers reduce the number of cables required from eight to ten per server to eight to ten for the entire enclosure (up to 16 servers). This saves money by reducing the number of infrastructure switch ports, cables, and simplifying the management of the server infrastructure. The HP BladeSystem design also eliminates the need to rack and wire each server independently. Once the HP BladeSystem enclosure is configured in an infrastructure rack, customers simply slide in server blades as the design requires. This provides a much simpler approach to server installation and eliminates the need to individually cable each server. This also means that, as the demand for capacity increases and the number of servers grow, server blades can be added incrementally to an existing enclosure without additional wiring or manipulation in a live data center. This can significantly reduce the amount of unplanned downtime due to human error when working in a denser server environment. With the HP Virtual Connect architecture, this server provisioning process is further simplified. HP Virtual Connect (VC) modules provide a layer of server-edge I/O virtualization, allowing customers to separate the management of the LAN and SAN configuration from the server configuration. With VC, server profiles can be created that contain a set of MAC (media access control) and WWPN (world wide port name) addresses. LAN and SAN administrators can then use these values to configure any necessary switch VLANs (virtual local area networks), zones, and LUN (logical unit number) selective storage presentations settings. The server administrator can then apply the profile to any slot in the blade enclosure. This means that the necessary LAN and SAN settings can be configured up front and as servers are added to the enclosure or the role of the server changes to meet business demands, the server administrator can control this change process without involving the LAN and SAN administrators. A potentially time consuming process if the LAN and SAN administrators are part of a separate team than the server administrators. For more information on using HP Virtual Connect with Exchange Server 2007 please read the white paper “HP Virtual Connect architecture and Microsoft Exchange Server 2007” available at www.hp.com/solutions/activeanswers/exchange. Once the servers are connected in the data center, IT departments are faced with the time consuming task of provisioning the hardware with the required software, including operating system, applications, and service pack updates. In the past, server deployment was mostly a manual process requiring administrators to go from server to server with the installation CDs. However, advancements in provisioning software have allowed administrators to automate the distribution of installation media to many servers in parallel, realizing a significant reduction in the time spent provisioning servers. The HP BladeSystem architecture is designed to provide the ability to remotely deploy operating system and application software by tightly integrating with the HP ProLiant Essentials Rapid Deployment Pack (RDP) software. The HP BladeSystem c7000 enclosure does not contain a CD-ROM or DVD drive. However, a virtual CD-ROM/DVD/.iso image can be mounted to each server blade via iLO 2. This means customers can still perform manual Windows Server 2003 and Exchange Server 2007 media based installations. Customers also have the option of using an external USB drive directly connected to each blade via the SUV (serial, USB, and video) dongle. However, both of these methods are inefficient and require a significant amount of administrator involvement. A significant component of building an Adaptive Infrastructure for Exchange Server 2007 with the HP BladeSystem architecture is to increase the level of data center automation and efficiency. This means reducing the time Exchange administrators lose on tasks that can be automated. To maximize the benefits of the HP BladeSystem, the HP RDP software can be used to deploy and provision the Exchange servers. The HP RDP software enables administrators to drop pre-configured deployment jobs onto bare-metal servers to perform system configuration tasks and scripted Windows operating system installs. Jobs can also be created to capture and deploy system images, hot fixes, and patches. These jobs can be deployed to a single server or to multiple servers simultaneously, reducing the time required for provisioning a base operating system and application install to a matter of hours for the 12 entire Exchange infrastructure. With the tight integration of RDP and the HP BladeSystem architecture, the deployment process can be automated one step further. RDP has awareness of the blade environment and can detect blade enclosures and specific enclosure bays. With this capability, administrators can pre-configure jobs based on the enclosure bay with a feature called virtual bay technology. The virtual bay technology allows administrators to associate a specific enclosure slot with a specific server and application configuration, see Figure 4. Figure 4. RDP virtual bay configuration RDP will detect when a server is slid into the blade enclosure and automatically launch a deployment job based on the enclosure slot. Thus, once the BladeSystem enclosure has been racked and the RDP virtual bay deployment jobs configured, the process of deploying the OS onto a blade is as simple as unpacking the equipment from the cardboard shipping box and sliding the servers into the desired enclosure slots. For more information on the HP ProLiant Essentials Rapid Deployment Pack, visit http://www.hp.com/go/rdp. Exchange Server 2007 deployment There are numerous new features and benefits that will drive upgrades from previous versions of Exchange to Exchange Server 2007. One of the features that will be extremely important for Exchange administrators is the redesign of the management environment. The management interfaces for Exchange Server 2007, the Exchange Management Shell (EMS) and Exchange Management Console (EMC), are now built on top of the Windows PowerShell interface. With this design, any task that can be executed in the GUI, or EMC, can also be executed on the command-line, or EMS. And 13 unlike previous programming interfaces for Exchange, the EMS provides a very simple, easy-to-use command-line interface for administrators to learn and use for day to day Exchange management tasks. By providing an optimized command line management interface, administrators will be able to automate more of the tasks associated with managing an Exchange deployment. This provides several significant benefits including reducing the burden on the IT staff to perform the more tedious, time-consuming tasks, along with increasing overall administrator efficiency and the server to administrator management ratio. Another significant improvement with Exchange Server 2007 is the deployment flexibility provided by an increased number of distinct Exchange server roles. With previous versions of Exchange, a standard install job was executed to lay down Exchange and then post-installation tasks were performed by the administrator to configure a specific server type, for example an Exchange front-end server. Much of the deployment work centered on administrator-run post configuration tasks to configure the server. With Exchange Server 2007, administrators now have five, distinct server roles to choose from at the time of install, thus reducing the configuration time required after installation. With these improvements in the Exchange Server 2007 software, coupled with Insight Control and the HP BladeSystem, administrators will be able to fully automate the deployment and provisioning of an Exchange server. RDP can be leveraged as an automation engine to drive the deployment of the various software components including Windows, the prerequisite Exchange files, and the Exchange binaries. EMS scripts can then be created to automate the Exchange post-configuration process. By combining the power of the EMS environment and the Exchange server role functionality, with the HP BladeSystem and RDP virtual bay technology, server blades can be taken out of the box, inserted into the enclosure, and automatically provisioned and configured as Exchange servers in a matter of hours. When this is implemented on multiple servers simultaneously, the amount of time required to fully deploy an Exchange infrastructure is significantly reduced. By fully automating the process, fewer manual steps requiring human intervention are necessary in either stage of the deployment process. This both lowers the cost of deployment and increases the simplicity associated with growing a data center as demand increases. Management The initial hardware acquisition costs of deploying Exchange Server 2007 can be a significant expense for organizations, but only makes up a percentage of the associated costs over the lifetime of an Exchange deployment. The majority of costs are incurred in managing and supporting the environment10. Supporting an Exchange Server 2007 environment includes handling day to day administrative tasks (moving mailboxes, backup/restore operations, etc.), monitoring performance and service levels, and dedicating time and resources to planned downtime such as maintaining updated software releases, patches, and system drivers and firmware. Exchange administrators must also be able to respond rapidly in the event of costly, unplanned downtime. With the HP BladeSystem and Insight Control Data Center Edition organizations can increase data center efficiency, achieve faster time to production, and realize tangible time and cost savings. To help increase the efficiency and lower the cost of managing an Exchange Server 2007 deployment, the HP BladeSystem architecture provides multiple layers of management. Administrators can manage the Exchange infrastructure components from the individual server level up to the data center level, with each layer of management building upon the previous layer. Working from the bottom of the management stack, the first layer of management is at the individual server level. As mentioned in the previous Deployment section above, each HP server blade contains an HP Integrated Lights-Out 2 management processor. The iLO 2 management processor enables administrator to remotely manage the HP server blades individually. If there is an error condition, 10 Please read the “IDC whitepaper: TCO of blades” available at www.hp.com/go/bladesystem for more information. 14 administrators have the ability to open a remote console session to any of the HP server blades and gain access to the server BIOS (basic input/output system), the POST process (Power on self test) and through the entire Windows boot sequence. Figure 5 provides an example of the iLO 2 remote console interface. Figure 5. iLO 2 remote console Beyond the ability to remotely connect to the server console, administrators can also remotely mount drives and remotely power on/power off each server blade. This eliminates the necessity for a KVM infrastructure, and the requirement of physical data center access for managing the Exchange server hardware. HP Onboard Administrator The next level of management is at the HP BladeSystem enclosure level. Each c-Class HP BladeSystem enclosure contains an Onboard Administrator (OA) module11 that provides the intelligence behind the HP BladeSystem enclosure. Through a web interface, or locally via the integrated HP Insight Display, administrators can manage and monitor enclosure level resources such as cooling requirements, 11 A second module can be added for redundancy. 15 power utilization and consumption, server health, and I/O interconnect keying mismatching12. Figure 6 shows a screenshot of the OA web interface. Figure 6. HP OA enclosure view The overview page depicts the front and rear view of the HP BladeSystem enclosure13. From here, administrators can drill down on the individual infrastructure components. For example, clicking on a server blade gives administrators the option of launching the iLO 2 remote console window or viewing the port mapping for that server to the interconnect modules in the rear of the enclosure. Selecting a power supply or fan provides information on the current power and cooling resource consumptions. For more information on the HP Onboard Administrator technology, please visit www.hp.com/info/onboard. HP Insight Control In order to fully benefit from the server level and enclosure level management layers, a third layer giving administrators a view of the entire data center infrastructure is required. HP Insight Control 12 I/O keying mismatch prevents a server blade from powering on with a mezzanine card installed that does not match the interconnect module fabric installed in the rear of the enclosure. 13 Administrators can cascade and manage up to four enclosures from a single OA interface. 16 provides this layer of management on top of iLO 2 and Onboard Administrator. Insight Control Data Center Edition is an integrated software suite that simplifies the provisioning and management of HP BladeSystem infrastructures. Insight Control builds on HP Systems Insight Manager and ProLiant Essentials software to manage HP BladeSystem lifecycles, including hardware resource deployment, health monitoring, performance monitoring, virtual machine management, and server vulnerability scanning and patch management. Insight Control includes the following software components: • HP Systems Insight Manager (HP SIM) • HP Rapid Deployment Pack (RDP) • HP Performance Management Pack (PMP) • HP Vulnerability and Patch Management Pack (VPM) • HP Virtualization Management Software (VMS) HP Systems Insight Manager is a free, integrated management tool used to centrally manage (fault, inventory and configuration management) all HP server and storage platforms via a web-based GUI or command line. HP SIM is another key enable of the Adaptive Infrastructure for Exchange 2007. With HP SIM, administrators can manage the messaging infrastructure from a single interface allowing monitoring of system hardware components such as internal/external storage, processor, memory, network/HBA adapters. Figure 7 depicts a high-level view of the HP unified management approach. HP SIM is the common management platform that forms the foundation of this approach when building an Adaptive Infrastructure. Figure 7. HP unified management Within HP SIM, there is a dedicated interface for managing the HP BladeSystem, the HP BladeSystem Integrated Manager console. The HP BladeSystem Integrated Manager monitors and reports on events within the BladeSystem environment. The integrated console provides a graphical representation of the BladeSystem and the capability to monitor all the components of the entire BladeSystem environment simultaneously from a single interface. Administrators can monitor the health and performance of the server blades, check the utilization of the SAN and network switches, verify the latest hardware driver versions, and monitor the shared power subsystem. Figure 8 below shows a picture of the HP BladeSystem Integrate Manager within HP SIM. 17 Figure 8. HP BladeSystem Integrated Manager within HP SIM From HP SIM, the administrator can monitor the health status of the Exchange 2007 demo enclosure. HP SIM will also collect and display the power and cooling data that rolls up from the OA modules in the enclosures. This allows administrators to monitor thermal requirements as a resource in the Exchange deployments. Beyond monitoring the health and status of the Exchange infrastructure, HP SIM can help administrators maintain and track driver and firmware versions using the Version Control Repository (VCR). HP publishes new driver and firmware updates as part of a bundled set of software called the ProLiant Support Packs (PSP). With HP SIM and VCR, as a new PSP becomes available, it is automatically downloaded into the repository. The administrator can then execute an update task from HP SIM targeted on a single blade, or group of server blades, to simultaneously update the server(s) to the proper driver and firmware levels. This can significantly cut down on the time required to ensure systems are up to date in the Exchange infrastructure. HP SIM is also tightly integrated with RDP, and deployment tasks can be launched from the HP SIM console. With this integration, administrators can perform bare-metal deployment tasks on one or a number of server blades concurrently, all from within the HP BladeSystem Integrated Manager console. HP ProLiant Essentials As part of the unified management approach, additional software, called the HP ProLiant Essentials, can leverage the HP SIM framework to provide additional management functionality. The HP Insight 18 Control software suite bundles together several HP ProLiant Essentials with the HP SIM installation. These include the HP Performance Management Pack (PMP), HP Vulnerability and Patch Management Pack (VPM), and HP Virtualization Management Software (VMS). Out of the box, HP SIM provides a high level overview of the performance statistics of each server blade. To provide in-depth performance examination and monitoring capability for specific Exchange servers, PMP can be used. The HP ProLiant Essentials PMP software provides a detailed real-time performance analysis of the server blade and detects and predicts hardware bottlenecking conditions. PMP is a performance monitoring tool that helps administrators diagnose future performance problems proactively rather than reactively. The ProLiant Essentials VPM software integrates the capabilities of vulnerability and patch management into the HP SIM console. This gives administrators the tool to not only constantly monitor all the servers in the environment for critical security vulnerabilities, but also provides patch management capabilities. With VPM, patches can be downloaded to the HP SIM repository, and HP SIM can deploy the critical patches to the server blades in the Exchange environment. Combining VPM with the capabilities of the VCR provides administrators a comprehensive solution to managing software and hardware updates on the Exchange servers. With Microsoft Exchange Server 2007, the use of virtual servers to provide high availability for the scale-out server roles (Exchange Hub Transport, Exchange Client Access Server) becomes an intriguing solution for certain customer environments. A key component of moving to a virtual server environment is ensuring that a consistent server management process can be applied; independent of whether the administrator is managing a physical or virtual resource. The HP VMS software provides the ability to integrate the management of the virtual machine environment directly within HP SIM, helping administrators reduce the complexity of managing both a virtual and physical server environment. Beyond the ProLiant Essentials that are bundled as part of the HP Insight Control software suite, there are other ProLiant Essentials that provide additional functionality to the customer’s SIM installation. Some of the more important ones for an Exchange deployment are the iLO 2 Power Management Pack (iPMP), HP Insight Power Manager (IPM) and the Intelligent Networking Pack (INP). With IPM and iPMP administrators can: • Measure and manage power of the HP ProLiant servers • Determine how much power the servers are actually drawing • Determine what the inlet air temperatures are • Throttle CPU usage in the desired servers to save power INP enables ProLiant servers that are running basic ProLiant teaming software to adapt to and change the network path to achieve maximum reliability and performance. INP can monitor and analyze network conditions, detect and analyze network bottlenecks or broken network linkages, and redirect traffic to the optimum path. HP Virus Throttle (VT) is part of INP. Virus Throttle protects the messaging infrastructure from spreading unknown viruses or worms that managed to penetrate the Exchange server infrastructure. HP INP VT monitors network adapter’s outbound traffic for unusual network behavior and slows the spread of potential viruses and reduces network disruption. Upon detection of abnormal outbound traffic behavior, HP INP VT can be configured to report an alarm to network administrator via HP Systems Insight Manager. For a complete list of HP ProLiant Essentials and additional information, please visit www.hp.com/go/proliantessentials. Bringing it all together with Exchange Server 2007 Utilization of the HP BladeSystem and HP Insight Control can enable Exchange administrators to conveniently configure, deploy, and manage individual or groups of server blades. Figure 9 below 19 provides a look at how the HP Insight Control software can be leveraged by Exchange administrators over a three year lifecycle. HP RDP can be used for initial server deployment. Rack awareness and the integration of HP SIM with RDP yields convenient deployment and location benefits when managing blades. Once the OS and Exchange files have been installed and configured, PMP can be used to monitor the server environment for issues that may impact Exchange server performance. Both unplanned and planned downtime can be reduced by leveraging the VCR and VPM to automate common management tasks such as firmware updates and patch deployment. Figure 9. Three year management lifecycle example using HP Insight Control The HP BladeSystem unified management environment enables administrators to reduce the amount of Exchange planned and unplanned downtime by automating common management tasks, providing a global view of resource utilization and performance, and easing the cost and complexity of managing everything in the data center separately. For more information on HP BladeSystem management, please visit www.hp.com/go/bladesystem/management. Enterprise management integration Depending on the size of the customer’s organization, HP SIM and the additional HP ProLiant Essentials may provide a sufficient management and monitoring infrastructure for the Exchange environment. However, larger customers that already have an enterprise level management solution such as HP OpenView or Microsoft Operations Manager (MOM) 2005 can also benefit from HP SIM. HP OpenView can leverage the data and information from HP SIM via the HP SIM integration software for HP OpenView Operations (OVO). For customers that are using Microsoft MOM, the HP ProLiant Management Pack for MOM 2005 provides direct integration with HP SIM as well. To provide further management and monitoring support for Microsoft Exchange Server 2007, customers can deploy the HP OpenView Exchange Server 2007 Smart Plug-ins (SPI). With the Exchange SPI, customers can monitor their entire exchange organization with HP OpenView. This includes monitoring for Exchange service level performance, along with the health status and 20 performance of the Exchange Server 2007 subsystem (for example disk performance, queue growth, Continuous Cluster Replication (CCR) or Local Continuous Replication (LCR) cluster status, etc). For additional information on HP OpenView and the Exchange Server 2007 SPI, please visit http://h20229.www2.hp.com/Power and cooling Energy usage is a growing issue in IT environments. Applications such as Exchange Server 2007 require high levels of performance, and demands for higher performance drive the deployment of faster, and thus, hotter servers. Faster processors and higher memory demands causes an increase in the amount of heat that is generated and the amount of cooling that is needed. This cycle can eventually impact all aspects of performance, reliability and cost. To help customers break out of this cycle, the HP BladeSystem was designed to provide an intelligent and optimized infrastructure to help manage the thermal resource consumption of the Exchange deployment. The following is a list of some of the HP Thermal Logic improvements with the HP BladeSystem14: • Active Cool fans • Parallel Redundant Scalable Enclosure Cooling (PARSEC) design • Instant Thermal Monitoring • Pooled power for true N+N power redundancy • Dynamic Power Saver mode • Power Regulator • Power workload balancing The HP BladeSystem design utilizes an intelligent shared power and cooling architecture to provide more efficient utilization of thermal resources. This is different from the traditional requirements of rack and tower servers which have independent power supplies and fans within each server. For example, during the day when Exchange Servers are running at higher utilization levels, power and cooling requirements will be dynamically increased to meet the server demands. Then, during off-hours when Exchange server utilization is typically not as high, the thermal resources allocated for the servers will dynamically decrease. This results in a more efficient usage of thermal resources as compared to traditional rack or tower mount servers. A common misconception with server blades is that server blades consume more power and require more cooling then comparable rack servers. In fact, this is not the case, a server blade will actually consumer less power and cooling then a comparable rack mount server. Figure 10 shows a projection of a 320-server environment comparing the HP BladeSystem versus a comparable rack-mount ProLiant DL360 server model using the HP BladeSystem TCO tool. The HP BladeSystem environment provided nearly a 40% savings (greater than $160,000) in terms of power and cooling costs. 14 For documentation and additional information on these technologies and the benefits of HP Thermal Logic, please visit www.hp.com/go/bladesystem/thermallogic. 21 Figure 10. 320-server environment HVAC and power cost comparison for DL versus BL servers There is a publicly available HP BladeSystem Power Sizer available at, http://h71019.www7.hp.com/ActiveAnswers/cache/347628-0-0-225-121.html, which can help in estimating the power and cooling requirements for the Exchange configuration. One important thing customers must keep in mind is that the HP BladeSystem does allow for increased server density within the data center, and this will require proper evaluation of the cooling resources in the data center. The cooling system must adequately cool a dense server infrastructure, as more power will be drawn in a smaller space with a blade deployment. Looking beyond the technical innovations like the Active Cool fans, PARSEC architecture, and Dynamic Power Saver and the benefits of the shared power and cooling resources, another significant benefit of the HP BladeSystem is the real-time power and cooling monitoring. Exchange administrators can quickly see, in real-time, the power consumption and cooling requirements of the Exchange servers. 22 Figures 11 and 12 below show the output from the HP BladeSystem Onboard Administrator enclosure level power reporting. The first screen shot reports the information in terms of the number of Watts that the enclosure is consuming. The second figure, Figure 12, shows the output in terms of the amount of Btu/hr (British thermal units per hour) being generated. This data enables an Exchange administrator to accurately plan for the thermal resource consumption of the Exchange environment. Figure 11. Enclosure power level reporting -Watts 23 Figure 12. Enclosure power level reporting – Btu/hr This information is also fed up through the management layers and is reported in HP SIM. Figure 8 above in the management section monitors the power consumption and temperature for the enclosure that is being viewed. Looking at Figure 8, one can see the visible impact of Dynamic Power Saver mode. The figure shows the current power consumption of the six power supplies. However, only four power supplies are currently consuming any power. This is because of the impact of Dynamic Power Saver mode. The idea behind Dynamic Power Saver is that power supplies are more efficient when operating at higher power utilizations. For example, a power supply will be more efficient when drawing 75% of its maximum power draw, rather then 25%. The HP BladeSystem Thermal Logic design leverages this benefit and draws more power from fewer power supplies. That is why in Figure 8, two of the power supplies are in standby mode. If additional power was required, these power supplies could be turned on as necessary. I/O connectivity For applications such as Exchange, an important consideration is the number and type of I/O connections a server can support. Depending upon the configuration, an Exchange server may require four or more network cards and two or more FC connections. Thus, I/O scalability is going to be an important design factor. Though, it will be likely that not all of the servers deployed to support Exchange Server 2007 will require the same I/O connectivity. The various Exchange server roles have different performance requirements and demands on I/O subsystems. Thus, another important criterion is that the server blades are flexible in the type and number of I/O connections. With the c-Class HP BladeSystem, customers have the choice and flexibility to mix and match various I/O interconnect modules in the enclosure to meet the demands of the application. This includes support for Ethernet, Fibre Channel, and the InfiniBand protocol15. For both Ethernet and FC protocols, customers can choose between three different types of connectivity modules: pass-thru, integrated switches, and Virtual Connect. 15 Given the I/O footprint and cost, InfiniBand is not really a play for Exchange Server 2007 so the focus will be on the Ethernet and FC module options. 24 Figure 13 below shows a diagram of the rear of the HP BladeSystem c7000 enclosure. As the diagram shows, the c7000 contains 8 slots for I/O modules. To provide I/O communication from each server blade there are signal traces routed from each of the server blade slots to the I/O interconnect slots. To ensure the maximum amount of I/O flexibility in the c7000 design, these I/O signal paths are protocol independent. Thus, any protocol (Ethernet, FC, InfiniBand, PCI-eXpress) can be routed over any signal trace. Figure 13. Diagram of I/O interconnect bays Each server blade contains hardwired embedded NIC ports along with additional mezzanine card slots that are wired to a specific interconnect slot in the rear of the enclosure. Thus, in order to support a specific I/O fabric, the administrator must match the mezzanine card’s protocol to the protocol supported by the interconnect module. Figures 14 and 15 below show the internal view of both a half-height and full-height server blade and highlight the locations of the mezzanine cards on the system board. Figure 14. Internal view of half-height server blade 25 Figure 15. Internal view of full-height server blade For example, with the HP BL480c blade, a customer can configure up to 8 NIC and 4 HBA ports per server blade. To accomplish this, two network modules must be installed in interconnect slots 1 and 2 to support the 4 embedded NICs. An additional quad-port NIC mezzanine card should be placed in mezzanine slot 1 and two network modules installed in interconnect slots 3 and 4. This will provide the 8 NIC ports. Then two dual-port HBA mezzanine cards can be installed in mezzanine slots 2 and 3. Four FC modules would then need to be installed (filling up the rear of the enclosure) to provide the 4 FC communication paths. This is more than sufficient to meet the I/O demands of most Exchange 2007 server roles. And to help Exchange server administrators understand the I/O connection paths, the Onboard Administrator contains a detailed port mapping for each server blade. Figure 16 shows the port mapping for a BL685c server blade with 4 embedded NIC connections via 2 VC Ethernet modules in slots 1 and 2, and 2 FC connections via VC FC modules in slots 5 and 6. 26 Figure 16. OA port mapping There are potential reasons to deploy any of the module types depending on the specific customer requirements. The following three sections will further examine the network and storage configuration options and why the HP Virtual Connect modules provide significant benefits for an Exchange Server 2007 deployment. Please visit www.hp.com/go/bladesystem/interconnects for more information on the interconnect options for the HP BladeSystem. HP Virtual Connect As described in the “HP Virtual Connect architecture and Microsoft Exchange Server 2007” white paper16, and at www.hp.com/go/bladesystem/virtualconnect, Virtual Connect is an implementation of server-edge I/O virtualization, based on industry standard protocols, and provides an abstraction layer between the LAN and SAN connections and the server. There are two VC interconnect options, the VC Ethernet and VC Fibre Channel modules. A single VC module plugs into one of the eight standard interconnect bays in the rear of the c-Class enclosure. For redundancy, an additional VC module can be installed in an adjacent interconnect bay. The VC modules are not switch devices; rather the VC modules aggregate the LAN and SAN connections running from each of the server blades in the enclosure. External communications are then routed over uplinks on the VC Ethernet and FC modules to the first layer of switches in the 16 White paper is available for download at www.hp.com/solutions/activeanswers/exchange. 27 environment. This eliminates the extra layer of switch management required when integrating the switches within the blade enclosure, but also simplifies and reduces the cabling for the enclosure. With the Virtual Connect Manager, an administrator can create server profiles with pre-defined MAC and WWPN addresses. These addresses can then be utilized by the LAN and SAN administrators to configure various LAN and SAN settings such as VLANs, SAN switch zones, and selective storage presentation (SSP) on the storage subsystem. The server profile can then be applied to any slot in an HP BladeSystem c7000 enclosure in the VC domain and those MAC and WWN address settings will be applied to the server blade in that slot. And all this can be completed prior to the servers ever being inserted into the enclosure. For example, the server administrator can create server profiles for 16 servers in the enclosure. The appropriate MAC and WWN address information for those profiles can then be provided to the LAN and SAN administrators and the network properly configured. Then, down the road, when IT demands require additional Exchange servers, the administrator can insert a blade into the enclosure and apply the necessary VC server profile. The server can be provisioned without ever having to reconfigure the LAN and SAN. And if an administrator needs to recover a server due to a hardware issue, the server profile can be applied to a new slot in the enclosure and recovered to a new server blade. Again without having to change the LAN or SAN configuration, even with the change in underlying server hardware. With the rack and wire once design of the c7000 enclosure and Virtual Connect architecture, administrators can greatly simplify server provisioning and change events. Network configuration When deploying Exchange Server 2007 on rack mount servers, each server requires physical network connections for every network port in the server. This can result in a significant cable management issue when deploying a dense server environment and can result in increased unplanned downtime due to human error when working in the data center. With the HP BladeSystem, the customer has the flexibility to choose various networking options to meet the requirements of different Exchange designs. It is important to understand the networking capabilities of the individual HP server blades. The ProLiant BL460c and BL465c half-height server blades have 2 onboard 1Gb NICs, the BL480c and BL685c full-height blades have 4 onboard 1 Gb NICs. This provides redundant networking capabilities out of the box. Also, as described in the Management section above, each server blade also has a separate dedicated iLO 2 network interface that is consolidated within the enclosure through the Onboard Administrator module. Additional I/O mezzanine cards can then be added to the server blades to provide additional NICs as needed by the Exchange configuration. There are several types of mezzanine cards available for the server blades, including both a dual port and quad port network mezzanine card. The half-height server blades have two additional mezzanine slots while the full-height server blades have three additional mezzanine slots. (See Figures 14 and 15 above for an internal view of the mezzanine card locations on both a half-height and full-height server blades.) This provides the ability to scale up to a maximum of 12 NIC ports on a full-height server blade. One of the benefits of choosing a full-height over a half-height server blade is the increased I/O capabilities. Though as most Exchange administrators realize, there aren’t very many Exchange deployments requiring 12 network connections on an Exchange server! In order to support the onboard network ports, slots 1 and 2 in the rear of the enclosure must contain a network module (see Figure 13 above). A single module in slot 1 or slot 2 would provide half the network connections, and redundancy is provided via the second module in the adjacent slot. Adding additional network ports to the Exchange environment is then a two-step process. First, network mezzanine cards must be installed into the server blades. Second, corresponding network interconnect modules must be added to the corresponding I/O slots in the rear of the enclosure. There are three modules the customer can choose, a pass-thru, integrated switch, or VC module. At the basic level of configuration, a pass-thru patch panel can be installed into the enclosure providing 28 physical network cable connections out from the back of the enclosure into the data center and configured in the same manner as traditional deployments. This would eliminate any requirements for an integrated switch in the enclosure; however, this wouldn’t provide any cost or management benefits associated with reduced cabling. The Ethernet pass-thru patch panel modules would provide 1Gb uplinks for any network connections from the server blades. The second option for the customer is to utilize integrated Cisco or Blade Network Technology (BNT) network switches. Integrated switches provide a significant reduction in both the number of networking cables, and the setup time required to configure each server’s networking requirements. However, some companies do not want to utilize integrated switches in the HP BladeSystem enclosure as this can add complexity in managing the networking environment. In order to realize the benefits of both the pass-thru and integrated switch modules, customers can utilize the Virtual Connect modules. Virtual Connect provides customers with a third connection choice that combines the benefits of both the pass-thru and switch modules, and then some. VC helps Exchange administrators simplify the network configuration for the Exchange design. Using either the VC or switch modules, isolation of network traffic requires the utilization of virtual LANs, or VLANs, within the enclosure. The number of VLANs will depend on the specific Exchange customer requirements for the data center, but there are some general Exchange recommendations. For the Edge server role, traffic originating from and destined to an external source should be kept on a separate VLAN from traffic originating from and destined to an internal source. Thus, when using a half-height server blade (BL460/465c) for the Edge role, administrators should add an additional network mezzanine card to each Edge server and insert two additional I/O network modules in the rear enclosure. This will provide a total of 4 network ports, two per mezzanine card. The two onboard NICs can then be configured on the internal VLAN, and the two ports from the mezzanine card can be configured on the external VLAN. This will provide physical isolation between the internal and external networks as the traffic will be routed out of different physical switches in the back of the enclosure. If a BL480c full-height server blade is used for the Edge role then the four onboard network ports can be split between the internal and external VLANs. These four ports will route out through the same two switches in interconnect slots 1 and 2 with the VLAN configuration providing a logical data separation. To provide physical switch isolation for the full-height server blades, an additional mezzanine card and two additional network modules should be added to the servers and enclosure. The Edge server role brings up some interesting questions from a security standpoint. Some customers will not want to mix externally facing server roles (Edge, Microsoft Internet Security and Accelerator (ISA), etc) in the same enclosure as internally facing servers (AD, HT, etc). Even though there is a logical separation of network traffic using VLANs, network packets may be routed through the same physical switches depending upon the configuration. To provide complete physical isolation, customers can deploy any externally facing Edge servers in an isolated enclosure. Depending on the configuration requirements of the remaining Exchange servers in the enclosure, customers could also dedicate specific network interconnect modules to external and internal traffic using an additional mezzanine card as described above. Internal traffic would be routed through the onboard mezzanine cards connected to the modules in slots 1 and 2. External traffic would be routed through the additional mezzanine card and the network modules in another two slots. Another option for physical isolation would be to utilize a pass-through Ethernet module and directly connect the network cables to the core switches in the environment that are physically separated to route internal and external traffic. The remaining Exchange 2007 server roles would reside solely on the internal network. Specific VLANs should be created to support logical network traffic isolation where necessary. This may include a production, management, backup, and private heartbeat VLAN depending on the Exchange server role and configuration. Best practices for the production network VLAN would be to configure network teaming to provide load balancing between multiple NICs. Each HP server blade supports basic network teaming and NIC failover which can be automatically configured with RDP 29 scripting. However, to provide advanced networking intelligence on the server blades, HP provides the ProLiant Essentials Intelligent Networking Pack. INP enables ProLiant servers that are running basic ProLiant teaming software to adapt to and change the network path to achieve maximum reliability and performance. INP can monitor and analyze network conditions, detect and analyze network bottlenecks or broken network linkages, and redirect traffic to the optimum path. For more information on INP, visit http://h18004.www1.hp.com/products/servers/networking/teaming.html. The actual number of NICs, VLANs, and network modules necessary to support the customer’s Exchange configuration will vary greatly from deployment to deployment. In general, a base Exchange server will require between 2 and 4 NICs. Full-height server blades would require network modules in interconnect slots 1 and 2 to support the 4 onboard NICs. Half-height blades would require an additional dual-port NIC mezzanine card and 2 additional network interconnect modules in slots 3/4 or 5/6. Additional considerations such as Exchange clusters or network based backups may necessitate additional NIC ports for specific Exchange servers. The HP BladeSystem provides the flexibility to support these I/O requirements for the Exchange deployment. Storage configuration Storage design and performance is a critical component of any Exchange implementation and deployment. With the HP BladeSystem architecture flexibility, customers can leverage numerous storage attach options for Exchange from DAS to FC and iSCSI SAN storage arrays. Looking first at the internal storage options, each server blade has the capability to provide internal DAS storage via an integrated Smart Array (SA) controller and hot-swappable SFF SAS or SATA drives. The half-height server blades and the full-height BL685c server blade utilize the SA E200i controller and can support two internal SFF drives. The full-height BL480c server blade utilizes the SA P400i controller. The BL480c can support up to four internal drives, the BL685c can support up to two internal drives. Both the E200i and P400i controllers also support an attached battery-backed write cache (BBWC) controller which can significantly improve write performance. At a base level, this provides internal DAS storage for the OS and Exchange Server 2007 binaries files. On top of that, the Exchange Server 2007 Hub Transport and Edge server roles can also benefit from internal storage. These server roles require disk I/O and capacity to ensure proper message routing. Depending on the Exchange workload, the local Smart Array controller and BBWC may provide the necessary disk resources for these Exchange roles without requiring additional external storage. And to reinforce the importance of enabling the BBWC, preliminary testing on the Exchange Hub Transport server has shown substantial write performance degradation when the BBWC is not present. If additional storage is required for these server roles, one option is the HP SB40c storage blade. The SB40c storage blade provides an additional six SFF SAS or SATA drives to a directly attached server blade. With the SB40c and BL480c blades, administrators can scale up to ten SFF drives for the server configuration. The SB40c is the same size as the half-height server and consumes a single half-height blade slot. The SB40c storage blade provides an excellent solution for smaller Exchange Mailbox servers or Exchange Hub Transport and Edge servers that require additional disk resources. The document “HP StorageWorks SB40c 1,000 user Exchange 2007 Storage Solution” available at www.hp.com/solutions/activeanswers/exchange, describes the results of a Microsoft Exchange Solution Reviewed Program (ESRP) – Storage program V2.0 submission. This solution brief describes a tested configuration for deploying Microsoft Exchange Server 2007 Enterprise Edition (Exchange) in a 1,000 user environment. The performance results and best practices outlined in this document provide tested guidelines for configuring Exchange using the SB40c. For more details on the Microsoft ESRP – Storage program, please visit http://technet.microsoft.com/en-us/exchange/bb412164.aspx 30 Note: SATA drives are not recommended for high I/O or 24x7 workloads. It is highly recommended that SAS drives be utilized for Exchange Server 2007 configurations. For Exchange environments in which internal storage is not sufficient to meet the I/O or capacity demands of the solution, the HP BladeSystem can connect to either a FC or iSCSI (SAN). This is a common requirement to support the Exchange Server 2007 Mailbox server role and for many Exchange public folder requirements. For communication with an iSCSI SAN, the priority is to ensure that accessing the storage over the network is not a performance bottleneck. This entails verifying that enough bandwidth exists on the current network or configuring a separate, isolated data VLAN to ensure the highest levels of performance when accessing the SAN over the network. Depending on the server blade model, either 2 or 4 Gb NICs are embedded on the system board for network connectivity. If additional network bandwidth is required, additional NIC mezzanine and I/O interconnect modules can be added to the server blades and enclosure respectively. For communications with a FC SAN, the same general considerations as described above in the Network configuration section will apply. To configure a FC I/O fabric, a FC mezzanine card must be installed in the server blade and FC interconnect modules must be installed in the rear of the blade enclosure. The FC mezzanine cards for the server blades operate at a speed of 4 Gb. Again, the customer has several options of providing SAN connectivity from the back of the enclosure. A pass-thru module can be connected to the blade enclosure, providing a pass-thru connection for each HBA port. This would be the equivalent of connecting to a SAN in a rack-or tower-server environment, with a dedicated fibre cable running to each HBA. This increases the number of cables connected to the enclosure and the likelihood of damaging the delicate fibre cables. The second option is to utilize an integrated SAN switch directly in the enclosure. By utilizing either the Brocade 4Gb SAN Switch or Cisco MDS 9124e Fabric Switch, the customer can benefit from a shared SAN architecture within the enclosure, reducing the extensive cabling required for SAN connectivity, and the likelihood of unplanned downtime due to damaged cables. The third option is the HP Virtual Connect FC modules. This provides the same benefits as were outlined above and can be especially significant for FC domains. Depending on the number of SAN switches in the infrastructure, organizations may not be able to deploy SAN switches in the enclosure due to a limitation in the number of connected switches in the FC fabric domain. The VC FC module gets around this limitation since the module is a not a switch device; but still provides the benefits of cable reduction and provides a change-ready environment for server provisioning and recovery events. Performance Exchange Server 2007 performance is dependent upon the resources that are provided to the application. There is very little application tuning that is required to optimize Exchange Server 2007 performance. Performance is contingent upon proper sizing and configuration of the server, storage, and network resources. If one of those resources becomes a bottleneck, Exchange server performance will suffer. As noted in the Exchange Server 2007 roles-based architecture section below, performance requirements will vary depending upon the server role in the architecture. The common resource requirements when sizing an Exchange server are CPU, memory, storage requirements (I/O and capacity) and network I/O. A frequent misconception around the HP BladeSystem infrastructure is that server blades are low end, lower performing servers and only suitable as niche servers for web farms or front-end web services support. However, this 31 misconception is finally starting to unravel as customers are realizing the full breadth of the performance of the HP BladeSystem. Microsoft Exchange Server 2007 is only supported in production on either the AMD64 or Intel 64, x64-bit hardware platforms. Microsoft made this decision to enable the application to scale beyond the current memory limitations imposed by the Exchange 2003 32-bit architecture. All of the HP BladeSystem c-Class server blades support the Microsoft Windows Server x64-bit operating system that Exchange Server 2007 requires. With two and four processor servers, supporting dual-core and quad-core technology, HP server blades can support up to 8 processor cores in a single enclosure slot. This provides excellent performance in a dense form factor. The HP server blades can also scale up to support a maximum of 32 or 64 GB of memory (depending on the server model) to ensure that there is sufficient memory to support the 64-bit Exchange server roles. From a performance perspective, server blades are no different than a comparable DL or ML HP ProLiant server without the fans or power supplies. At the time of publishing, Microsoft had not finalized the benchmark process for Exchange Server 2007. However, looking at the latest Exchange Server 2003 benchmarks shows that the BL480c server blade holds the MMB3 (Microsoft Messaging Benchmark 3) world record at 13,50417. Now, the benchmark process is not indicative of a real world Exchange server deployment, but does provide a valid means to compare server to server performance. This result showcases the outstanding performance that server blades are capable of. Stay tuned for upcoming Exchange Server 2007 benchmarks. Server recovery Any server downtime can be costly for customers with mission-critical Exchange environments. The HP BladeSystem can provide a significant savings in the amount of unplanned downtime inside an organization versus rack and tower servers. This is in part linked to a reduction in the amount of human error in the data center with a consolidated cabling infrastructure. This is also in part due to the high degree of redundancy built into the HP BladeSystem design including redundant power supplies, cooling fans, management modules, and I/O fabrics. However, in the event of an Exchange server failure, there are a number of server recovery options available to the administrator to quickly respond and minimize system downtime. For Single Copy Clusters (SCC), the BladeSystem hardware is fully supported on the Microsoft cluster Windows Hardware Compatibility List or WHCL18 and supports Exchange clusters for application high availability in the event of server or application failure. Whether clustering is implemented or not, a strategy must exist that can be implemented for server recovery. If the OS needs to be recovered the base-level process is to approach server recovery from a new server standpoint and rebuild the server blade utilizing RDP-scripted installation techniques as overviewed in the Deployment section above. However, to reduce the downtime associated with application or server failure, the server recovery “rip and replace” technique can be utilized. If the OS is still intact the administrator has several options. The simplest option if the administrator has access to the data center is to simply move the internal disks from the failed server to a new server in the enclosure. If Virtual Connect is being utilized, the administrator could then reapply the server profile to the new server slot in the enclosure and the recovery process would be complete. However, this involves the manual process of moving the physical disks. Another approach is to implement an automated node recovery strategy to further reduce the downtime and administrator involvement in the recovery process. 17 http://www.microsoft.com/technet/prodtechnol/exchange/2003/performance.mspx18 For Windows Clusters, the entire solutions including storage components must be listed on the WHCL http://www.windowsservercatalog.com/32 Rip and replace “Rip and replace” is a server recovery process that leverages the awareness capabilities of HP server blades and RDP technology. Unlike HP rack mount or tower servers, HP blades have a locality based upon the slot that houses the blade in the blade enclosure. Once a server has been provisioned, RDP can be used to capture a server image of the system and a job is created that will deploy the server image. The deployment job is then set up to automatically execute whenever a new server is detected in that enclosure slot: thus, the term “rip-and-replace.” An administrator simply needs to remove the failed blade and insert a new blade into the enclosure slot and the RDP job to provision the server is automatically initiated. For example, once an Exchange server has the operating system, service packs, and hot fixes configured and deployed, a base image of the server can be captured with RDP and set up to deploy when a new server blade is detected in the enclosure. RDP can then be used to automate the Exchange deployment, and the remaining configuration steps can be completed by the administrator to set up the Exchange environment (Storage Groups, Databases, etc). In the event of a server failure, the administrator pulls out the failed blade, inserts a new blade into the same slot, and RDP automates the provisioning of the base server image to the server. Part of the RDP job that automatically executes can also include scripts to perform the installation of any additional hot fixes or patches that have been deployed post-Image-creation and to launch the Exchange Server 2007 recovery process to restore the Exchange configuration data from AD. This process is further simplified with the use of the Virtual Connect modules. In this case, the same server profile would still be applied to the new server. This means that the MAC and WWPN IDs would remain the same; thus, eliminating the need to reconfigure any LAN and SAN settings. Without VC, additional automation could be executed to configure new LAN and SAN options, such as selective storage presentation on the data LUNs for the HBA World Wide Name (WWN) IDs of the new server blade. Automated node recovery With a rip-and-replace server recovery strategy, downtime is still incurred from the point a server fails until the administrator is aware of the failure and the manual process of replacing the failed blade is completed and the RDP jobs are finished executing. With HP Insight Control and HP SIM managing the environment, an administrator can be notified via e-mail that a fault has occurred; however, someone still needs to go into the data center, and the failure may not occur during normal business hours. Automated node recovery allows customers to implement an instant and completely automated server recovery strategy by utilizing the concept of active and spare server blades. In the event that HP SIM detects a server failure, a spare server is automatically brought online and provisioned to replace the role of the failed server. This technology provides the ability to hardware “cluster” server nodes within an HP BladeSystem environment, independent of the Exchange application role, which can be implemented with either a local boot or boot-from-SAN (BFS) server configuration. In a BFS configuration, the recovery process can be completed in a matter of minutes. For example, in the event of an Exchange Server 2007 server failure detected by HP SIM, a spare server can be automatically powered on and provisioned to boot to the LUN of the failed server19, and given access to the Exchange data LUNs. In this case, the administrator can be notified after the event has occurred and does not incur the downtime of having to enter the data center to bring the Exchange servers back online. As with the “rip-and-replace” recovery strategy, VC can also simplify this recovery scenario as well. For example, when using VC in a BFS configuration, if a server in slot 1 were to fail, that server profile could be reapplied to a spare server in another slot in the enclosure. The new server would then have the MAC and WWPN IDs from the failed server and boot directly to the boot LUN on the SAN without having to script any HBA configuration changes. This process is documented for an 19 This would only work in the event that the OS itself is not corrupted. 33 Exchange Server 2007 mailbox server recovery scenario in the “HP Virtual Connect architecture and Microsoft Exchange Server 2007” white paper20. Best practices for choosing an HP server blade for Microsoft Exchange Server 2007 The “right” server architecture for an Exchange Server 2007 deployment will vary widely from customer to customer. Anyone that has ever been involved with designing the server hardware requirements for an Exchange deployment knows that the most common answer to the questions of “What type of servers should we deploy?” and “How many users can those servers support?” is ”It depends!” The answers to these questions can vary greatly based upon the customer’s definition of what an Exchange user is. For example, variables such as the number of users in the organization, the size of the user’s mailbox, the average number of messages a user will send and receive in a typical day, and the average size of the message will all factor into the equation to determine the performance requirements of the particular Exchange server roles. The next section, Exchange Server 2007 roles-based architecture, will discuss the requirements for the specific Exchange server roles and why a particular HP server blade is a good fit for the server role deployment. This will provide a good foundation for a discussion around sizing the particular server roles on specific server hardware. To further facilitate the sizing process for the customer, HP has developed a free Exchange Server 2007 sizing tool. This Exchange sizer provides server and storage hardware requirements based upon a set of input variables that help define the customer environment. This allows each customer to generate a recommend hardware configuration for their specific Exchange deployment. This tool is available in both online and downloadable form from the HP ActiveAnswers website, www.hp.com/solutions/activeanswers/exchange. The final section, Exchange Server 2007 example configurations, provide a set of example configurations for 5000, 10000, and 15000 users as generated by the Exchange Server 2007 sizing tool. These example configurations can help customers understand the flexibility of the HP BladeSystem and provide a foundation to build upon for designing an Exchange Server 2007 deployment on the HP BladeSystem. Exchange Server 2007 roles-based architecture Microsoft Exchange Server 2003 didn’t have an officially defined concept of server roles. There is a single base set of code that gets installed on every server installation. Best practices have been formulated over the years that have defined an informal set of Exchange 2003 server roles based on the separation of functions and services. For example, these include the Exchange back-end mailbox, the front-end, and the bridgehead server roles. However, to install a specific Exchange 2003 server role, the administrator first installs the Exchange binaries, and then configures the necessary Exchange and Windows settings as part of the post-configuration process. With Exchange Server 2007, Microsoft has formalized the server role concept and provided a more granular deployment model21. Exchange Server 2007 now consists of five distinct Exchange server roles including22: • Mailbox Server (Mbx) • Client Access Server (CAS) 20 White paper is available for download at www.hp.com/solutions/activeanswers/exchange. 21 When an Exchange 2007 server is deployed, the server role(s) is specified as input to the installation process. 22 Exchange Server 2007 is tightly integrated with Active Directory (AD) for user lookups, distribution list expansion, and configuration data storage and reference. AD servers should also be deployed on the HP BladeSystem infrastructure. 34 • Hub Transport Server (HT) • Edge Server • Unified Messaging Server (UM) For the majority of customer deployments, these server roles will be deployed in a multi-server configuration. This type of multi-tiered application deployment requires a server architecture design that can meet the varying performance demands and requirements of the different Exchange roles. The HP BladeSystem architecture provides this necessary flexibility, giving customers the choice of HP server blades to meet the performance demands of a complete multi-tiered Exchange deployment within a single infrastructure. Within a single enclosure, customers can mix and match two or four processor, Intel or AMD™ server blades, with either dual or quad core processors for additional performance in the same server footprint. But server choice goes beyond just meeting the CPU demands of the application. With the HP BladeSystem, customers can also choose server blades based upon the amount of memory, disk I/O or network bandwidth that the server role requires. This is particularly important for Exchange Server 2007 as the different server roles have unique performance requirements in terms of CPU, memory, and I/O. Figure 17 below highlights the general recommendations for matching the HP server blade models with a particular Exchange Server 2007 role. These guidelines are based upon typical server performance requirements and hardware features and should serve as a baseline reference for deployment recommendations. 35 Figure 17. Exchange Server 2007 role mapping to HP server blades Edge Server The Edge server role is a new role introduced in Microsoft Exchange Server 2007, designed to serve as the first line of defense for external mail flow into the organization. The Edge server is a stand-alone server and can not be configured on the same server as any other Exchange server roles. The server sits in the organization’s DMZ23 and provides Anti-SPAM and messaging hygiene capabilities for the Exchange organization. In general, as messages flow into the organization, the Edge server scans each email, and if a virus is detected or the message is deemed to be SPAM, the email is quarantined or deleted24. This process of SPAM and virus detection will consume both CPU and memory resources. In addition, the Edge server must be designed to support fast and efficient message queuing and routing which will also consume disk resources. Based on guidelines from Microsoft25, the recommended CPU and memory configuration for an Edge server is between 2 and 4 processor cores and one GB of memory for each core. This will generally be sufficient for most customer deployments. The disk sizing requirements on the other hand will vary depending on the message traffic footprint for the organization. Initial recommendations have ranged between 2 to 8 spindles for the Edge server disk I/O and capacity requirements26. 23 The term DMZ is derived from a military term, demilitarized zone, and in IT lingo refers to the area between the internal trusted network and external untrusted network 24 The specific function of the Edge server role is configurable and will vary based upon the customer requirements. 25 http://msexchangeteam.com/archive/2007/01/16/432222.aspx 26 See the blog on Edge and HT disk sizing for more information, http://msexchangeteam.com/archive/2007/02/26/435846.aspx 36 In situations where the Exchange deployment requires 2 spindles, the BL460c and BL465c dual-processor, dual-core servers would be the recommended HP server blade platform. The BL460/465c server blades are a half-height server models and can scale to 16 server blades in the enclosure. Both of the models contain an internal HP Smart Array E200i controller (with up to 128 MB of BBWC) and support for two internal hot-swappable SFF SAS drives. The BBWC controller is a very important component for improving disk performance for applications with local disk I/O requirements such as the Exchange Edge role. For customers looking to standardize on the BL460/465c, but requiring additional disk I/O or capacity for the Edge server, the HP SB40c storage blade can provide an additional six, SFF SAS or SATA drives, to a directly attached BL460/465c server blade. This will provide a maximum of 8 spindles for Edge server disk requirements. As noted above, SATA drives are not recommended for high I/O or 24x7 workloads. It is highly recommended that SAS drives be utilized for Exchange Server 2007 configurations. Depending on the specific DMZ configuration, external storage on a SAN could be provided as an alternative to the direct attached storage (DAS) provided by the storage blade. The second server option for the Edge server role is the HP BL480c server blade, a full-height server, with the ability to scale up to 8 servers in an enclosure. This server can also provide the four recommended cores in a dual-processor, dual-core configuration. However, this server supports additional internal storage. The HP BL480c contains an internal HP Smart Array P400i controller (with up to 256 MB of BBWC) and support for 4 internal SFF SAS drives. The HP SB40c could then be used to provide a total of 10 SFF SAS drives for Edge server deployments demanding very heavy disk resources. Edge Network considerations Given the location of the Edge server within the DMZ, the network design becomes extremely important, especially in cases where the DMZ and internal network servers are sharing the same physical HP BladeSystem enclosure. At a minimum, customers would require two NIC ports, one for internal and one for external communication. However, this would not provide any redundancy in the event of a NIC failure. Thus, most customers are likely to require four NIC ports for the Edge server role. This would come standard on the full-height server blades. With the half-height blades, an additional mezzanine card would be required to get an additional two NICs. There are several ways that customers may choose to architect their HP BladeSystem deployment. To provide complete physical isolation, customers may choose to place an entire HP BladeSystem enclosure in the DMZ which might include the Edge server, along with any other servers required to sit in the DMZ (ISA for example). If the Edge server is installed in an enclosure which also includes internal network servers, then the customer can configure virtual local area networks (VLANs) to logically isolate the network traffic of untrusted and trusted servers over a single group of I/O interconnect Ethernet modules (pass-thru, switch, or Virtual Connect modules). The customer could also decide to physically isolate the Edge server network traffic across different Ethernet modules than the internal network servers are utilizing.27 Another option would be to utilize Ethernet pass-through modules to physically isolate the network traffic across different physical Ethernet cables. Hub Transport Server The Hub Transport server is responsible for routing mail to the correct mailbox server in the Exchange environment. Every email message that is sent in the organization must traverse through a Hub Transport server. Thus, like the Edge server role, there is a disk I/O and capacity requirement associated with message routing and queuing that must be taken in to consideration when planning the HT hardware configuration. The HT server also plays a role if Microsoft’s Cluster Continuous 27 This was discussed in greater detail in the “I/O connectivity” section above. 37 Replication (CCR) technology is implemented for mailbox database replication. When CCR is utilized, the HT server can include a transport dumpster that puts additional overhead on disk performance and capacity. Given the similarities between the demands of the Edge and HT server, the recommendations for which blade to deploy for the HT server role are the same as the Edge server recommendations above. Microsoft is recommending up to 4 processor cores with 1 GB of memory per core. Thus, depending on the specific disk requirements for the customer profile, either a BL460/465c half-height or a BL480c full-height server blade, with an optional HP SB40c DAS storage blade, should be chosen as the HT server. This will satisfy the necessary CPU and memory requirements and provide between 2 and 10 internal DAS SFF SAS drives. As an alternative to the DAS storage blade, customers may satisfy the necessary storage requirements from an external storage subsystem using either an iSCSI or Fibre Channel storage array. This would eliminate the need for additional internal storage requirements beyond the OS and Exchange binaries. In this case, the half-height BL460/465c server blades would provide the best density and scalability for the Exchange organization. From a networking standpoint, two 1 Gb NICs would provide sufficient bandwidth and availability for the networking requirements of the HT role. Additional NIC ports may be required depending upon the complexity of the network configuration and the number of VLANs. The BL460/465c server blades would also be the server choice in the customer scenario in which the HT servers are configured in a boot-from-SAN (BFS) configuration. In a BFS scenario, any storage LUNs, including the OS/boot LUN, would be located on an external storage array, eliminating the requirement for local storage. Since there isn’t a requirement for local storage, the half-height servers will provide the best server density for the Exchange deployment. Client Access Server The Exchange 2007 CAS server handles connections to the mailbox server from all non-MAPI protocol clients including Outlook Web Access (OWA), RPC over HTTPS, Exchange ActiveSync (EAS), and POP3/IMAP. The Exchange 2007 CAS server was significantly redesigned from the Exchange 2003 front-end/backend protocol server architecture. OWA rendering and non-MAPI client (POP3/IMAP) protocol conversion have now been moved to the CAS server, offloading significant processing from the Mailbox role. This places an additional burden on the CAS server from a CPU and memory perspective. Microsoft recommends up to four processor cores and 1 GB of memory per processor core. And unlike the HT and Edge server roles, there is no requirement for additional local disk I/O. Thus, 2 internal spindles for the OS/Exchange binaries will provide sufficient disk resources for the CAS server role. Using the BL460/BL465c server blades will provide the necessary processor, memory, and disk resources for the CAS server role From a networking standpoint, two 1 Gb NICs would provide sufficient bandwidth and availability for the networking requirements of the CAS role. Additional NIC ports may be required depending upon the complexity of the network configuration and the number of physically isolated VLANs that are required. To provide high availability for the CAS server role, additional CAS servers can be added in a scale-out topology. DNS round robin or hardware-based load balancers could then be utilized to distribute the load between a pool of CAS servers that accept client connections from the Internet. In this case, server blades can be easily added to the HP BladeSystem enclosure if additional CAS processing is required. Unified Messaging Server Like the Edge server role, the Unified Messaging server role is a new role introduced with Microsoft Exchange Server 2007. The UM server provides a single point of integration between your voicemail, fax, and email, allowing users to dial in to read their email, and play voicemails back on their 38 computer. Since the UM server is brand new for Exchange, concrete sizing guidelines are still being developed for this server role. Microsoft recommends the same processor and memory configurations as the Edge, HT, and CAS server roles, a four processor core server with 1 GB of memory per core. And like the CAS server, there is not a significant requirement for local disk resources. Using the BL460/465c server blades will provide the necessary processor, memory, and disk resources for the UM server role. From a networking standpoint, two 1 Gb NICs would provide sufficient bandwidth and availability for the networking requirements of the UM role. Additional NIC ports may be required depending upon the complexity of the network configuration and the number of VLANs. Mailbox Server The Exchange Mailbox server role makes up the back-end of the multi-tiered Exchange layout. This server role hosts the databases and transaction logs containing the users’ data, and generally requires a higher level of performance then the other server roles. Microsoft’s recommendation for mailbox server sizing is based on the number of users that can be supported per server core. Currently the recommendation is 500 “Heavy” users and 1000 “Average” users per core. The profiles are defined based upon the average message traffic per day. An “Average” user profile is based on 10 messages sent and 40 messages received per day. A “Heavy” user profile is based on 20 messages sent and 80 messages received per day. Thus, when looking at designing the server hardware requirements for the Exchange Mailbox Server role, choice and flexibility are key factors. The HP BladeSystem infrastructure provides both. As Figure 17 indicates, any of the HP server blades can be utilized as Exchange Mailbox servers depending on the specific customer user profile(s) and number of concurrent users hosted on the server. The BL460c and BL480c server blades can support either dual-core or quad-core processors. Thus, either of these server blades can support 8 server cores. The BL685c is a four processor, dual-core server, also supporting 8 processor cores. The following general recommendations can serve as a starting point for determining the most appropriate server blade for the specific customer requirements: • For mailbox server configurations up to 2000 “Heavy” or 4000 “Average” users, the BL460c, BL465c, or BL480c dual-core processor server • For mailbox server configurations between 2000-3000 “Heavy” or 4000-6000 “Average” users, the BL460c or BL480c quad-core processor server • For mailbox server configurations between 3000-5000 “Heavy” or 6000-8000 “Average” users, the BL685c dual-core processor server As a result of the shift to an x64 server architecture, Exchange Server 2007 can and will utilize greater then 4 GB of server memory. Thus, it is also important when looking at the Exchange Mailbox server role to satisfy the memory demands along with the CPU requirements. The server memory recommendations are again based upon the user profile. Table 1 below provides these recommendations. Table 1. Exchange Mailbox server memory recommendations User Type Mailbox Memory Recommendation Light 2 GB + 2 MB/Mailbox Average 2 GB + 3.5MB/Mailbox Heavy 2 GB + 5MB/Mailbox 39 The current recommended maximum for memory is 32 GB. Above 32 GB, the price for memory outweighs the performance benefit of additional Exchange store cache. Each of the HP server blades can support up to 32 GB. However, getting to 32 GB will differ depending on the server model. The BL460c/BL465c server blades have 8 DIMM (dual in-line memory module) slots. To build a 32 GB server requires 4 GB DIMMs which are currently much more expensive then either 1 GB or 2 GB DIMMs. The BL480c has 12 DIMM slots and the BL685c has 16 DIMM slots. Thus, to get to 32 GB with the BL685c, the customer will only need 2GB DIMMs for the Exchange server. This trade-off in price and memory configuration in a blade environment is very important to consider when building the Exchange Mailbox server. From a networking standpoint, two 1 Gb NICs would provide sufficient bandwidth and availability for the production networking requirements of the Mailbox server role. However, there may be other networking requirements depending upon the complexity of the network configuration, the number of VLANs, and technologies such as CCR that might require additional NIC ports. These additional networking requirements may drive the customer to choose between the BL460c/BL465c and BL480c for the equivalent number of Exchange users. The full-height BL480c server blade will provide an additional 2 embedded NICs and an additional third mezzanine card if that is a requirement for the configuration. Keep in mind that these recommendations may change over time and will vary significantly as the user profile changes. It is highly recommended to run the Exchange Server 2007 sizing tool to generate a design based on the customer’s specific configuration. Active Directory Moving to Microsoft Exchange Server 2007 requires a shift in server architecture to support the x64-bit Windows operating system. HP has been shipping x64-bit capable servers for the past several years; but, depending on how old the customer’s current server hardware is, upgrading to Exchange may require an entire server upgrade. As with Microsoft Exchange Server 2003, Exchange Server 2007 is heavily reliant upon Active Directory for a significant amount of data: for example, server configuration information, recipient data, transport policies, etc. While it is not a requirement that the AD servers are running an x64-bit OS like Exchange, customers may realize significant benefits by moving to x64-bit AD servers. And the migration to Exchange Server 2007 may provide the perfect opportunity to redesign the AD infrastructure at the same time as the Exchange components. By installing AD on an x64-bit OS server, additional memory can be configured on the server beyond 4 GB. This additional memory will allow AD to cache more (if not all) of the AD database file (NTDS.DIT) in memory, depending on the size of the file. Accessing the NTDS.DIT file in memory can provide significant performance improvements on the AD server and allow the customer to reduce the number of AD servers supporting the Exchange environment. The BL460c/465c server blades can provide the necessary performance needed to host the AD server, helping customers further consolidate the messaging infrastructure on the HP BladeSystem. Multi-role servers The discussion around the specific Exchange roles thus far has focused on the case where the roles are deployed on dedicated server hardware. However, in many cases, multiple roles may be installed on the same server: for example, combining the CAS and HT server on the same machine. In these cases, the customer needs to analyze the requirements of all the server roles that may reside on the single server. Microsoft has different guidelines based on multiple server roles that are available at http://technet.microsoft.com/en-us/library/aa998874.aspx. 40 Exchange Server 2007 example configurations The following section provides example configurations for 5,000, 10,000, and 15,000 Exchange users. These reference configurations are based upon the recommended best practices from the proceeding section, along with recommendations from the Exchange Server 2007 sizing tool on the HP ActiveAnswers website. The purpose of these example configurations is to demonstrate how an organization can deploy Exchange Server 2007 using the HP BladeSystem infrastructure and HP StorageWorks arrays. These configurations demonstrate the flexibility of the HP BladeSystem architecture to scale from 5,000 to 15,000 Exchange users within a single enclosure, and also show the storage upgrade path a customer might follow as the number of Exchange users increase. The configurations also demonstrate various high availability options that can be utilized for Exchange Server 2007. Each of the example configurations is based upon the Microsoft Exchange “Heavy” user profile which generates 0.32 I/O operations per second per user (IOPS/u). The mailbox size is set to 600 MB. Both I/O and capacity overhead is factored into the configurations. For example, this includes a 20% peak I/O overhead, along with the additional capacity requirements for deleted item retention, white space, fluff factor, and storage capacity headroom. The solutions also include space for an Exchange maintenance LUN, separate from the production database and transaction log LUNs. There are no backup/restore components configured as part of these example configurations. Each of the example configurations utilizes the HP Virtual Connect modules to connect to the network and storage arrays. Core SAN switches are depicted in each of the examples, while additional network components are not displayed. The first example in Figure 18 is the 5,000 user example configuration. This configuration depicts a full multi-tiered Exchange Server 2007 role deployment. The reference architecture contains redundant Edge, HT, CAS, and AD servers. For the Exchange Mailbox server role, high availability is provided using Microsoft’s new data replication technology, CCR. 41 Figure 18. 5,000 user building block configuration The redundant Edge and HT servers are deployed on the BL480c with four internal SAS drives and BBWC to support the disk requirements of these server roles. The remaining server roles are deployed on the BL460c server blade for a complete Intel processor architecture configuration28. This example utilizes two, 2500 user CCR configurations on the BL460c server blades. Two HP StorageWorks 1500cs Modular Smart Arrays (MSA1500cs) storage arrays are used to satisfy the primary and secondary storage requirements. A complete bill of materials (BOM) listing the hardware components 28 Intel and AMD based servers can be mixed within the same enclosure. 42 for this example configuration can be found in “Appendix A – 5,000 user example configuration BOM”. The next example configuration, Figure 19, is for 10,000 Exchange users. This solution also demonstrates a complete multi-tiered Exchange Server 2007 deployment. There are redundant Edge, HT, CAS, and AD servers. To achieve high availability for the Mailbox server role, single copy clustering (SCC) is utilized. SCC is the traditional mode of clustering used to provide high availability in the event of a server failure. Figure 19. 10,000 user building block configuration To provide additional I/O and capacity for the Edge and HT servers, this example configuration utilizes the BL460c with a direct attached SB40c storage blade; yielding a total of 8 SFF SAS drives for these server roles. The AD, CAS, and Mailbox servers are also deployed on BL460c servers. This provides a solution that utilizes a single server model for all server roles. The four Exchange Mailbox servers are setup as part of a four node, active/active/active/passive cluster configuration. Each active mailbox server supports 3,333 (or 3,334) Exchange users with a single passive node in the event of a server failure. For external data storage including the cluster quorum drive and the Exchange Mailbox server database and transaction log requirements, an HP StorageWorks 6000A 43 Enterprise Virtual Array (EVA6000A) with 112, 300 GB 10K FC spindles is utilized. A complete BOM listing the hardware components for this example configuration can be found in “Appendix B – 10,000 user example configuration BOM”. The third example configuration in Figure 20 below is an all AMD based solution demonstrating 15,000 Exchange users. In this configuration, the Edge server role is not deployed within this enclosure. The customer could either install the Edge server role in a separate enclosure, or utilize a third-party or Microsoft Exchange Hosted Filtering29 solution for messaging hygiene. Figure 20. 15,000 user building block configuration 29 For more information on Microsoft Exchange Hosted Filtering please visit, http://technet.microsoft.com/en-us/exchange/bb288512.aspx 44 The redundant HT, CAS, and AD servers are deployed on BL465c server blades. With the increase to 15,000 users, a third HT and CAS server are configured in the enclosure to provide additional scalability and redundancy in the event of a single server failure. Three HP BL685c four-socket blades running 5,000 users each support the Mailbox server role. The fourth HP BL685c server blade is a standby Mailbox server. The solution leverages the Virtual Connect architecture for I/O communications. In the event of an active server failure on one of the Mailbox servers, the administrator can quickly change the VC profile to point to the standby server. This provides a level of server availability and recoverability without requiring Microsoft clustering. To provide external storage for the 15,000 user configuration, an HP EVA8000A with 168, 300 GB 10K FC spindles is utilized. This includes storage space for the Mailbox server role, as well as LUNs for the HT server. In this example, FC mezzanines were included in the HT servers to provide LUN access on the EVA, rather then using the BL480c or HP SB40c for local I/O. Depending on SAN I/O and available capacity, using the SAN may be a more cost effective option. A complete BOM listing the hardware components for this example configuration can be found in “Appendix C – 15,000 user example configuration BOM”. 45 Appendix A – 5,000 user example configuration BOM Quantity Part number Description HP BladeSystem Infrastructure 1 412152-B21 HP BLc7000 CTO Enclosure 4 399593-B22 HP BLc 1/10Gb VC-Enet Module Opt Kit 2 409513-B21 HP BLc 4G Virtual Connect FC Opt Kit 6 412138-B21 HP BLc7000 Encl Pwr Sply IEC320 Option 10 412140-B21 HP BLc Encl Single Fan Option 1 412142-B21 HP BLc7000 Encl Mgmt Module Option 1 413380-B21 HP BLc7000 3 PH NA/Jp FIO Power Mod Opt 1 417688-B21 HP BLc7000 FIO 16 Insight Ctrl License 1 433718-B21 HP BLc7000 10K Rack Ship Brkt Opt Kit Exchange Server 2007 Edge Servers 2 404707-B21 HP ProLiant BL480c G1 MOD-FX Svr 2 416673-L21 HP X5160 BL480c G1 FIO Kit 2 416673-B21 HP X5160 BL480c G1 Kit 2 405148-B21 HP SA P-Series 512MB SA BBWC Kit 8 431958-B21 146GB 10K SAS 2.5 HP HDD 4 397411-B21 HP 2GB FBD PC2-5300 2x1GB Kit Exchange Server 2007 Hub Transport Servers 2 404707-B21 HP ProLiant BL480c G1 MOD-FX Svr 2 416673-L21 HP X5160 BL480c G1 FIO Kit 2 416673-B21 HP X5160 BL480c G1 Kit 2 405148-B21 HP SA P-Series 512MB SA BBWC Kit 8 431958-B21 146GB 10K SAS 2.5 HP HDD 4 397411-B21 HP 2GB FBD PC2-5300 2x1GB Kit 46 Quantity Part number Description Exchange Server 2007 Client Access Servers 2 404667-B21 HP ProLiant BL460c G1 CTO Chassis 2 416660-L21 HP X5160 BL460c G1 FIO Kit 2 416660-B21 HP X5160 BL460c G1 Kit 4 431958-B21 146GB 10K SAS 2.5 HP HDD 2 406770-B21 HP BLc NC373m NIC Adapter Opt Kit 2 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 4 397411-B21 HP 2GB FBD PC2-5300 2x1GB Kit Microsoft Active Directory Servers 2 404667-B21 HP ProLiant BL460c G1 CTO Chassis 2 416660-L21 HP X5160 BL460c G1 FIO Kit 2 416660-B21 HP X5160 BL460c G1 Kit 4 431958-B21 146GB 10K SAS 2.5 HP HDD 2 406770-B21 HP BLc NC373m NIC Adapter Opt Kit 2 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 4 397413-B21 HP 4GB FBD PC2-5300 2x2GB Kit Exchange Server 2007 Mailbox Servers 4 404667-B21 HP ProLiant BL460c G1 CTO Chassis 4 435565-L21 HP X5355 BL460c G1 FIO Kit 4 435565-B21 HP X5355 BL460c G1 Kit 16 397413-B21 HP 4GB FBD PC2-5300 2x2GB Kit 8 431958-B21 146GB 10K SAS 2.5 HP HDD 4 406770-B21 HP BLc NC373m NIC Adapter Opt Kit 4 403619-B21 HP BLc QLogic QMH2462 FC HBA Opt Kit 4 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 47 Quantity Part number Description HP StorageWorks Infrastructure 2 AA986A MSA1500 controller shelf 2 218231-B22 HP MSA1000/1500 Controller 256 Cache All 4 254786-B21 HP 256MB Battery-Backed Cache Module 2 AA987A MSA 1500 Fibre Channel I/O Module 6 AA988A MSA 1500 Dual Channel SCSI I/O Module 8 302969-606 HP MSA30 S-B Rack FIO Enclosure 112 411089-B22 HP 300GB 15k U320 HDD 2 A7394A HP StorageWorks 4/32 SAN Switch Pwr Pack 20 221692-B21 Storage Works LC/LC 2m Cable 16 A7446B HP 4GB SW Single Pack SFP Transceiver HP Rack and Power Infrastructure 1 AF002A HP Universal Rack 10642 G2 Shock Rack 1 AF062A HP 10K G2 600W Stabilizer Kit 1 AF054A HP 10642 G2 Sidepanel Kit 2 340653-001 HP UPS3 KVA 20A 250V US Power Cord 2 252663-B24 HP 16A High Voltage Modular PDU 48 Appendix B – 10,000 user example configuration BOM Quantity Part number Description HP BladeSystem Infrastructure 1 412152-B21 HP BLc7000 CTO Enclosure 4 399593-B22 HP BLc 1/10Gb VC-Enet Module Opt Kit 2 409513-B21 HP BLc 4G Virtual Connect FC Opt Kit 6 412138-B21 HP BLc7000 Encl Pwr Sply IEC320 Option 10 412140-B21 HP BLc Encl Single Fan Option 1 412142-B21 HP BLc7000 Encl Mgmt Module Option 1 413380-B21 HP BLc7000 3 PH NA/Jp FIO Power Mod Opt 1 417688-B21 HP BLc7000 FIO 16 Insight Ctrl License 1 433718-B21 HP BLc7000 10K Rack Ship Brkt Opt Kit Exchange Server 2007 Edge Servers 2 404667-B21 HP ProLiant BL460c G1 CTO Chassis 2 416660-L21 HP X5160 BL460c G1 FIO Kit 2 416660-B21 HP X5160 BL460c G1 Kit 2 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 16 431958-B21 146GB 10K SAS 2.5 HP HDD 4 397411-B21 HP 2GB FBD PC2-5300 2x1GB Kit 2 434879-B21 HP BLc SB40c CTO Storage Blade Exchange Server 2007 Hub Transport Servers 2 404667-B21 HP ProLiant BL460c G1 CTO Chassis 2 416660-L21 HP X5160 BL460c G1 FIO Kit 2 416660-B21 HP X5160 BL460c G1 Kit 2 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 16 431958-B21 146GB 10K SAS 2.5 HP HDD 4 397411-B21 HP 2GB FBD PC2-5300 2x1GB Kit 2 434879-B21 HP BLc SB40c CTO Storage Blade 49 Quantity Part number Description Exchange Server 2007 Client Access Servers 2 404667-B21 HP ProLiant BL460c G1 CTO Chassis 2 416660-L21 HP X5160 BL460c G1 FIO Kit 2 416660-B21 HP X5160 BL460c G1 Kit 4 431958-B21 146GB 10K SAS 2.5 HP HDD 2 406770-B21 HP BLc NC373m NIC Adapter Opt Kit 2 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 4 397411-B21 HP 2GB FBD PC2-5300 2x1GB Kit Microsoft Active Directory Servers 2 404667-B21 HP ProLiant BL460c G1 CTO Chassis 2 416660-L21 HP X5160 BL460c G1 FIO Kit 2 416660-B21 HP X5160 BL460c G1 Kit 4 431958-B21 146GB 10K SAS 2.5 HP HDD 2 406770-B21 HP BLc NC373m NIC Adapter Opt Kit 2 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 4 397413-B21 HP 4GB FBD PC2-5300 2x2GB Kit Exchange Server 2007 Mailbox Servers 4 404667-B21 HP ProLiant BL460c G1 CTO Chassis 4 435565-L21 HP X5355 BL460c G1 FIO Kit 4 435565-B21 HP X5355 BL460c G1 Kit 16 397413-B21 HP 4GB FBD PC2-5300 2x2GB Kit 8 431958-B21 146GB 10K SAS 2.5 HP HDD 4 406770-B21 HP BLc NC373m NIC Adapter Opt Kit 4 403619-B21 HP BLc QLogic QMH2462 FC HBA Opt Kit 4 351580-B21 HP SA 641/642/E200 128MB BBWC Kit 50 Quantity Part number Description HP StorageWorks Infrastructure 1 AF012A HP Universal Rack 10636 G2 Shock Rack 1 AD556B HP EVA6000-A 2C4D Array 4 AD542B HP M5314B FC Drive Enclosure 112 364622-B23 HP EVA 300GB 10K Fibre Channel HDD 2 252663-D72 HP 24A High Voltage US/JP Modular PDU 1 AF062A HP 10K G2 600W Stabilizer Kit 1 AF056A HP 10636 G2 Sidepanel Kit 1 T3724D HP Co