Virtualization in a SAN

Redbooks Paper Jon Tate Virtualization in a SAN Introduction Successful businesses require real-time responsiveness to change — whether it is due to new customer needs, changes in the supply chain, unexpected competitive moves, external threats, or changes in the economic climate. Rapid response to change requires an IT infrastructure that can turn information into a competitive advantage; the IT infrastructure must provide maximum benefit at an affordable cost, and must have the flexibility to support changes in business processes. An on demand operating environment provides a cost effective and flexible IT environment. With information at the heart of competitiveness, storage becomes an ever more critical component of an on demand operating environment. The IBM TotalStorage strategy addresses some of the most pressing needs currently facing Chief Information Officers (CIO) and IT managers. As part of its strategy, IBM intends to deliver industry leading technologies that will help dramatically reduce the total cost of ownership (TCO) for storage, and help turn fixed costs into variable costs that scale with business volume. Success in the on demand world will depend on the ability to leverage information technology. A greater dependence on information means a greater dependence on storage. What differentiates an on demand business is the ability to quickly sense and rapidly respond to a dynamic marketplace — to do this there are challenges that an on demand business must overcome. © Copyright IBM Corp. 2003. All rights reserved. ibm.com/redbooks 1 At the business level, customers are faced with three major storage challenges: Managing storage growth: Storage needs continue to grow at over 50% per year. Managing storage is becoming more complex than ever, as we now have to deal with multiple server platforms and different operating systems, which may be connected to a storage area network (SAN) with multiple and diverse storage platforms. Increasing complexity: Although the declining cost of storage per megabyte makes it attractive to add additional disks, the increasing complexity of managing this storage results in over-utilized staff and under-utilized IT resources. Combining this with the shortage of skilled storage administrators, it is possible to add significant cost and introduce risk to storage management. Maintaining availability: The added complexity of 24x7 environments significantly reduces, for example, the efficiency of conducting routine maintenance, scheduling backups, data migration, and introducing new software and hardware. This problem is compounded by the fact that as availability increases, so does the cost inherent with making it so. These challenges still exist, although large SANs do offer desirable and tangible benefits, for example, better connectivity, improved performance, distance flexibility, and scalability. Yet even these benefits may be outweighed by the added complexity that they introduce. As an example, large enterprise SANs often contain different types of storage devices. These differences could be in the types of disk deployed, their level of performance, or the functionality provided such as RAID or mirroring. Often customers have different vendor storage devices as the result of mergers or consolidations. The result, however, is that storage and SAN administrators need to configure storage to servers, and then keep track of which servers own or have access to, that storage. The storage administrative tasks can become daunting as the SAN grows and as the storage administrators manually attempt to manage the SAN. Furthermore the complexity of different file systems in the same SAN requires that storage administrators know how to administer each client operating system (OS) platform. The management interfaces for each may be different, as there is no common standard that all vendors adhere to. Lastly, since the file systems are tied to each of the servers, storage management functions potentially have to be run on hundreds of servers. It is easy to see why manageability and interoperability are the top areas for concern, especially in a SAN where the number of possible storage and OS platform permutations are considerable. These challenges are at odds with the commonly held belief that storage is decreasing in cost per megabyte. It is clear that the cost of managing storage is 2 Virtualization in a SAN greater than the initial purchase price. A strategy is needed to address storage manageability, while at the same time addressing the need for interoperability. This strategy is the IBM TotalStorage Open Software Family. This strategy represents the next stage in the evolution of storage networking. It affords you the opportunity to fundamentally improve your company’s effectiveness and efficiency in managing its storage resources. With the IBM SAN virtualization products you are witnessing IBM deliver on its continued promise to provide superior on demand solutions that will assist in driving down costs, and reduce TCO. In this redpaper we describe the IBM technology and initiatives that form the IBM approach: SAN Volume Controller: A virtualization appliance solution that maps virtualized volumes visible to hosts and applications to physical volumes on storage devices. All servers being served by the SAN could be connected to the SVC. This may include all servers on the SAN or only a subset. This enables the system administrators to view, access and control a common pool of storage on a SAN so they can use storage resources more efficiently. Each server within the SAN has its own set of virtual storage addresses which are mapped to a physical address. If the physical addresses change, the server continues running using the same virtual addresses it had before. This means that volumes or storage can be added or moved while the server is still running. IBM's virtualization technology improves management of information at the “block” level in a network — enabling applications and servers to share storage devices on a network. SAN Volume Controller Storage Software for Cisco MDS 9000: A joint project between IBM and Cisco, where IBM provides the virtualization software (SAN Volume Controller), and Cisco provides the hardware platform (MDS 9000), and the Cisco MDS 9000 Caching Services Module (CSM). A virtualization solution that creates a pool of managed disks from the attached storage subsystems, which are then mapped to a set of virtual disks for use by various attached host computer systems. The system administrators can view and access a common pool of storage on the SAN which allows them to use storage resources more efficiently, and provides a common base for advanced functions similar to those provided by the SAN Volume Controller. SAN Integration Server A solution designed to integrate IBM’s virtualization technology, Fibre Channel switches and Redundant Array of Independent Disks (RAID) storage technologies into a preconfigured, complete solution. Delivered and installed as an integrated unit, it offers upgrade options for connectivity, storage capacity and performance levels. The solution has been developed to provide the benefits of SANs with faster installation and configuration along with ease of manageability. The SAN Integration Server provides centralized management through a single interface to support easier Virtualization in a SAN 3 storage allocation and address application demands. This flexibility provides the benefit of better storage utilization by reducing or eliminating the problem of unused storage found in direct attached storage implementations, and reducing required administrative time and resources. Like the SAN Volume Controller, IBM has designed and tested the SAN Integration Server to integrate into existing environments, including heterogeneous hardware and operating systems. SAN File System: A SAN-wide file system, based on Storage Tank technology, for accessing data on storage networks across multiple application OS platforms and heterogeneous storage devices. The SAN File System also provides centralized, policy-based management of the data in the SAN. With the SAN File System, all of the files owned by the servers are made visible to clients of the SAN File System as a single file system in the SAN. This means that all files in the SAN can be made accessible to all of the servers, if the customer chooses. This eliminates the need to maintain copies of the same file for use by multiple servers. Because there is a central catalog of all files in the SAN, policies can be established for each file in areas such as file placement, security, and service level requirements. With the SAN File System, the storage administrators do not need to assign storage volumes to individual servers. By not having to partition the storage across the application servers, there can be more efficient use of the storage and less storage administration required. For those familiar with DFSMS in the zOS environment, think of the SAN File System as the open systems version of DFSMS. Multiple Device Manager: Planned for availability in 2004, the Multiple Device Manager is software that has been designed to enable administrators to manage SANs and storage from a single console. This software solution is designed specifically for managing networked storage components based on the SMI-S, such as: – – – – IBM TotalStorage SAN File System IBM TotalStorage SAN Integration Server IBM TotalStorage SAN Volume Controller IBM Enterprise Storage Server® (ESS) The IBM multiple device management solution is built on IBM Director, a comprehensive server management solution. Using Director with the multiple device management solution enables administrators to consolidate the administration of IBM storage subsystems and provide advanced storage management functions (including replication and performance management) across multiple IBM storage subsystems. It interoperates with SAN Management and Enterprise System Resource Manager (ESRM) products from IBM and with products from other vendors, including IBM Tivoli® Storage Resource Management (SRM) products and SAN Management products from other vendors. 4 Virtualization in a SAN These virtualization products are part of IBM's commitment to the open standards adopted by the Storage Networking Industry Association (SNIA). They implement standard CIM-based API's to allow management applications from IBM, and other vendors to administer and monitor their activities. In this redpaper we also describe the customer and industry trends that have shaped IBM’s approach to the development of these products, and IBM’s ability to provide a complete virtualization solution. Storage networking technology In the early 1980's, data sharing emerged as a critical necessity for improving efficiency within enterprises. This term is used somewhat loosely. It is sometimes interpreted to mean the replication of files or databases to enable two or more users, or applications, to concurrently use separate copies of the data. The applications concerned may operate on different host platforms. Data sharing may also be used to describe multiple users accessing a single copy of a file. This could be called true data sharing. In a homogeneous server environment where all servers run the same OS, with the appropriate application software controls (for example, a clustered file system), multiple servers may access a single copy of data stored on a consolidated storage subsystem. If attached servers are heterogeneous platforms (for example, a mix of UNIX and Windows), sharing of data between such unlike operating system environments is complex. The consolidation and management of information is an important requirement to effectively share information. In the late 1990s storage networking emerged in the form of SANs, Network Attached Storage (NAS), and Internet Small Computer System Interface (iSCSI). These technologies were aimed at reducing the TCO of storage by managing islands of information among heterogeneous environments with disparate operating systems, data formats, and user interfaces in a more efficient way. SANs enable you to consolidate storage and share resources because storage capacity can be connected to multiple servers, and at a greater distance. By separating storage resource management from individual hosts, a SAN enables disk storage capacity to be consolidated. The results can be lower overall costs through better utilization of the storage, lower management costs, increased flexibility, and increased control. This consolidation can be achieved physically or logically. Physical consolidation Data from disparate storage subsystems can be combined within shared disk arrays, which may be located at some distance from the servers. The capacity of these disk arrays can be shared by multiple servers, and the array capacity may Virtualization in a SAN 5 be partitioned and zoned, so that each server has access to an appropriate portion of the available gigabytes. Users may also benefit from the advanced functions typically offered with such subsystems. For example, RAID capabilities, remote mirroring, and instantaneous data replication functions might not be available with smaller, direct-attached disks. Available capacity can be dynamically allocated to any server requiring additional space. Capacity not required by a server application can be reallocated to other servers. Dynamic capacity allocation avoids the inefficiency associated with free disk capacity attached to one server not being usable by other servers. Extra capacity may be added, in a non-disruptive manner. Physical consolidation does not mean that all wasted space concerns are addressed, but it is a significant step forward in available capacity management. Logical consolidation It is possible to achieve shared resource benefits from the SAN without moving existing equipment. A SAN relationship can be established between a client and a group of storage devices that are not physically co-located (excluding devices that are internally attached to servers). A logical view of the combined disk resources may allow available capacity to be allocated and reallocated between different applications running on distributed servers, achieving better utilization. Step into the future Many contemporary authors state that storage has become a commodity. This implies that people want to be able to simply use storage without limitations or worries, to completely disregard its whereabouts and management, yet always be sure of its abundance and availability. At the same time storage costs have been steadily decreasing, and new ways of connecting storage devices have been implemented. The volume of data storage required in daily life and business has exploded. Each year capacity is growing by 50% and hardware cost is decreasing by 30%, but availability requirements are approaching 100%. Users are mobile, access patterns are unpredictable, and the content of data is more interactive. Storage itself may well be treated as a commodity — however the management of it is certainly not. It has been estimated that the cost of managing storage can be up to eleven times the cost of the storage itself. Storage virtualization addresses the increasing complexity of managing storage and will reduce the associated costs dramatically. Its main purpose is the full exploitation of the benefits promised by a SAN. Virtualization will become an enabler for sharing data, ensuring higher availability, providing disaster 6 Virtualization in a SAN tolerance, and improving performance. Virtualization will allow for consolidation of resources, provide policy-based automation, and enable several other benefits, which do not automatically result from the implementation of SAN hardware components. Storage virtualization in the SAN The Storage Networking Industry Association (SNIA) defines storage virtualization as: “The act of integrating one or more (back end) services or functions with additional (front end) functionality for the purpose of providing useful abstractions. Typically virtualization hides some of the back-end complexity, or adds or integrates new functionality with existing back end services. Examples of virtualization are the aggregation of multiple instances of a service into one virtualized service, or to add security to an otherwise insecure service. Virtualization can be nested or applied to multiple layers of a system.” Or, to put it in more practical terms, storage virtualization is the pooling of physical storage from multiple network storage devices into what appears to be a single storage device that is managed from a central console. Storage virtualization techniques are becoming increasingly more prevalent in the IT industry today. Storage virtualization forms one of several layers of virtualization in a storage network, and can be described as the abstraction from physical volumes of data storage to a logical view of data storage. This abstraction can be made on several levels of the components of storage networks and is not limited to the disk subsystem. Storage virtualization separates the representation of storage to the operating system (and its users) from the actual physical components. Storage virtualization has been represented, and taken for granted in the mainframe environment for many years. The SAN is making it easier for customers to spread their IT systems out geographically, but even in networks, different types of servers that use different operating systems do not get the full benefit of sharing storage. Instead, the storage is partitioned to each different type of server, which creates complex management and inefficient use of storage. When storage must be added, applications are often disrupted. At the same time, the reduced cost of storage and the technology of storage networks, with faster data transfer rates, have enabled customers to use increasingly sophisticated applications, such as digital media. This has caused even greater complexity and difficulty of management as Virtualization in a SAN 7 the amount of storage required grows at unprecedented rates. The IBM storage strategy introduces ways to eliminate these problems. Block virtualization at the LUN level Virtualization manages multiple storage devices and volumes as groups. These groups are managed independently of the physical layout of the storage. Because of this independence, new disk systems can be added to a storage network, and data can be migrated to them without causing disruption to applications that use the storage. Since the storage is no longer controlled by individual servers, it can be used by any server as needed. Capacity can be added or removed on demand without affecting the application servers. Storage virtualization simplifies storage management and reduces the cost of managing the SAN environment. Levels of storage virtualization We will define the different levels that virtualization can be achieved at in a storage network, as illustrated in Figure 1. SUN Solaris IBM AIX HP-UX Windows Server level SAN File system level Fabric level Storage subsystem level Figure 1 Levels that storage virtualization can be applied at Server level Abstraction at the server level is by means of the logical volume management of the operating systems on the servers. At first sight, increasing the level of 8 Virtualization in a SAN abstraction on the server seems well suited for environments without storage networks, but this can be vitally important in storage networks too. Fabric level At the fabric level, virtualization can enable the independence of storage pools from heterogeneous servers. The SAN fabric would be zoned to allow the virtualization appliances to see the storage subsystems, and for the servers to see the virtualization appliances. Servers would not be able to directly see or operate on the storage subsystems. Storage subsystem level Disk storage systems can provide some level of virtualization already by subdividing disks into smaller virtual drives. Conversely, more storage devices could be consolidated together to form one large virtual drive. RAID subsystems are an example of virtualization at the storage level. Storage virtualization can take this to the next level by enabling the presentation, and the management, of disparate storage systems. File system level Virtualization at the file system level provides the highest level of virtual storage. It can also provide the highest benefit, because it is data that is to be shared, allocated, and protected; not volumes. Server, fabric, storage subsystem or file system virtualization? As we stated, virtualization can be implemented at any of these levels. The IBM strategy is to move the storage device management intelligence out of the server, reducing the dependency of having to implement specialized software, like Logical Volume Managers (LVM), at the server level. We also intend to reduce the requirement for intelligence at the storage subsystem level, which will decrease the dependency on having to implement intelligent storage subsystems. By implementing at a fabric level, storage control is moved into the network, which gives the opportunity to all for virtualization, and at the same time reduces complexity by providing a single view of storage. The storage network can be used to leverage all kinds of services across multiple storage devices, including virtualization. By implementing at a file system level, file details are effectively stored on the storage network instead of in individual servers. This design means the file system intelligence is available to all application servers. Doing so provides immediate benefits: a single namespace and a single point of management. This eliminates the need to manage files on a server by server basis. Virtualization in a SAN 9 Virtualization models In-band and out-of-band models can be drawn for storage virtualization, as illustrated in Figure 2. These models are not mutually exclusive. In many environments a combination of both may be desired. Server Server Server Server Server Server Data + Control Data Control Storage Storage In-band Figure 2 In-band and out-of-band models Out-of-band In-band When we implement an in-band virtual storage network, both data and control flow over the same path. Levels of abstraction exist in the data path, and storage can be pooled under the control of a domain manager. In general, in-band solutions are perceived to be simpler to implement, especially because they do not require special software to be installed in servers (other than conventional multi-pathing software). In-band solutions can also provide caching and advanced functions within the storage network. This can help to improve the performance of existing disk systems and can extend their useful life, and reduce the cost of new storage capacity by enabling the use of lower function, lower cost disk systems, without the loss of performance. Other advantages include: Ability to offload function from the host Providing storage management for the SAN Performing performance optimizations in the data path Supporting host systems not in a cluster Supporting multiple heterogeneous hosts Integrating well with storage management software Releasing the customer from a particular vendor’s storage 10 Virtualization in a SAN Integrating with storage to create a better management picture Offering excellent scalability The SAN Volume Controller is an example of an in-band solution. Out-of-band In an out-of-band implementation, the data flow is separated from the control flow. This is achieved by separating the data and meta-data (data about the data) into different places. Out-of-band virtualization involves moving all mapping and locking tables to a separate server (the meta-data controller) that contains the meta-data of the files. In an out-of-band solution the servers request authorization to data from the meta-data controller, which grants it, handles locking, and so on. Once they are authorized, servers access the data directly without any meta-data controller intervention. Once a client has obtained access to a file, all I/O will go directly over the SAN to the storage devices. For many operations, the meta-data controller does not even intervene. Separating the flow of control and data in this manner allows the I/O to use the full bandwidth that a SAN provides, while control could go over a separate network or routes in the SAN that are isolated for this purpose. This results in performance that is nearly equal to local file system performance with all of the benefits and added functionality that comes with an out-of-band implementation. Other advantages include: Releasing the customer from a particular vendor’s storage Providing storage management for the SAN Offering excellent scalability Offloading host processing Supporting storage management from multiple vendors Integrating well with storage management software Supporting multiple heterogeneous hosts Relatively low overhead in the data path The SAN File System is an example of an out-of-band virtualization implementation. Marketplace storage architectural directions In this section we describe the architectural influences and standards that are driving the IBM TotalStorage Software Strategy. Virtualization in a SAN 11 Storage Network Industry Association The Storage Network Industry Association (SNIA) was incorporated in December 1997 as a nonprofit trade association that is made up of over 200 companies. SNIA includes well established storage component vendors as well as emerging storage technology companies. The SNIA mission is “to ensure that storage networks become efficient, complete, and trusted solutions across the IT community”. The SNIA vision is to provide a point of cohesion for developers of storage and networking products in addition to system integrators, application vendors, and service providers for storage networking. SNIA provides architectures, education, and services that will propel storage networking solutions into the broader market. The SNIA Shared Storage Model IBM is an active member of SNIA and fully supports SNIA’s goals to produce the open architectures, protocols, and APIs required to make storage networking successful. IBM has adopted the SNIA Storage Model and we are basing our storage software strategy and roadmap on this industry-adopted architectural model for storage, as depicted in Figure 3. 12 Virtualization in a SAN The SNIA Storage Model Application File/record subsystem Database (dbms) File system (FS) Redundancy mgmt (backup, …) Resource mgmt, configuration High availability (fail-over, …) Services subsystem Storage domain Discovery, monitoring Block aggregation SN-based block aggregation Device-based block aggregation Storage devices (disks, tape, etc.) Block subsystem Copyright 2000, Storage Network Industry Association Figure 3 The SNIA Storage Model IBM is committed to deliver the best-of-breed products in all aspects of the SNIA storage model: Block aggregation File/record subsystems Storage devices/block subsystems Services subsystems In the area of block aggregation, IBM provides the SAN Volume Controller, the SAN Volume Controller for Cisco MDS 9000, and the SAN Integration Server, implemented in an in-band model. In the area of file/record subsystems, IBM provides the SAN File System, a SAN wide file system implemented in an out-of-band model. Both of these solutions adhere to open industry standards. You can learn more about the SNIA Shared Storage Model by visiting the SNIA Web site (available at http://www.snia.org). Security, billing Host-based block aggregation Capacity planning Virtualization in a SAN 13 The IBM approach Deploying a storage network requires many choices. Not only are there SAN and NAS to consider, but also other technologies such as iSCSI. The choice of when to deploy a SAN or use NAS continues to be debated. CIOs and IT professionals must plan to ensure that all of the components from multiple storage vendors will work together in a virtualization environment to enhance their existing storage infrastructures, and/or build new infrastructures, while keeping a sharp focus on business efficiency and business continuance. In order to maximize the full potential of the storage network and lay the foundation for an on demand storage environment, the storage infrastructure must be: Integrated – so IT resources can be expanded quickly and easily Open – to enable growth and integration by leveraging existing assets, and to allow products from different vendors to be used together in cost effective manner Virtualized – so complexity can be reduced by treating storage and IT resources as a single, common pool of resources Autonomic – so management can be simplified and staff productivity improved The IBM TotalStorage software strategy is intended to address each of these requirements. Integrated Storage for an on demand storage operating environment must provide the flexibility, responsiveness, and cost effectiveness needed for businesses to be successful and requires an architected approach that is comprehensive and modular. The SNIA Shared Storage Model illustrates this point. An effective storage infrastructure must address the entire data path from the application to the actual storage devices. All of the components in the data path must be wrapped in management software to support proper operation of the system as a whole. An integrated storage infrastructure can help customers improve the sharing of information that is needed to efficiently integrate applications and businesses processes. IBM’s strategy is to take a comprehensive, holistic approach that addresses all layers in the shared storage model: block subsystem, block aggregation, file/record subsystem, and the services (management) subsystem. Storage in an on demand operating environment can only provide significant reductions in total cost and improvements in organizational agility when all areas described in the 14 Virtualization in a SAN shared storage model can be addressed in an architected, modular and integrated manner. Open Storage for an on demand operating environment must provide customers with the flexibility to exploit the assets they have today. First, customers must have the flexibility to purchase components from a variety of vendors, both IBM and non-IBM. Second, they must have the ability to have all of the components to work together efficiently and cost-effectively. IBM’s storage strategy is to deliver products based on open standards. Products based on open industry standard interfaces can offer customers significantly more benefits than products based on proprietary interfaces. The history of the information technology industry has shown by and large that open systems offer three key benefits: Better solutions at a lower price. By harnessing the resources of multiple companies, more development resources are brought to bear on common customer requirements such as ease of management. Improved interoperability. Without open standards, every vendor would need to work with every other vendor to develop interfaces for interoperability, thus leading to very complex product interdependencies that are difficult for customers to install, configure, and maintain. Customer choice. By complying with standards developed jointly — products interoperate with each other, preventing vendors from locking in customers to their platform despite customer’s changing needs. Choice helps provide customers with more flexibility and improve competition among vendors. Today, IBM storage products adhere to a number of industry standards for connectivity and IBM is enabling a number of its products to adhere to the SNIA Storage Management Initiative Specification (SMI-S). IBM will provide SMI-S support for our new infrastructure products including our block aggregation and file aggregation offerings. These products are being designed to be managed with integrated device management software, IBM Tivoli Software, and other non-IBM management software, that support the SMI-S standard. Virtualized and autonomic Storage for an on demand operating environment must provide the flexibility to respond to changes in business processes and assist customers in lowering total costs. Improving organizational agility and lowering costs in a storage environment can be accomplished in two ways. First, by providing virtualization capabilities, complexity can be reduced by treating storage and IT resources as Virtualization in a SAN 15 single, common pool of resources. Second, autonomic capabilities can be used to help simplify management and improve staff productivity. Critical components of today’s storage environments were designed in the era of direct attached storage. While these components can effectively use SANs for connectivity they do not exploit the potential capabilities of being networked resources. By not exploiting the SAN, access to data is limited and data are not able to be fully shared. Lack of sharing can result in disruptive and time consuming data migrations. Under-utilized storage in one system can not be dynamically reallocated to compensate for out of space conditions in another. Changes in the physical infrastructure often require interruptions to application processing and must take place during planned downtime. Management is often performed on a device by device basis, impacting personnel productivity. Maximizing the potential benefits of networked storage and removing today’s limitations can be accomplished by adding intelligence to the SAN. By providing virtualization and file system capabilities in the SAN, these capabilities can now be made available to all application servers and storage devices on the SAN. With virtualization in the SAN, and a common SAN file system, application servers can be insulated from changes in the underlying storage infrastructure. Both volume and file data can be moved without impacting operations. Planned and unplanned outages can be reduced. With a common point of control in the SAN, storage can be more effectively utilized and managed; complexity is reduced and operations become streamlined. The IBM strategy for creating an on demand storage environment is based on adding intelligence and value to the storage network. This approach adds the ability to make, dynamic, non-disruptive changes across heterogeneous servers and storage devices. This approach can also significantly improve access to and management of stored data. IBM is delivering on this strategy with the IBM TotalStorage Open Software Family. The importance of a highly reliable, high performance hardware solution such as the IBM TotalStorage Enterprise Storage Server (ESS) that is the guardian of mission-critical data for a business, is still a cornerstone concept. However, software is emerging as a critical element of any SAN solution. Management and virtualization software provide advanced functionality for administering distributed IT assets, maintaining high availability, and minimizing downtime. To avoid the shortcomings of a server based or storage subsystem based virtualization, the function must be placed in the network itself. Adopting this approach means that the function can be applied to all the storage and the storage pool shared by all the servers. Putting it another way, these IBM solutions move the intelligence of the SAN into the network itself. A high-level view of this is shown in Figure 4. 16 Virtualization in a SAN Intelligence moving into the network Application Application DBMS File System DBMS Application Application File System Installable File System Device Driver Device Driver Device Driver Device Driver Traditional SAN SAN SAN Volume Controller Intelligent Storage Ctller RAID Controller Disk Disk Intelligent Storage Ctller RAID Controller Disk Disk Storage Network RAID Controller Disk Disk Figure 4 Intelligence moving into the network The effective management of resources from the data center across the network increases productivity and lowers TCO. In Figure 4 you can see how we will accomplish this effective management by moving the intelligence from the storage subsystems into the storage network using the SAN Volume Controller, the SAN Volume Controller for Cisco MDS 9000, the SAN Integration Server, and moving the intelligence of the file system into the storage network using the SAN File System. The IBM storage management software represented in Figure 4 as Hardware Element Management and Tivoli SAN Management, addresses administrative costs, downtime, backup and recovery, and hardware management. The management capabilities these products provide are standards-based and pragmatic. To address the downtime component of TCO, for example with the SAN Volume Controller, we have designed our products to provide non-disruptive capabilities, such as movement of data, by insulating application servers from problems in the underlying paths to storage, while allowing storage administrators to change and grow the storage infrastructure dynamically. Tivoli SAN Management SAN File System Hardware Element Management Virtualization in a SAN 17 Realizing improvements As we look at the TCO of storage that is not related to purchase price, estimated at an overwhelming 90%, we have identified three major product areas where significant improvements can be achieved. The SNIA Storage Model describes these areas as Block Aggregation, File Aggregation, and Services Subsystem Management. The following sections describe these areas related to the SNIA models, and the IBM product initiatives in each of these. Block aggregation and block virtualization Block aggregation provides what is also referred to in the industry as block virtualization. Block level virtualization provides servers with a logical view of physical storage. The SAN Volume Controller product provides advanced block virtualization capabilities. In Figure 5 we show the SNIA block aggregation model. Block Aggregation Model: SNIA Application File/Record Layer Host with LVM Host with LVM Host with LVM Host block aggregation Block Layer Aggregation Appliance Aggregation Appliance SN block aggregation Device block aggregation Intelligent Disk Array Low Function Disk Array Copyright 2000, Storage Network Industry Association Figure 5 Block aggregation model: SNIA 18 Virtualization in a SAN Block aggregation provides three significant benefits to customers: Increased storage administrator productivity: Administrators can manage, add and migrate physical disks transparently. This is accomplished by providing insulation between the server’s view of the logical disks and the actual physical disks. Productivity is improved by reducing planned downtime and allowing administrators to perform management functions when convenient rather than waiting for ever-decreasing downtime windows. Providing a common platform for advanced functions: By providing a logical view of physical storage, advanced functions can be done at a single point in the SAN in a common way regardless of the underlying physical storage. FlashCopy, peer-to-peer data copy, and data migration can also be performed in a common way. This common platform will be used to provide other advanced functions over time like advanced security and quality of service capabilities. Improved capacity utilization: Spare capacity on underlying physical disks can be reallocated without impact on servers, irrespective of the server operating system or platform type. Logical disks can be created from any of the physical disks being managed by the virtualization device. In Figure 6 we show the IBM block aggregation plan. Virtualization in a SAN 19 Block Aggregation: IBM Plan Application Application Application Application DBMS File System File System In the network In the data path Move intelligence of controller into network Enterprise reliability DBMS Installable File System Device Driver Device Driver Device Driver Device Driver SAN Volume Controller Intelligent Storage Ctller RAID Controller Disk Disk Storage Network Intelligent Storage Ctller RAID Controller Disk Disk RAID Controller Disk Disk Figure 6 Block aggregation: IBM Plan We have chosen to develop our block aggregation product in the storage network using an in-band approach. We believe that this approach will provide a superior solution for customers needing the benefits of block virtualization. Our solutions are designed to be modular, redundant, and scalable. These solutions are based on clustered IBM eServer xSeries servers, which support high availability and performance that is horizontally scalable. The system will allow for the addition of nodes (engines) non-disruptively to provide enterprise-class scalability. Our long history of storage controller development has enabled us to develop systems where, in the rare case that a component failure occurs, the virtualization device can continue to operate without disruption. The SAN Volume Controller, SAN Integration Server, and the SAN Volume Controller for Cisco MDS 9000 are the IBM solutions for block aggregation. SAN Volume Controller characteristics The IBM TotalStorage SAN Volume Controller is an in-band implementation that minimizes the dependency on unique hardware and software, decoupling the storage functions expected in a SAN environment from the storage subsystems and managing storage resources. 20 Virtualization in a SAN Tivoli SAN Management SAN Hardware Element Management Traditional SAN SAN File System In SANs today, shown to the left in Figure 7, servers are mapped to specific devices. With the SAN Volume Controller, servers are mapped to virtual disks, thus creating a virtualization layer. SANs Today Block Virtualization Storage Network SAN Virtualization Layer Servers are mapped to specific physical disks i.e., "physical mapping" Servers are mapped to a virtual disk i.e., "logical mapping" Figure 7 Block level virtualization With a SAN Volume Controller implementation two zones are created, the host zone and the disk zone, to ensure disk storage devices are protected from being accessed by the application servers, and to ensure I/O management is handled through the SAN Volume Controller. The IBM SAN Volume Controller is designed to provide a redundant, modular, scalable, complete solution, as shown in Figure 8. Virtualization in a SAN 21 Redundant, modular, scalable, complete solution Storage Network Virtual LUNs Virtual LUNs Virtual LUNs Virtual LUNs Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node Node Managed Disks Figure 8 IBM SAN Volume Controller Each SAN Volume Controller consists of one or more pairs of engines, where each pair operates as a single controller with failover redundancy. Each node is an xSeries eServer with a large read/write cache mirrored across the pair. Virtual volumes are shared between a pair of nodes. The pool of managed disks is controlled by a cluster of paired nodes. The SAN Volume Controller is designed to provide complete copy services for data migration and business continuity. Since these copy services operate on the virtual volumes, dramatically simpler replication configurations can be created using the SAN Volume Controller, rather than replicating each physical volume in the managed storage pool. The IBM SAN Volume Controller improves storage administrator productivity, provides a common base for advanced functions, and provides for more efficient use of storage. The value that the SAN Volume Controller can deliver is the ability to: Reduce IT administration costs by improving IT administration productivity with a single point of control, administration, planning, and security Increase the managed data to IT administrator ratio Improve flexibility by providing system availability and data protection Improve overall capacity utilization across all application servers — regardless of operating system or platform type 22 Virtualization in a SAN Reutilize older storage enclosures Provide a single consistent set of high value functions across all OS or platforms usable on all SAN-attached storage (for example, copy services, mirroring, remote mirroring, backup/restore) Provide scalable performance with the addition of relatively low cost components (HBAs, Intel Server memory) The SAN Volume Controller consists of software and hardware components delivered as a packaged appliance solution in a variety of form factors. The IBM SAN Volume Controller solution can be preconfigured to the customer's specification, and will be installed by an IBM customer engineer. Customer data can be migrated from existing storage environments into an IBM SAN Volume Controller environment, and thereafter grown into a SAN Volume Controller managed environment, providing protection of investment and lower storage TCO. In summary, the value that the IBM SAN Volume Controller solution provides is increased system availability, greater storage capacity utilization, improved protection capability, and enhanced scalability. The SAN Volume Controller supports FAStT, ESS, and other vendors’ storage. SAN Volume Controller for Cisco MDS 9000 characteristics The IBM TotalStorage SAN Volume Controller Storage Software for Cisco MDS 9000 uses in-band virtualization. Single virtualization engines, which are known as nodes, are combined to create clusters. A cluster contains four nodes. A node is a single engine. Each Caching Services Module (CSM) supports two engines, or nodes. Nodes within the cluster are grouped in pairs known as an I/O group. Nodes within an I/O group back up one another. Data written to the nodes is duplicated across caches in both nodes. Virtual disks are shared between nodes in an I/O group. To eliminate any single point of failure, nodes in an I/O group must be on a separate CSM. The SAN Volume Controller Storage Software for Cisco MDS 9000 I/O groups see the storage presented to the SAN by the backend controllers as a number of disks, known as managed disks. The application servers do not see these managed disks. This is achieved by zoning or by using Virtual SANs (VSAN). Instead, they see a number of logical disks, known as virtual disks, that are presented to the SAN by the SAN Volume Controller Storage Software for Cisco MDS 9000. Each node must only be in one I/O group and provide access to the virtual disks in the I/O group. The SAN Volume Controller Storage Software for Cisco MDS 9000 helps to provide continuous operations and can also optimize the data path to ensure performance levels are maintained. Virtualization in a SAN 23 The solution offers the following benefits and advantages Reduces complexity Lowers the cost of managing SAN-based storage Creates a single pool of storage from disparate storage devices to increase capacity utilization Implements a cache-based, clustered architecture to provide a highly available solution Provides the scalability and performance required in today’s demanding storage environments. In summary, the value that the SAN Volume Controller Storage Software for Cisco MDS 9000 provides is increased system availability, greater storage capacity utilization, improved protection capability, and enhanced scalability similar to that provided by the SAN Volume Controller. SAN Integration Server characteristics The IBM TotalStorage SAN Integration Server is a pre-configured complete SAN solution. It contains the SAN Volume Controller technology along with Fibre Channel switches, IBM TotalStorage FAStT storage and a master console. Orderable configurations address different storage capacity requirements and contain either one or two pairs of SAN Volume Controllers. Standard upgrade paths are available to provide capacity growth. The solution can leverage the built-in SAN Volume Controller to virtualize the storage from other storage sub-systems. SAN Integration Server offers the following features to attached hosts: A single view of attached storage Logical unit virtualization Large, scalable cache Scalable performance Point-in-Time copy Synchronous Remote Copy Applications will be provided to individually manage the devices that comprise a SAN Integration Server configuration The architecture of the SAN Integration Server is designed to bring enterprise class reliability and performance to open-systems environments. It features hardware redundancy and elements of IBM’s advanced autonomic computing technologies. The intent is to help minimize downtime and improve availability while performing remote mirroring; point-in-time copies; backup and restore; maintenance functions; and performance, capacity and connectivity upgrades. IBM has designed and tested the SAN Integration Server for integration into existing environments, including heterogeneous hardware and operating systems. It is interoperable with a wide range of servers running Linux, UNIX® 24 Virtualization in a SAN and Microsoft® Windows® operating systems, whether from IBM or other vendors. The SAN Integration Server provides centralized management through a single interface to support easier storage allocation and address application demands. This flexibility helps to provide the benefit of better storage utilization by reducing or eliminating the problem of unused storage found in direct attached storage (DAS) implementations, and to reduce required administrative time and resources. These are key factors in realizing a lower total cost of ownership (TCO). File aggregation While block aggregation provides flexibility when working with blocks of data stored in volumes, file aggregation provides flexibility when accessing and managing data stored in files. The SAN File System product provides advanced file aggregation. In Figure 9 we show the SNIA approach to file aggregation. File Aggregation Model: SNIA Application Host with LVM LAN File System Metadata File/Record Subsystem Hosts get file meta data from FS/NAS controller, then access the data directly FS controller can also be NAS server Host block aggregation SN block aggregation Device block aggregation Host with LVM Block Accesses Block Subsystem Disk Array SN Copyright 2000, Storage Network Industry Association Figure 9 File aggregation model: SNIA Virtualization in a SAN 25 In the SNIA model, hosts get file meta-data from file systems or NAS controllers, then access the data directly. With NAS each device is a self-contained file system island. The IBM approach to file aggregation is to provide a meta-data controller in the storage network providing a single global namespace for accessing data on storage devices. The file aggregation solution is based on IBM’s Storage Tank technology. SAN File System characteristics The IBM TotalStorage SAN File System architecture makes it possible to bring the benefits of the existing mainframe system-managed storage (SMS) to the SAN environment. Features such as policy-based allocation, volume management, and file management have long been available on IBM mainframe systems. However, the infrastructure for such centralized, automated management has been lacking in the open systems world of Linux, Windows and UNIX. On conventional systems, storage management is platform dependent. The SAN File System provides a single, centralized point of control to better manage files and data, and is platform independent. Centralized file and data management dramatically simplifies storage administration and lowers TCO. We show the IBM plan for file aggregation, based on the SAN File System architecture, in Figure 10. 26 Virtualization in a SAN File Aggregation: IBM Plan Application Application DBMS File System DBMS File System Application Application Installable File System Meta data controller Direct I/O from appl. servers to either virtual volumes or real volumes SAN/NAS convergence Figure 10 File aggregation: IBM plan SAN SAN Volume Controller Intelligent Storage Ctller Storage Network Intelligent Storage Ctller RAID Controller Disk Disk RAID Controller RAID Controller Disk Disk Disk Disk The SAN File System is a common file system specifically designed for storage networks. By managing file details (via the meta-data controller) on the storage network instead of in individual servers, the SAN File System design moves the file system intelligence into the storage network where it can be available to all application servers. Doing so provides immediate benefits: a single namespace and a single point of management. This eliminates the need to manage files on a server by server basis. The SAN File System automates routine and error-prone tasks such as file placement and handles out of space conditions. The SAN File System will allow true heterogeneous file sharing — where the reader and writer of the exact same data can run different operating systems. The SAN File System meta-data controller is designed as a cluster of servers attached to a SAN and a small software addition to the application servers. Other than installing the SAN File System client on the application servers, no changes are required to applications to use the SAN File System. Hardware Element Management Tivoli SAN Management Traditional SAN Common file system (SAN - wide today, enterprise - wide over time) Device Driver Device Driver Device Driver Device Driver SAN File System Virtualization in a SAN 27 In Figure 11 we show a pictorial representation of the SAN File System environment. SAN File System Server Cluster NFS and CIFS clients Existing IP Network for Participant/File Directory Cluster Communications Client Client AIX VFS w/ cache SAN File System Solaris VFS w/ cache Linux VFS w/ cache Windows IFS w/ cache Admin Client Centralized Administration Space Allocation Policy Mgt R/W Lock Mgt File Directory . . . SAN Fabric Shared Storage Devices SAN File System SAN File System Meta-data controllers Multiple Storage Pools Data Store Figure 11 SAN File System approach Application servers that request a file obtain information about the file (the meta-data) from the SAN File System meta-data controller that manages file locks and all other file information. The SAN File System then provides that information to the application server, which then accesses the blocks comprising that file directly through the SAN. By caching the meta-data in the client and providing direct access from the application server to the underlying storage, the SAN File System provides local file system performance over the SAN. The SAN File System consists of a small module of enablement code that runs on application servers and a meta-data controller based on clustered IBM xSeries servers for redundancy and fault tolerance. IBM supports AIX and Windows with support for other platforms provided in future releases. The features of the SAN File System work together to provide a variety of benefits to customers. One of the major benefits is a single image or global namespace. This function shields the end user from storage network complexity, and dramatically reduces administrative workload. 28 Virtualization in a SAN Since the SAN File System is designed to be implemented on a variety of operating systems from Windows to various flavors of Linux and UNIX, it will allow all of these to share files. A file created in Windows will be as accessible from a Windows client as it is from Solaris, AIX, or any other supported platform, and vice versa. However, bear in mind that an application is still required to be able to read that file, however accessible it is. Since the SAN File System will have a complete understanding of all files on the SAN, including the essential meta-data to make important decisions, it is a logical point to manage the storage on the network through policy-based controls. For example, the SAN File System can decide where to place each file based on user-defined criteria, such as file type, using policy-based automation. Setting these policies relieves the storage administrator of the burden of repetitive tasks, and forms the basis of automation. The SAN File System provides the ability to group storage devices according to their characteristics, such as latency and throughput. These groupings, called storage pools, allow administrators to manage data according to the characteristics that matter to them. For example, an administrator will be able to define a storage pool for mission-critical applications using highly reliable storage arrays that are backed up nightly and have full disaster recovery capabilities. The administrator could also define a storage pool for less critical applications based on JBODs with weekly tape backups and minimal disaster recovery capabilities. Because the SAN File System meta-data is separate from the application data, files can be manipulated while remaining active. For example, files being processed by a mission critical application could be non-disruptively moved within or across storage pools without stopping the application. Data migration from one storage system to another can be handled non-disruptively by having the SAN File System move the pools (data) to new physical disks, then disconnecting the old disks, all done without quiescing applications. The SAN File System approach allows users and administrators to access, save, share and centrally manage files on storage networks. It can leverage policies to direct files into specific storage pools with different class of service characteristics. For example, these may include mirrored pools for disaster recovery, striped pools for performance, or a pool of slower, low cost drives. Storage can be added to these pools dynamically and will be immediately available for use by applications. When files are removed from service, the SAN File System will automatically reallocate the space without disruption. If a LUN is removed from the SAN File System control, the data on that LUN is automatically moved. The SAN File System helps reduce TCO by simplifying the management of files in a storage network. No application changes will be required to realize these benefits. Virtualization in a SAN 29 The SAN File System offers a logical extension to current NAS and SAN environments. While NAS has proven successful in the marketplace, it does not take advantage of a SAN infrastructure. NAS becomes the keeper of file meta-data for their files on the SAN, and requires unique management distinct from the management of files on SAN attached application servers. Our approach to NAS for customers with SANs will be to add NAS capabilities to the SAN File System thereby allowing storage administrators to manage the NAS file data with the same tools as for their application servers also using the SAN File System. Data will not have to be duplicated across multiple NAS devices. This approach of SAN/NAS convergence will lower TCO in these environments. The SAN File System protocol specification and client reference implementation source code for Linux are available publicly and we are working with the industry to encourage convergence of the SAN File System protocols with other standards. Availability of the protocol and reference implementation source code may help third parties to develop additional SAN File System clients beyond what IBM may supply, and reflects the continuing demonstration of IBM's focus on openness. In summary, the SAN File System is a common SAN-wide file system that permits centralization of management and improved storage utilization at the file level. The SAN File System is delivered in a highly available configuration based on IBM eServer xSeries with clustering for the meta-data controllers, providing redundancy and fault tolerance. The SAN File System is designed to provide policy-based storage automation capabilities for provisioning and data placement, non-disruptive data migration, and a single point of management for files on a storage network. The use of the SAN File System can greatly help simplify the management of files on SANs and result in a significant reduction in TCO. Management and productivity In 1999, the Storage Networking Industry Association (SNIA) and Distributed Management Task Force (DMTF) introduced open standards for managing storage devices. These standards use a common protocol called the Common Information Model (CIM) to enable interoperability. The Web-based version of CIM (WBEM) uses XML to define CIM objects and process transactions within sessions. This standard proposes a CIM Object Manager (CIMOM) to manage CIM objects and interactions. CIM is used to define objects and their interactions. Management applications then use the CIM object model and XML over HTTP to provide for the management of storage devices. This enables central management through the use of open standards. 30 Virtualization in a SAN IBM is committed to implementing the SNIA standards-based model to allow IBM products, such as IBM Tivoli Storage Resource Manager (ITSRM), and other vendor management applications, to more easily administer, monitor, and control IBM storage devices. Standard CIM support is included with the SAN Volume Controller, SAN Volume Controller for Cisco MDS 9000, SAN Integration Server, and the SAN File System products, as well as ESS, FAStT and storage from other software vendors. The Storage Management Initiative SNIA is using its Storage Management Initiative (SMI) to create and promote adoption of a highly functional interoperable management interface for multi-vendor storage networking products. The SNIA strategic imperative is to have all storage managed by the SMI interface by 2005. The adoption of this interface will allow the focus to switch to the development of value-add functionality. IBM is one of the industry vendors promoting the drive towards this vendor-neutral approach to SAN management. The Storage Management Interface Specification (SMI-S) for SAN-based storage management provides basic device management, support for copy services, and virtualization. As defined by the standard, the CIM services are registered in a directory to make them available to device management applications and subsystems. SNIA uses the xmlCIM protocol to describe storage management objects and their behavior. CIM allows management applications to communicate with devices using object messaging encoded in xmlCIM. For more information on SMI-S go to: http://www.snia.org Open storage management with CIM SAN management involves configuration, provisioning, LUN assignment, zoning, and masking, as well as monitoring and optimizing performance, capacity, and availability. In addition, support for continuous availability and disaster recovery requires that device copy services are available as a viable failover and disaster recovery environment. Traditionally, each device provides a command line interface (CLI) as well as a graphical user interface (GUI) to support these kinds of administrative tasks. Many devices also provide proprietary API's that allow other programs to access their internal capabilities. Virtualization in a SAN 31 For complex SAN environments, management applications are now available that make it easier to perform these kinds of administrative tasks over a variety of devices. The CIM interface and SMI-S object model adopted by SNIA provide a standard model for accessing devices, which allows management applications and devices from a variety of vendors to work with each other's products. This means that customers have more choice as to which devices will work with their chosen management application, and which management applications they can use with their devices. IBM has embraced the concept of building open standards-based storage management solutions. Our management applications are designed to work across multiple vendors’ devices, and our devices are being CIM-enabled to allow them to be controlled by other vendors’ management applications. CIM Object Manager The SMI-S standard designates that either a proxy or embedded agent may be used to implement CIM. In each case, the CIM objects are supported by a CIM Object Manager. External applications communicate with CIM via HTTP to exchange XML messages which are used to configure and manage the device. In a proxy configuration, the CIMOM runs outside of the device and can manage multiple devices. In this case, a "provider" component is installed into the CIMOM to enable the CIMOM to manage specific devices such as the ESS or SAN Volume Controller. The providers adapt the CIMOM to work with different devices and subsystems. In this way, a single CIMOM installation can be used to access more than one device type, and more than one device of each type on a subsystem. The CIMOM acts as a catcher for requests that are sent from storage management applications. The interactions between catcher and sender use the language and models defined by the SMI-S standard. This allows storage management applications, regardless of vendor, to query status and perform command and control using XML-based CIM interactions. IBM has developed its storage management solutions based on the CIMOM architecture, as shown in Figure 12. 32 Virtualization in a SAN Management Application SMIS Object Model xmlCIM over HTTP SMIS Object Model Disk Provider Tape Provider Virtualization Provider Anonymous Provider CIM Object Manager (CIMOM) Device -specific Managed Device or Subsystem Figure 12 CIMOM component structure IBM storage management of the virtualized SAN IBM uses the SNIA model for CIM enablement to ensure that our storage products can be easily managed in a simple and open way. In addition, each device supports a command line interface (CLI) to allow scripting of repeatable operator tasks. Advanced user interfaces are also provided to simplify the management of each device. In a SAN environment, it is typical for multiple devices to work together to create a storage solution. IBM is now working on a multiple device management product that provides integrated administration, optimization, and replication features for interacting SAN devices, including the SAN Volume Controller, the SAN File Virtualization in a SAN 33 System, SAN Integration Server, SAN Volume Controller for Cisco MDS 9000, ESS, and FAStT devices, and other vendors products. Multiple Device Manager The Multiple Device Manager (MDM) is software that has been designed to allow administrators to manage SANs and storage from a single console. This software solution is designed specifically for managing networked storage components based on the SMI-S, such as the: IBM TotalStorage SAN File System IBM TotalStorage SAN Integration Server IBM TotalStorage SAN Volume Controller IBM Enterprise Storage Server The IBM multiple device management solution is built on IBM Director, a comprehensive server management solution. Using Director with the multiple device management solution will allow administrators to consolidate the administration of IBM storage subsystems and provide advanced storage management functions — including replication and performance management — across multiple IBM storage subsystems. It will interoperate with SAN Management and Enterprise System Resource Manager (ESRM) products from IBM and with products from other vendors, including IBM Tivoli® Storage Resource Management (SRM) products and SAN management products from other vendors. This multiple device management layer provides an integrated view of the underlying system such that administrators can drill down through the virtualized layers to easily perform complex configuration tasks and more productively manage the SAN infrastructure. Since the virtualization layers, as well as IBM's ESS products, support advanced replication configurations, the multiple device management product offers features that simplify the configuration, monitoring, and control of disaster recovery and data migration solutions. Finally, specialized performance data collection, analysis and optimization features are provided. As the SNIA standards become more mature, this multiple device infrastructure management view will be expanded to include CIM-enabled devices from other vendors, in addition to IBM storage. MDM provides simplified support for the particular complexities involved in configuration, tuning, and replication of the virtualized SAN. As with the individual devices, the multiple device management layer is open and can be accessed via a GUI, CLI, or standards-based Web Services. In addition, other management components from IBM and other vendors can be plugged into this layer to enhance the virtualized SAN. 34 Virtualization in a SAN To map host servers and applications to storage resources, ESRM products are used. The IBM Tivoli Storage Resource Manager (ITSRM) product is an ESRM product used to monitor hosts, paths, and devices to ensure that business policies are properly implemented, and to optimize application access to storage resources. With the availability of standard CIM interfaces, ESRM products can more easily manage a wider variety of storage devices and servers from IBM and other vendors. Since the multiple device management layer is also open, ESRM products like ITSRM can also access the multi-device management data and services. This allows ESRM products to access analyzed performance metrics collected over time, consolidated error logs, and other optimization and problem determination aids. In addition, the performance, replication and configuration services can be integrated by the ESRM products to ensure that the virtualized SAN is compliant with business policies. Since the IBM SAN infrastructure is open, customers investing in either IBM or non-IBM management products will benefit from choosing IBM for their SAN infrastructure. By using ITSRM, the multiple device management layer, virtualization solutions, and advanced storage devices like FAStT and ESS, IBM provides open, broad and deep management coverage for its storage solutions, and in the future other vendors products. MDM features are shown in Figure 13. Virtualization in a SAN 35 Multiple Device Manager Features Performance Manager Collect, store, alert Optimized storage allocation Tuning SAN devices Best LUN Analysis Replication Manager PPRC Management FlashCopy Management Atomated Configuration Monitoring of copy operations Performance Manager Replication Manager Device Manager IBM Director, TSANM WebSphere, DB/2 Device Manager Device configuration Multi-Device Management SYM TRI ME X Basic SAN Managemet LUN Discovery Zoning & configuration LUN allocation SAN Integration SMI-enabled Server FAStT ESS SAN Volume Controller SAN File System Figure 13 Multiple Device Manager features The Multiple Device Manager product provides support for configuration, tuning, and replication for the on demand SAN. As with the individual devices, the multiple device management layer is open and can be accessed via a GUI, CLI, or standards-based Web Services. Multiple Device Manager consists of three major components: Device Manager Performance Manager Replication Manager Device Manager The Device Manager is responsible for the discovery of supported devices; collecting asset, configuration, and availability data from the supported devices; and providing a limited topography view of the storage usage relationships between those devices. The Device Manager builds on the IBM Director discovery infrastructure. Discovery of storage devices adheres to the SNIA SMI-S specification standards. Device Manager uses the Service Level Protocol (SLP) to discover supported storage subsystems on the SAN. The Device Manager creates managed objects to represent these discovered devices. 36 Virtualization in a SAN Device Manager provides a subset of configuration functions for the managed devices, primarily LUN allocation and assignment. Its function includes certain cross-device configuration, as well as the ability to show and traverse inter-device relationships. These services communicate with the CIM Agents that are associated with the particular devices, to perform any required configuration. Devices that are not SMI-S compliant are not supported. The Device Manager also interacts and provides some SAN management functionality when IBM Tivoli SAN Manager is installed. The Device Manager health monitoring keeps you aware of hardware status changes in the discovered storage devices. You can drill down to the status of the hardware device, if applicable. This enables you to understand which components of a device are malfunctioning and causing an error status for the device. SAN management When a supported SAN Manager is installed and configured, the Device Manger leverages the SAN Manager to provide enhanced function. Along with basic device configuration functions such as LUN creation, allocation, assignment, and deletion for single and multiple devices, basic SAN management functions such as LUN discovery, allocation, and zoning are provided. In the first release of Multiple Device Manager, IBM Tivoli SAN Manager is the supported SAN Manager. The set of SAN Manager functions that will be exploited are: The ability to retrieve the SAN topology information, including switches, hosts, ports, and storage devices The ability to retrieve and to modify the zoning configuration on the SAN The ability to register for event notification, to ensure Multiple Device Manager is aware when the topology or zoning changes, when new devices are discovered by the SAN Manager, and when hosts' LUN configurations change Performance Manager The Performance Manager component provides the raw capabilities of initiating and scheduling performance data collection on the supported devices, of storing the received performance statistics into database tables for later use, and of analyzing the stored data and generating reports for various metrics of the monitored devices. In conjunction with data collection, the Performance Manager is responsible for managing and monitoring the performance of the supported storage devices. This includes the ability to configure performance thresholds for the devices based on performance metrics, the generation of alerts when these thresholds are exceeded, the collection and maintenance of historical Virtualization in a SAN 37 performance data, and the creation of gauges, or performance reports, for the various metrics to display the collected historical data to the end user. The Performance Manager enables you to perform sophisticated performance analysis for the supported storage devices. Functions Performance Manager provides the following functions: Collect data from devices The Performance Manager collects data from the IBM TotalStorage Enterprise Storage Server (ESS), and the SAN Volume Controller (SVC). Each Performance Collector collects performance data from one or more storage groups, all of the same device type (for example, ESS or SAN Volume Controller). Each Performance Collector has a start time, a stop time, and a sampling frequency. The performance sample data is stored in DB2® database tables. Configure performance thresholds You can use the Performance Manager to set performance thresholds for each device type. Setting thresholds for certain criteria enables Performance Manager to notify you when a certain threshold has been exceeded, so that you to take action before a critical event occurs. You can specify what action should be taken when a threshold-exceeded condition occurs. The action may be to log the occurrence or to trigger an event. The threshold settings can vary by individual device. Advanced Performance Analysis The advanced performance analysis provided by Multiple Device Manager is intended to address the challenge of allocating more storage in a storage system so that the users of the newly allocated storage achieve the best possible performance. Performance Allocation Advisor The Performance Allocation Advisor is an automated tool that helps the storage administrator pick the best possible placement of a new LUN to be allocated (that is, the best placement from a performance perspective). It also uses the historical performance statistics collected from the supported devices to locate unused storage capacity on the SAN that exhibits the best (estimated) performance characteristics. Futures - Performance Bottleneck Analyzer Planned for a future release of Multiple Device Manager, the Performance Bottleneck Analyzer is a tool to aid storage administrators with locating and fixing SAN performance bottlenecks. It utilizes the historical performance data 38 Virtualization in a SAN collected and stored in the database by the monitoring functions, and also uses a means of host-based performance data collection. Replication Manager Data replication is the core function required for data protection and disaster recovery. It provides advanced copy services functions for supported storage subsystems on the SAN. Replication manager administers and configures the copy services functions and monitors the replication actions. Its capabilities consist of the management of two types of copy services: the Continuous Copy (also known as Peer-to-Peer, PPRC, or Remote Copy), and the Point-in-Time Copy (also known as FlashCopy®). In the first release of Multiple Device Manager support will be provided for the SAN Volume Controller and the IBM TotalStorage ESS. Replication Manager includes support for replica sessions, which ensures that data on multiple related heterogeneous volumes is kept consistent, provided that the underlying hardware supports the necessary primitive operations. Replication Manager also supports the session concept, such that multiple pairs are handled as a consistent unit, and that Freeze-and-Go functions can be performed when errors in mirroring occur. Replication Manager is designed to control and monitor the copy services operations in large-scale customer environments. Replication Manager is controlled by applying predefined policies to Groups and Pools, which are groupings of LUNs that are managed by the Replication Manager. It provides the ability to copy a group to a pool, in which case it creates valid mappings for source and target volumes and optionally presents them to the user for verification that the mapping is acceptable. In this case, it manages pool membership by removing target volumes from the pool when they are used, and by returning them to the pool only if the target is specified as being discarded when it is deleted. A follow-on version of Replication Manager may provide the ability to copy a group to another group, in which case the element ordering within the groups is used to determine the pairings between source and target volumes. Event services and Multiple Device Manager At the heart of any systems management solution is the ability to alert the system administrator in the event of a system problem. IBM Director provides a method of alerting called Event Action Plans, which enables the definition of event triggers independently from actions that might be taken. An event is an occurrence of a predefined condition relating to a specific managed object that identifies a change in a system process or a device. The notification of that change can be generated and tracked (for example, Virtualization in a SAN 39 notification that a Multiple Device Manager component is not available). Multiple Device Manager will take full advantage of, and build upon, the IBM Director Event Services. The IBM Director includes sophisticated event-handling support. Event action plans can be set up that specify what steps, if any, should be taken when particular events occur in the environment. Director Event Management encompasses the following concepts: Events can be generated by any managed object. IBM Director receives such events and calls appropriate internal event handlers that have been registered. Actions are user-configured steps to be taken for a particular event or type of event. There can be zero or more actions associated with a particular action plan. System administrators can create their own actions by customizing particular predefined actions. Event Filters are a set of characteristics or criteria that determine whether an incoming event should be acted on. Event Action Plans are associations of one or more event filters with one or more actions. Event Action Plans become active when you apply them to a system or a group of systems. The IBM Director Console includes an extensive set of GUI panels, called the Event Action Plan Builder, that enable the user to create action plans and event filters. Event Filters can be configured using the Event Action Plan Builder and set up with a variety of criteria, such as event types, event severities, day and time of event occurrence, and event categories. This allows control over exactly what action plans are invoked for each specific event. Multiple Device Manager provides extensions to the IBM Director event management support. Multiple Device Manager will take full advantage of the IBM Director built-in support for event logging and viewing. Multiple Device Manager will generate events that will be externalized. Action plans can be created based on filtering criteria for these events. The default action plan is to log all events in the event log. Multiple Device manager will create additional event families, and event types within those families, that will be listed in the Event Action Plan Builder. Event actions that enable Multiple Device Manager functions to be exploited from within action plans will be provided. An example is the action to indicate the amount of historical data to be kept. More comprehensive information about MDM can be found in this IBM Redpaper: http://www.redbooks.ibm.com/redpapers/pdfs/redp3407.pdf 40 Virtualization in a SAN Conclusion In summary, the IBM storage initiative presents a roadmap that adds intelligence to storage networks. This is achieved through a combination of IBM technologies and rapidly-emerging storage management standards, reducing the overall TCO for storage. These technologies, aimed at cutting through the thicket of proprietary approaches that multiply customer cost and limit manageability, include: SAN Volume Controller: A virtualization appliance solution that maps virtualized volumes visible to hosts and applications to physical volumes on storage devices. All servers being served by the SAN could be connected to the SVC. This may include all servers on the SAN or only a subset. This enables the system administrators to view, access and control a common pool of storage on a SAN so they can use storage resources more efficiently. Each server within the SAN has its own set of virtual storage addresses which are mapped to a physical address. If the physical addresses change, the server continues running using the same virtual addresses it had before. This means that volumes or storage can be added or moved while the server is still running. IBM's virtualization technology improves management of information at the “block” level in a network — enabling applications and servers to share storage devices on a network. SAN Volume Controller Storage Software for Cisco MDS 9000: A joint project between IBM and Cisco, where IBM provides the virtualization software (SVC), and Cisco provides the hardware platform (MDS 9000), and the Cisco MDS 9000 Caching Services Module (CSM). A virtualization solution that creates a pool of managed disks from the attached storage subsystems, which are then mapped to a set of virtual disks for use by various attached host computer systems. The system administrators can view and access a common pool of storage on the SAN which allows them to use storage resources more efficiently, and provides a common base for advanced functions similar to those provided by the SAN Volume Controller. SAN Integration Server A solution designed to integrate IBM’s virtualization technology, Fibre Channel switches and Redundant Array of Independent Disks (RAID) storage technologies into a preconfigured, complete solution. Delivered and installed as an integrated unit, it offers upgrade options for connectivity, storage capacity and performance levels. The solution has been developed to provide the benefits of SANs with faster installation and configuration along with ease of manageability. The SAN Integration Server provides centralized management through a single interface to support easier storage allocation and address application demands. This flexibility provides the benefit of better storage utilization by reducing or eliminating the problem of unused storage found in direct attached storage implementations, and Virtualization in a SAN 41 reducing required administrative time and resources. Like the SAN Volume Controller, IBM has designed and tested the SAN Integration Server to integrate into existing environments, including heterogeneous hardware and operating systems. SAN File System: A SAN-wide file system, based on Storage Tank technology, for accessing data on storage networks across multiple application OS platforms and heterogeneous storage devices. The SAN File System also provides centralized, policy-based management of the data in the SAN. With the SAN File System, all of the files owned by the servers are made visible to clients of the SAN File System as a single file system in the SAN. This means that all files in the SAN can be made accessible to all of the servers, if the customer chooses. This eliminates the need to maintain copies of the same file for use by multiple servers. Because there is a central catalog of all files in the SAN, policies can be established for each file in areas such as file placement, security, and service level requirements. With the SAN File System, the storage administrators do not need to assign storage volumes to individual servers. By not having to partition the storage across the application servers, there can be more efficient use of the storage and less storage administration required. For those familiar with DFSMS in the zOS environment, think of the SAN File System as the open systems version of DFSMS. Multiple Device Manager: Software that has been designed to enable administrators to manage SANs and storage from a single console. This software solution is designed specifically for managing networked storage components based on the SMI-S. The IBM multiple device management solution is built on IBM Director, a comprehensive server management solution. Using Director with the multiple device management solution enables administrators to consolidate the administration of IBM storage subsystems and provide advanced storage management functions (including replication and performance management) across multiple IBM storage subsystems. It interoperates with SAN Management and Enterprise System Resource Manager (ESRM) products from IBM and with products from other vendors, including IBM Tivoli® Storage Resource Management (SRM) products and SAN Management products from other vendors. This represents the industry's first comprehensive plan for virtualization, SAN-wide file sharing, and centralized management of these technologies. Our initiative comes as the storage industry shifts from its direct-attached model to a networked storage model, increasing the importance of software and services as a competitive advantage. Earlier generation virtualization products have focused their efforts narrowly at the storage block level. The SAN Volume Controller and SAN File System provide next generation virtualization solutions that introduce support for both 42 Virtualization in a SAN block aggregation and file aggregation in an open, managed environment. The benefits of these products include eliminating downtime associated with SAN hardware upgrades and improving an administrators effectiveness, enabling the amount of storage maintained per administrator to be increased. This efficiency will lead to better disk utilization while still maintaining a wide choice of storage options and minimizing risk. In summary, in an era in which businesses are integrating processes across the enterprise, the IBM TotalStorage virtualization family can help companies become on demand businesses, responding with flexibility and speed to customer demands, market opportunities or competitive threats. With integration, support of open standards, and virtualization and autonomic capabilities, companies can be better positioned to increase the flexibility, reliability and performance of the SAN environment — while improving utilization and return on investment to reduce the total cost of ownership of storage resources. IBM TotalStorage for an on demand world: Connected. Protected. Complete. Thanks to the following people for their contributions: Gordon Arnold Tom Cady Bill Cochran Greg Cole Scott Drummond Linda Duyanovich Leslie Estroff Kim Fountain John Gressett Chuck Grimm Ron Henkhaus Fred Hoogland Emma Jacobs Harish Krishnamurthy Sally Krusing John Langlois Yvonne Lyon Mary Lovelace Richard Moore Nigel Morton Kjell E Nystrom Dan O’Hare Jeff Ozvold Alan Petersburg Yvonne Sargeant Virtualization in a SAN 43 Chris Saul Brian Sherman Lawrence C Smith Roger Wofford 44 Virtualization in a SAN Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces. © Copyright IBM Corp. 2003. All rights reserved. 45 This document created or updated on December 27, 2003. Send us your comments in one of the following ways: Use the online Contact us review redbook form found at: ibm.com/redbooks Send your comments in an Internet note to: redbook@us.ibm.com Mail your comments to: IBM Corporation, International Technical Support Organization Dept. QXXE Building 80-E2, 650 Harry Road San Jose, California 95120-6099 U.S.A. ® Trademarks The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both: IBM eServer™ Redbooks(logo)™ AIX® DFS™ e-business on demand™ Enterprise Storage Server™ ESCON® FICON™ FlashCopy® IBM® Lotus® MVS™ MVS/XA™ OS/390® Perform™ RAMAC® S/390® Seascape® Tivoli® TotalStorage™ WebSphere® Word Pro® xSeries™ The following terms are trademarks of other companies: ActionMedia, LANDesk, MMX, Pentium and ProShare are trademarks of Intel Corporation in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. C-bus is a trademark of Corollary, Inc. in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. SET, SET Secure Electronic Transaction, and the SET Logo are trademarks owned by SET Secure Electronic Transaction LLC. Other company, product, and service names may be trademarks or service marks of others. 46 Virtualization in a SAN