Get documents about "wp-gluster-vm-environments
Document Sample
Whitepaper
Gluster Virtual
Machine Environments
Abstract
Until recently, enterprises using server virtualization had to rely on a collection of
inefficient and costly techniques to create and manage storage for their virtual servers.
In this paper, we’ll demonstrate how technology from Gluster has finally closed the gap
between virtual servers and their data storage needs.
Introduction
The momentum towards server virtualization continues unabated. Increasing numbers of
organizations are turning to virtualization as a technique to wring more productivity from
existing servers, control costs while enhancing flexibility and agility, and leverage
commodity hardware and open-source software. In addition, virtualization is playing a
key role in support of cloud computing.
However, as is the case with any new disruptive trend, an enterprise that is deploying
virtualization in a meaningful way is faced with a collection of formidable impediments to
success. One of the most demanding challenges is the reality that while server
virtualization has a well-proven track record and assemblage of guidelines, best
practices, and supporting technologies, storage has struggled to keep pace. The gap
between these two disciplines manifests itself in a number of dramatic ways.
To begin, maintaining virtual server images naturally consumes massive amounts of
storage space since there are typically a large number of images and their underlying
files are large. In a traditional IT environment, the ratio of computers to applications is
relatively fixed and slow to change. On the other hand, in a virtualized environment it’s
easy to add virtual machines and expand them quickly. Since these images are vital to
the successful operation of the enterprise, every virtual machine image must be
protected and managed. This is particularly troublesome when you consider that virtual
machines are in constant operation, leaving no opportunity to perform a traditional
backup.
Administering the network connections among these virtual machines and any
associated storage is also an essential responsibility that can’t be overlooked. This
problem is compounded when factoring in the frequent creation of virtual machine
Copyright 2009, Gluster, Inc. Page 2 of 11
All Rights Reserved. www.gluster.com
images and the growth of application data, which are often kept in their own individual
silos. Finally, cloud computing introduces its own set of demands and is tightly coupled
with both server and storage virtualization.
In an effort to surmount these storage complications imposed by server virtualization,
enterprises have tried a variety of traditional information management approaches.
These include setting up dedicated silos, deploying multiple logical unit numbers (LUNs)
and connections, as well as manual administration. Unfortunately, these hand-crafted
approaches generally fall short of the mark, imposing substantial burdens on the
organization. Traditional block-based SAN storage systems are monolithic in nature.
iSCSI or fiber channel SAN storage systems are expensive, hard to manage and have
scalability limitations. Every virtual machine requires dedicated block storage. In both
virtualized and cloud environments, storage volumes are provisioned on a daily or hourly
basis, sometimes more frequently. Volumes become unmanageable and hard to back
up. Monolithic architectures are subject to painful I/O bottlenecks when required to
support multiple clients running multiple virtual machines.
Thus, to be successful, virtualized computing requires a true virtualized storage solution
such as Gluster Storage Platform. It is a fully distributed architecture with no single point
of failure, running on scale-out commodity hardware. Administrators can create a single
global volume for storing all virtual machines and let the file system automatically
distribute data and load balance the resources. In addition to storing the virtual machine
disk images themselves, Gluster Storage Platform also provides NAS volumes to
applications on top of virtual machines. It supports clients for GlusterFS native, CIFS,
NFS and WebDAV protocols. Volumes are able to grow to multiple petabytes, all within a
single global namespace. Finally, replication and online self-healing capabilities brings
non-stop availability to virtual machines.
Introducing Gluster
Gluster, based in Milpitas, California, is exclusively focused on simplifying the task of
storing and managing large quantities of unstructured data. Furthermore, Gluster does
this in a cost-effective way using commodity hardware and storage, with easy
management, high performance, and high availability as fundamental cornerstones of
their mission. Replication and online self-healing capabilities brings non-stop availability
Copyright 2009, Gluster, Inc. Page 3 of 11
All Rights Reserved. www.gluster.com
to virtual machines on top of commodity hardware platforms. Gluster’s solutions are
offered using an open-source approach, offering complete transparency to their clients.
Gluster Storage Platform is an ideal complementary technology for virtual servers,
creating a single resource pool from a collection of low-cost, commodity storage
components clustered together in building-block fashion. To simplify management
responsibilities, it provides a single, global namespace, along with a single volume for
application data and virtual machine images. This approach centralizes administrative
responsibilities while reducing complexity. It also includes a collection of graphical tools
along with a full XML-based, RESTful-style API to further streamline the day-to-day
tasks for administrators.
Since organizations are relying on their virtual servers to support mission-critical
applications, Gluster Storage Platform is designed to maximize uptime while minimizing
administrative responsibilities. For example, it employs continuous self-healing
capabilities, which allows for online, automatic correction of active virtual machines.
Administrators are also free to create file-level snapshots without bringing the virtual
machine down. Finally, Gluster Storage Platform offers continuous, on-the-fly backups –
analogous to an implementation of intelligent rsync.
Four Steps to Storage Virtualization Success
For the remainder of this guide, we’ll explore a series of straightforward steps employing
Gluster Storage Platform to create a robust, scalable, easily-administered pool of virtual
storage to support your virtual servers.
1. Select the type of back-end storage you want to use
Depending on budget and existing hardware constraints, you have several options when
deciding on the physical storage to employ.
In a Direct Attached Storage (DAS) configuration, disks are directly connected to the
storage server, either within the chassis or through an external JBOD (just-a-bunch-of-
disks) enclosure. DAS is simple and scalable to multiple petabytes. Gluster's replication
and self-healing capabilities bring enterprise class availability to commodity hardware.
Even with replication enabled, it is usually less expensive than a employing a SAN. Data
is protected at two levels: RAID at disk level and replication at system level. However,
Copyright 2009, Gluster, Inc. Page 4 of 11
All Rights Reserved. www.gluster.com
replication does cut your usable capacity in half. If replication is not required (such as
when you are building a near-line archival storage), you can turn off the replication
feature and simply rely on RAID6 parity protection. SATA-II disks are sufficient for many
workloads. However, when higher performance is required, it’s more appropriate to use
SAS disks.
For example, in a DAS configuration, an eight node, 120TB storage cluster will consist of
each node configured with two CPUs, 16GB of RAM, 15 1TB SATA-II disks, one GbE
network card and a RAID controller.
In a Storage Area Network (SAN) configuration, disks are connected to the storage
server via fiber channel (FC) or iSCSI network. SAN systems have redundancy built in to
the hardware for high availability and therefore do not require data replication. A cluster
of blades or 1U servers are placed in front of the SAN as NAS storage heads. The
administrator assigns one LUN per NAS head. Gluster then aggregates all of the LUNs
under a single global namespace with built-in high availability load-balancing.
For example, an eight node, 10TB FC SAN will have each NAS storage head configured
with two CPUs, 16GB RAM, one FC controller and a GbE network card.
Regardless of your physical storage selection, Gluster optimizes the creation and
maintenance of your virtual storage infrastructure. Data is stored in a non-proprietary
format (NFS-like) on the back end disks, allowing access with standard tools even if
Gluster is removed.
2. Decide how to connect clients to your storage
With your physical storage defined, the next step is to identify one or more
communication protocols to use between your virtual servers and virtual storage.
GlusterFS provides maximum flexibility: you’re free to export storage volumes using
multiple protocols, all of which are available concurrently. For example, this lets a
document placed on a shared folder (CIFS) on a Windows Vista client be accessed via
the Network File System protocol (NFS) from a UNIX server. Each protocol supported by
Gluster has its own performance, manageability, and operating system support
advantages, which we summarize below:
Copyright 2009, Gluster, Inc. Page 5 of 11
All Rights Reserved. www.gluster.com
1. Gluster native protocol: In the past few years, storage requirements
have grown several-fold in terms of capacity and performance,
outstripping the capabilities of many of the most popular protocols in
existence. In response, Gluster designed this protocol from the ground-up
to support massive scale-out and simpler manageability. Despite all of its
power and flexibility, it installs like a standard application and doesn’t
require any kernel modifications. This protocol is automatically supported
in all Gluster implementations; you’re also free to use any of the protocols
described next. It is POSIX compliant, which allows existing applications
run access storage without any changes. It is supported on GNU/Linux
and Mac OS X (10.4 and above).
2. NFS: GNU/Linux and UNIX operating systems have built-in NFS client
support, which lets applications access storage volumes without requiring
any additional software installation. This is a useful, easily-deployed
protocol for general-purpose file storage and archival.
3. Server Message Block/Common Internet File System (SMB/CIFS):
Microsoft operating systems have built-in CIFS client support, making this
protocol ideal for general-purpose file storage and archival in a Windows
environment.
4. HTTP GET/PUT: GlusterFS provides a distributed object storage
mechanism compatible to Amazon’s S3 cloud storage service. It’s ideal for
scalable Web or cloud-based applications. It offers a well-documented
RESTful API, with bindings for popular programming languages. This
interface is so simple that it can even be accessed from your JavaScript
code inside your browser client.
5. Web-based Distributed Authoring and Versioning (WebDAV): This
protocol provides a standards-based, distributed object storage
mechanism using HTTP as its underlying transport. More powerful than
Copyright 2009, Gluster, Inc. Page 6 of 11
All Rights Reserved. www.gluster.com
basic HTTP GET/PUT commands, it includes support for files, folders and
meta properties with a global name space.
6. File Transfer Protocol (FTP): This is one of the most popular
mechanisms for downloading files across the Internet. It is good for
transferring large amounts of files over the network. FTP is supported by
all major operating systems. Both console based and graphical clients are
available for all major operating systems.. Since it works at the application
layer, it is the most interoperable protocol of all.
7. Virtual iSCSI: This SAN protocol, particularly useful under Microsoft’s
Hyper-V virtualization environment, provides block-based storage access
virtualized across a cluster of storage servers. These volumes are
redundant and highly available.
3. Configure your storage environment
With the physical storage assigned, and protocol(s) selected, your next responsibility is
to organize your virtual storage in preparation for usage. Using Gluster Storage Platform,
configuring your storage environment can be as simple as setting up a wireless router.
Its Server Manager makes it easy to locate and add storage servers, as shown in
figure 1:
Copyright 2009, Gluster, Inc. Page 7 of 11
All Rights Reserved. www.gluster.com
Once the server has been added, your next responsibility is to use the Volume Manager
to create storage volumes, as illustrated in figure 2:
If you select more than one server from the list, clustering is automatically enabled for
your volume. Clustering provides intelligent hash-based file-level distribution across the
servers. It is similar to intelligent file-level RAID-0 across the servers.
Another approach is to use Gluster’s mirror feature, which provides distributed
replication of data (similar to intelligent RAID-10) across servers. Every file is replicated
twice, allowing individual systems to fail completely without disrupting the storage
cluster. If your data is mission-critical and needs to be highly available, this is the
appropriate mode for you.
Copyright 2009, Gluster, Inc. Page 8 of 11
All Rights Reserved. www.gluster.com
Finally, the stripe feature is encountered primarily in high performance clustering (HPC)
environments where a large number of clients concurrently require intensive access to a
few large files. These files are generally significantly larger than the available RAM.
Note that you’re free to mix-and-match configurations; Gluster automatically manages
these arrangements for you. Gluster Volumes are thin-provisioned automatically,
meaning no physical disk resources are reserved in advance. 100TB of storage can be
virtually provisioned into 10 different volumes of each 100TB capacity. This makes
efficient use of disk space and storage management simpler.
Once you’ve named your volume and selected a structure, your next responsibility is to
enable one or more protocols. As described earlier, Gluster supports multiple protocols
concurrently, giving you maximum access flexibility. It also supports Gigabit Ethernet, 10
Gigabit Ethernet, and InfiniBand interconnect options. Once you’ve enabled your
protocols, you also set access control levels to determine which clients may interact with
information.
4. Manage your environment
Once your virtual storage is up and running, your responsibilities are just beginning.
However, in keeping with its philosophy of excellent manageability, high performance,
and high availability, Gluster has streamlined the vital tasks necessary to support
mission-critical virtual storage in the following ways:
1. Automated operations. Instead of forcing administrators to struggle with
keeping a massive storage pool online and running smoothly, Gluster
automates such chores as error correction, load balancing, and live data
copying. When administrators do need to directly perform tasks such as
adding new storage nodes, Gluster lets them proceed without mandating
disruptive downtime.
2. Information asset protection. Gluster’s automated operational
capabilities have the side benefit of safeguarding information from events
such as loss of a machine, disk corruption, and file damage.
3. Powerful management graphical user interface. By centralizing virtual
storage into a single namespace and providing high quality graphical
Copyright 2009, Gluster, Inc. Page 9 of 11
All Rights Reserved. www.gluster.com
management tools, Gluster has made it much easier to administer your
environment. Gluster Storage Platform’s Web Management Console,
shown in figure 3, includes the following tools to let administrators
command their storage resources from a single location:
• Volume Manager
• Log Viewer
• Resource Monitor
• Network manager
• Admin Settings
4. Full XML REST-style service management API. In addition to providing
a complete suite of graphical management tools, Gluster Storage Platform
also offers a comprehensive XML-based, RESTful-style service API that
allows programmatic monitoring and management of your storage
infrastructure.
These four simple steps are all you need to do to reap the benefits of virtualized storage
in support of your virtual computing environment. Furthermore, Gluster Storage
Platform’s solid infrastructure and outstanding administrative tools will maximize your
virtual storage configuration’s uptime, performance, and manageability.
Copyright 2009, Gluster, Inc. Page 10 of 11
All Rights Reserved. www.gluster.com
Conclusion
The drive towards virtual servers and cloud computing will continue to be an overarching
technology trend. In response, IT management and administrators can expect to be
called upon to support these powerful, but challenging, new environments. Providing an
effective storage foundation for virtualized servers will remain a central responsibility.
With technologies such as those offered by Gluster, however, storage options for virtual
servers are finally catching up. It’s now possible to configure and manage massive
amounts of virtual storage in a cost-effective, easily administered way. Even though this
technology is far less expensive than older, proprietary solutions, it’s still able to provide
high performance and high availability.
To download a free copy of GlusterFS, visit us at
http://gluster.com/products/download.php. If you’d like a 30 day trial subscription to
Gluster Storage Platform, check out http://gluster.com/products/trial.php. To speak with
a Gluster representative about how to solve your particular storage challenges, phone us
at +1 (800) 805 5215.
Copyright 2009, Gluster, Inc. Page 11 of 11
All Rights Reserved. www.gluster.com
