StorageSwitch FS Gateway Integrated with the Sun StorageTek TM 5800 system
A Solution Brief StorageSwitch LLC. November 13, 2007
Sun Microsystems, Inc.
Table of Contents
Introduction.......................................................................................................................................................... 3 Market and Business Problem Addressed by StorageSwitch FS-Gateway.......................................................... 3 How Storageswitch FS Gateway Addresses the Problem.................................................................................... 3 High-level Overview............................................................................................................................................. 3 StorageSwitch Architecture ................................................................................................................................ 4 StorageSwitch and the StorageTek 5800 system Implementation....................................................................... ShareView .......................................................................................................................................................... File Versioning & Delayed Delete......................................................................................................................... Compliance and Governance.............................................................................................................................. Architected for the Most Demanding Clients........................................................................................................ StorageSwitch Appliance Architecture................................................................................................................ System Data Protection....................................................................................................................................... 6 6 7 7 7 8 9
3 Introduction
Sun Microsystems, Inc.
Introduction
StorageSwitch® is a multifaceted, robust filesystem gateway offering unlimited scalability, enterprise-class reliability, complete flexibility and advanced functionality. The StorageSwitch product is ideal for managing fixed-content information, whatever the source, wherever the destination. StorageSwitch is specifically designed to virtualize the presentation of fixed-content data. The StorageSwitch can take advantage of standard file system protocols (NFS, CIFS, HTTP, FTP) to store, retrieve and manage data, eliminating the need for device-specific API coding. StorageSwitch also removes the restrictive limits imposed by current file systems and enhances the ability to manage essential information assets. Storing data and associated metadata with StorageSwitch enables companies to take advantage of the benefits unique to the Sun StorageTekTM 5800 system. The StorageSwitch core engine features an extensible metadata database and a plug-in architecture to meet the requirements of virtually all dynamic storage environments for the life of the data. Metadata can be created, extracted and then mapped into multiple file system presentations for use by any application.
Market and Business Problem Addressed by StorageSwitch FS Gateway
StorageSwitch provides standard file system interfaces for NFS, CIFS, WebDAV and FTP to any client or application on the network. This enables the client or application to store objects on the StorageTek 5800 system without the requirement of developing to the API. The StorageSwitch includes an open database architecture that is capable of tracking and preserving all file system and data attributes including ctimes, mtimes, retention, ACLs and GUIDs. These features also enable the StorageSwitch to provide a tool for migrating data from traditional storage (block and NAS) to new fixed content storage such as the StorageTek 5800 system for long term repository storage and data consolidation projects.
How StorageSwitch FS Gateway Addresses the Problem
Objects that are stored on the StorageTek 5800 system are viewed as virtual filesystems through network shares from StorageSwitch and can then be accessed through standard interfaces by any authorized client or application. The StorageSwitch adheres, and is compliant with all network security protocols such as Windows AD, LDAP and NIS. Objects can also move transparently to multiple destinations manually or by policy based directives. Clients or applications are able to store data on the StorageTek 5800 system with the ease of operations of a traditional enterprise environment and in addition the customer enjoys the benefit of high density storage capacity for a fraction the cost of traditional storage models.
High-level Overview
Sun StorageTek 5800 system (also known as Honeycomb) is a new category of storage system that is optimized to store large amounts of digital content, such as x-ray scans, photographs, images, research data sets, digitized books (e.g. from the Google books project). The StorageTek 5800 system is the first in a new line of 3rd generation open, fixed content storage systems that includes extensive metadata facilities that describes the object being stored; and architecture that supports the ability to process the object being retrieved or stored (Storelets). Unlike traditional storage systems, customers deploying the StorageTek 5800 are able to improve data integrity assurances, reliability and efficiency while significantly helping lower their storage costs. In addition, the StorageTek 5800 system represents the only open, fixed content storage system in the market today, which means customers can be assured that they will always be able to get to their data. Applications can communicate to the StorageTek 5800 system either through WebDAV interface or the opensource API. For customers not interested in writing to the StorageTek 5800 API, StorageSwitch is the fast path for their fixed content storage needs.
4 StorageSwitch Architecture
Sun Microsystems, Inc.
StorageSwitch Architecture
StorageSwitch employs a three-tiered architecture to bring standard file system interfaces to the StorageTek 5800 system. The first tier is the client presentation layer. This provides standard protocol support to the client. These protocols include: NFS, CIFS, WebDAV, and FTP. Future protocols under consideration or development are DICOM/HL7, SysLog, SMTP, ATL and XAM. This is a multi-protocol tier, so data stored by one protocol is simultaneously visible and available via other similar protocols. StorageSwitch provides multi-protocol support to all NFS, CIFS, WebDAV and FTP. The second tier is the internal core engine (often referred to simply as the core). The core provides metadata storage and replication functions. Also data transformations and extractions can occur here through the use of plug-ins. Examples of plug-ins include data extractors (like pulling metadata from the header of a file), Hash, compression and encryption all exist as plugins to the core. The core is heavily database driven, utilizing MySQL. The database provides scalability and performance in operations that are impossible or extremely slow in standard file systems. The database is available for external access – via query for reporting and data extraction purposes. The third tier is the storage driver tier. This provides connectivity to the backend storage; in this case the StorageTek 5800. Each storage driver is tuned to the needs and performance characteristics of the backend device.
High-Level StorageSwitch Overview
Clients & Applications
Mgt UI, Policy Engine
Performance scales with additional nodes
Multiple connections
No loss of service in event of node failure
High Availability
Client Access Point Processor
NFS, CIFS, FTP, HTTP, WebDAV
Node 1
Active Data Protection
Node 2-N
In-line/Out Of band Data Processors
MetaData Infrastructure
Physical Data Storage Module
Multiple connections
Multiple connections
SUN 5800 Honeycomb
5 StorageSwitch Architecture
Sun Microsystems, Inc.
Descriptions of solution components
•
StorageSwitch Requirements
• •
Two Sun X86 Servers (Sun FireTM x2100 or x4100 servers) Disk space varies by customer requirements for activity and number of objects. Internal or external storage can be added to support staging and migration processes. Dual power Feeds are preferred but not required 3 GigE Network interfaces
-
• •
ETH0 is connected to the application server network ETH1 is connected to the Backend Storage Network ETH2 is the Internal Interconnect between Servers
• •
StorageSwitch for StorageTek 5800 software
Sun StorageTek 5800 system
Client Access
Open Optimized Single Virtual IP Highly Available Massively Scalable Geographically Replicated Dynamically Load Balanced FileSystem Presentation to StorageTek 5800 system
IP Switch
Application Server Access
StorageSwitch HA Cluster
IP Switch
The following diagram shows the use of the network ports on the StorageSwitch server. The servers are configured and accessed in an active/passive configuration.
IP Switch
StorageSwitch HA Cluster
6 StorageSwitch Architecture
Sun Microsystems, Inc.
StorageSwitch HA Cluster Network Connections
E0
Front-End Protocols
E2 E3
Heartbeat, Metadata Sync
E1
Front-End Protocols StorageTek 5800
Additional Recommendations
• • • • • • •
Dual power of which two separate power feeds is recommended for the highest availability Customer provided switches on separate networks for the application servers and the storage device StorageTek 5800 Management Access Remote access available for the StorageSwitch server StorageSwitch FS Gateway requires an SMTP server to deliver management alerts via email Metadata schema is required on the StorageTek 5800 system Optionally StorageSwitch will utilize an SNMP server to deliver management alerts to existing SNMP based management tools.
StorageSwitch and the StorageTek 5800 system Implementation
Confidently and securely manage fixed content and critical archive data with versatility and power of the FS gateway. StorageSwitch provides a multifaceted, robust gateway for the Sun StorageTek 5800 system, offering scalability, enterprise-class reliability, complete flexibility and advanced functionality. StorageSwitch FS Gateway can seamlessly manage all data requirements for creating and migrating data to the StorageTek 5800. StorageSwitch is specifically designed to virtualize the presentation of fixed-content data. Using this solution, use of standard file system protocols such as NFS, CIFS, HTTP, WebDAV and FTP to store, retrieve and manage data eliminates the need for device-specific coding. Storing data and associated metadata with StorageSwitch enables the ability to take advantage of the benefits unique to the StorageTek 5800 system. Open access to the databases provides additional access to the data. The core engine features an extensible metadata database and a plug-in architecture to meet the requirements of a dynamic storage environment for the life of the data. Metadata can be created, extracted and then mapped into multiple file system presentations. Migration tools provide the ability to maintain technology refreshes without extensive downtime. StorageSwitch FS Gateway product has been running in mission-critical production systems for over five years, proving itself as a sturdy virtualization infrastructure. It is a robust, highly scalable system, capable of managing the transport and storage of data in the most demanding environments.
ShareView
Create and access alternate file system views. Metadata can be mapped to standard file system fields to expose data available in standard file systems. ShareView mounts can cross mount points, facilitating
7 StorageSwitch and the StorageTek 5800 system Implementation
Sun Microsystems, Inc.
management and data sharing views. (ShareView currently requires a Sun Professional Services engagement for setup and configuration.)
File Versioning & Delayed Delete
Automatic file versioning is available, enabling enhanced user capabilities, file protection and compliance as implemented by StorageSwitch. Delayed delete provides a ‘trash can’ function.
Compliance and Governance
StorageSwitch preserves and honors data management policies for “retention set” documents. StorageSwitch supports both Snaplock and share-based retention-setting protocols. Customers with strict regulatory requirements depend on StorageSwitch to protect their data when accessed through StorageSwitch. Future releases of the StorageTek 5800 system will include additional features to further protect the data from direct access that bypasses the StorageSwitch .
Architected for the Most Demanding Clients
The StorageTek 5800 system together with the StorageSwitch has been architected with flexible attributes to meet the requirements of most demanding customers. Open: The open architecture is critical for the long term digital storage. Open interfaces and open access to all of the information about enterprise data provide a corporation with the ultimate in investment protection -Knowledge. Extensible Metadata: The StorageSwitch engine allows the automatic extraction and creation of relevant metadata, without sacrificing critical file information. Open Access to File Metadata: Industry standard SQL tools can be used to generate customized data storage reports. Optimized: The system has been tuned to provide maximum throughput to and from the StorageTek 5800 system to ensure the performance requirements. Single virtual IP: A single IP address is automatically load balanced across multiple NICs for fault tolerance and redundancy . Highly available: Data is synchronously replicated so that in event of a problem, the system will fail over in less than a second, with no impact to users and applications, in the event of a failure. Massively scalable: Designed for StorageTek 5800 system our virtual file system scales in both performance and capacity. Billions of files in a single mount point if required. And because of the architecture, none of the traditional file systems issues of managing that space. Geographically replicated: Asynchronous replication of metadata across long distances provides for maximum uptime, even in event of a complete site failure. Security and User Authentication: StorageSwitch provides multiple authentication schemes, including simple password protection, LDAP or Active Directory. Virtualized File System: StorageSwitch presents a uniform interface to the client applications, using standard protocols (NFS, CIFS, FTP, WebDAV, HTTP), from the StorageTek 5800 system. All protocols have the potential, controlled by the administrator, to access to the same shares.
8 StorageSwitch and the StorageTek 5800 system Implementation
Sun Microsystems, Inc.
StorageSwitch Appliance Architecture
Designed to be implemented as a transparent appliance, ease of use in setup and management is critical. Administrators simply create shares, (mount points) to the information and set basic operational parameters. It is truly a set and forget device, with minimal management requirements. The following is an example of the User Interface used in this configuration process.
System Data Protection
The StorageSwitch data is protected in a number of methods to provide the utmost in data availability and protection. Locally to each node—all data is written on mirrored disk. On a single site, the data is synchronously replicated to the other nodes in the cluster. If a remote site exists, data is replicated to that site asynchronously. Nightly, more frequently if required, the system databases are backed up. This is a ‘hot’ backup –thus no application down time is required. These backups are stored locally on the system hard disk, in the StorageTek 5800 system and also dropped on a separate NAS mount point for those customers wishing to drive this data to tape or other off-site media. Of course in event of catastrophic failure, complete rebuilds can be done from the data in the backups and / or StorageTek 5800 system – to the point of the last confirmed write.
9 StorageSwitch and the StorageTek 5800 system Implementation
Sun Microsystems, Inc.
Additional information
sun.com/storagetek/disk_systems/enterprise/5800/index.xml http://www.opensolaris.org/os/project/honeycomb/ http://www.storageswitch.com
Sun Contacts:
Contact your local sales representative for additional information about the StorageTekStorageTek 5800 system Integration with StorageSwitch to solve Fixed Content Storage requirements.
StorageSwitch Contacts
Clark Hodge StorageSwitch CEO Clark@storageswitch.com 303.859.3321
John Takita Itochu StorageSwitch Product manager john@itochu.net 646-862-6905
Sun Microsystems, Inc. 4150 Network Circle, Santa Clara, CA 95054 USA Phone 1-650-960-1300 or 1-800-555-9SUN Web sun.com
© 2007 Sun Microsystems, Inc. All rights reserved. Sun, Sun Microsystems, the Sun logo, Sun Fire, StorageTek are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries. Information is subject to change without notice.
11/07 SunWIN # 518554
Data Sheet Storage Solutions