Docstoc

File System Access

Document Sample
File System Access Powered By Docstoc
					File System Access
(XRootd) Andrew Hanushevsky
Stanford Linear Accelerator Center 13-Jan-03

Goals
High Performance File-Based Access Scalable Extensible Mass Storage integration Flexible Security Dynamically Configurable Rootd Compatable
Jauuary 13, 2003 2: XRootd

Achieving High Performance
Multi-threaded server


Ability to run multiple servers on one host
Last active connections serviced first Socket/thread bound for “active” connections Unlocked I/O, sync reduction, file descriptor partitioning
3: XRootd

MRU polling


Sticky sockets


Using high-performance oofs layer


Jauuary 13, 2003

Achieving Scalability
Socket multiplexing


One connection per client/host


Requires stream id protocol extensions

Dynamic Load Balancing


Requires client redirection protocol extensions File reuse, idle closeouts, cached file system

Using scalable oofs layer


Jauuary 13, 2003

4: XRootd

Achieving Extensibility
Opaque Information Protocol


Protocol extension allows client to relay outboard information to server
Messaging interface  Cache group  Request priority  Other things we haven’t thought of yet…


Jauuary 13, 2003

5: XRootd

Achieving MSS Integration
Using oofs layer


Already integrates with almost any MSS


No brainer

Jauuary 13, 2003

6: XRootd

Achieving Flexible Security
Negotiated Security Protocol Extensions


Allows client/server to agree on protocol


E.g., Kerberos, GSI, AFS Kerberos, etc.

Using oofs layer
Multi-protocol authentication support  Scalable authorization


Jauuary 13, 2003

7: XRootd

Achieving Configurability
Servers must come up/down without harm


Part of dynamic load balancing


Requires client retry extension

Jauuary 13, 2003

8: XRootd

Achieving Compatability
Client sensitive to server version number
Rootd servers default to base TNetFile class  XRootd servers use derived TNetFile class


Server requires client to turn on extensions
Base TNetFile class restricted to rootd ops  Derived TNetFile enables extensions



Multi-streaming, Redirection, Request Deferral, Generic Security, Opaque Information
9: XRootd

Jauuary 13, 2003

xrootd Architecture

Database Protocol Layer

Filesystem Logical Layer
Filesystem Physical Layer Filesystem Implementation
Jauuary 13, 2003 10: XRootd

Some Details
Compatible Protocol


Allows transparent use of rootd & xrootd
Allows multiple files per connection

Multi-Streaming


Cache File System Dynamic Load Balancing


Client directed to “appropriate” server

Configurability & Recoverability
Jauuary 13, 2003 11: XRootd

Compatible Protocol
Application
XTNetFile TNetFile xrootd xrootd
rootd compability

rootd

Jauuary 13, 2003

12: XRootd

Multi-Streaming
Each open() assigned a stream number
Four character id that starts with 0-9  All requests/responses identify relevant stream


Non-open file requests use arbitrary id
Any four characters not starting with 0-9  E.g., stat(), mv(), unlink(), etc.


Parallel Multi-Streaming allowed


Client must explicitly enable it
13: XRootd

Jauuary 13, 2003

Cache File System
/databases/mydbfile

Index Area Optional data cache Default data area
symlink Naming convention allows for audit and index recovery

Multiple Independent Filesystems

/cache1/databases:mydbfile

/cache2

/cache3
Jauuary 13, 2003 14: XRootd

Data Area Any number Any Size Chosen based on free space in LRU order

Dynamic Load Balancing

Dynamic Selection

Jauuary 13, 2003

15: XRootd

DLB Implementation
xrootd
dlbd subscribe open again open Client wait try host:port
Jauuary 13, 2003

xrootd
dlbd I do

xrootd
dlbd (any number)

who has the file?
dlbd xrootd (any number)

16: XRootd

Configurability & Recoverability
Go hand in hand
Configuration changes may be disruptive  Must be able to recover from dynamic configuration changes


Dynamic Load Balancing essential part

Jauuary 13, 2003

17: XRootd

Recoverability
Whenever client looses r/o connection


Back to distinguished xrootd(s) for reselection

Whenever client looses r/w connection


Limited wait/retry loop on the same server


We will be working to improve this next year!

All handled in the XTNetFile class


Disruptions merely delay the client

Jauuary 13, 2003

18: XRootd

Configurability
Since we have recoverability…


Servers may come and go
New servers can be added at any time  Servers may be brought down for maintenance  Files can be moved around in real-time


Client will recover and adjust to the new configuration

Jauuary 13, 2003

19: XRootd

Conclusion
Goals for rootd file access can be met


Most of this work relies on leveraging work done for the AMS

Compatability allows us to lobby the community for xrootd to become the default “rootd”

Jauuary 13, 2003

20: XRootd


				
DOCUMENT INFO
Shared By:
Categories:
Stats:
views:11
posted:12/18/2009
language:English
pages:20