Docstoc

Prototyping and Analyzing Delegation Models in NFS version 4

Document Sample
Prototyping and Analyzing Delegation Models in NFS version 4 Powered By Docstoc
					Directory Delegations in NFSv4



        Brian Wickman


                 CITI
                 University of Michigan
                 Ann Arbor
NFS version 4

   NFSv4 introduces state!
       OPEN and CLOSE
       integrated locking
       file delegations
   Delegation Paradigm
       Server-driven optimizations visible to the client
        but transparent to applications.
       More     client   responsibility;   but   better
        performance!


                                                     page2
NFSv4 File Delegations

   Server-driven optimization: Server has final
    authority over delegating ability to clients
   Transparent to applications: No special
    application-level tooling required, otherwise
    delegation utility would be undermined.
   Better performance: Local locking! Smart
    caching! Less latency! Less traffic!



                                               page3
NFS Directory Delegations

   Want to delegate some portion of directory
    state to client
   What to delegate?
   Potential state
       Filename to filehandle bindings (DNLC)
       File attributes
       Anything else?




                                                 page4
How to Enable Directory Delegations?

   Introduce pair of ops, OPENDIR/CLOSEDIR?
       Better mirroring of file delegation semantics
       Same interoperability concerns as delegating v4.0 servers
        speaking with v2/v3 clients.
   Introduce single-op?
    GIVE_ME_DELEGATION_PRETTY_PLEASE?
       Half as many ops to implement. Works, too.
   How information-rich? Notifications?
    Asynchronous, synchronous, whaaaa?



                                                               page5
draft-ietf-nfsv4-directory-delegation-01.txt

   Saadia’s proposal
       New op (GET_DIR_DELEGATION)
       New callback (CB_NOTIFY)
   Delegations and notifications, all in one
   Read-only, as specified
   Information rich, fine-grained
   Lots of new functionality to code, especially
    for server implementer


                                                page6
draft-wickman-nfsv4-directory-delegations-00.txt


   My proposal
       New op (REQUEST_DELEGATION)
       No new callbacks
   No notifications, existing data structures
    unchanged
   Read and write delegations specified
   Minimal information, coarse-grained
   Implementation more straightforward?


                                                   page7
Write directory delegations

   Server doesn’t need to change its behavior,
    relies more on client implementer
   If the client is clever it can
       Issue directory updates asynchronously
       Perform whole-tree write delegation on new
        directories
       Effectively own file write delegations on all files
        within a write delegated directory



                                                         page8
Write delegations and STATEIDs

   Overlooked in my draft, need errata
   Directory ops such as READDIR or MKDIR do
    not reference a delegation stateid
   Need PUT_STATEID function and “current”
    STATEID so that we can modify directories
    on which we hold delegations
   Discussed at Connectathon…?



                                          page9
Evaluating the proposals

   For starters, how do you define better?
   My take: how many ops can you service locally
    and at what extra cost, given common
    workloads?

   So far, I’ve investigated this for read-only
    directory delegations and for two common(?)
    workloads.



                                              page10
Steps to Evaluating Delegation Behavior

   Passively capture traces from NFSv3 deployments
    with thousands of users (v4 deployments, anyone?)
   Build an NFS (v3) metadata simulator
   Instrument extra information into traces (e.g.,
    OPEN, CLOSE, REQUEST_DELEGATION)
   Reconstruct filesystem image on the fly
   Observe delegation behavior!
   Measure what you see



                                                    page11
Traces Used
   From Dan Ellard’s PhD research
   CAMPUS trace
       Read dominated e-mail workload
       Good test of notification behavior? (taking pulse of mail
        spool)
       Digital UNIX server, one week, late October 2001, 242M v3
        ops
   EECS trace
       Write-dominated research workload, lots of small files
       NetApp filer, one week, late October 2001, 41M v3 ops



                                                                 page12
Simple use of Directory Delegations

   For fun, I emulated the v3 cache algorithm
    using the REQUEST_DELEGATION op
       When first caching directory entries,
        REQUEST_DELEGATION
       If clock expires and no revocation, do nothing
       If revocation occurs, issue all LOOKUPs to server,
        synchronously until clock expiration
       Reacquire delegation at clock expiration



                                                      page13
Simple use of Directory Delegations

   Use liberal server policy for granting
    directory delegations (i.e., always-grant)
   In both traces, >90%** of LOOKUPs are
    serviced locally and guaranteed consistent
   In both traces, >95%** of directory cache
    validation traffic is eliminated

** - with unbounded lookup caches


                                             page14
What about with notifications?

   I also implemented the same algorithm but
    with GET_DIR_DELEGATION
   We can service a bunch of extra LOOKUPs
    (practically all) because the server notifies
    us when our lookup cache changes and what is
    entailed in each change, rather than revoke
   How much can we service and at what cost?



                                              page15
Not much

                            CAMPUS   EECS

Extra LOOKUPs serviced      162K     14K
locally
Percentage of total LOOKUPs 2.1%     0.4%

Notifications received      9M       164K

Ratio of Notifications to   56x      12x
extra LOOKUPs

                                            page16
FILE_ATTR notifications

   If FILE_ATTR notifications are added, the
    ratio goes from 56x to 500x for CAMPUS

   Millions of DIR_ATTR notifications to
    service tens of thousands of LOOKUPs
   Tens of millions of FILE_ATTR notifications
    (67M) to service millions of GETATTRs
    (2.3M)



                                             page17
Reducing Notification Traffic

   Can we win if using asynchronous notifications?
   If nothing else, we can reduce traffic.
   Because the client can specify a notification window,
    the server has the opportunity to eliminate self-
    nullifying updates (e.g., ADD fh; REMOVE fh) within
    that window.
   Asynchronous notifications would still mean less
    LOOKUP consistency than no notifications at all, as
    long as revocations are synchronous




                                                      page18
Reducing Notification Traffic
                Ratio of Generated to Delivered Notifications*
           45
           40
           35
           30
           25
   Ratio




           20
           15
           10
            5
            0
                0.

                0.

                1

                            3

                                 10

                                      30

                                           10

                                           30

                                           10

                                           30
                  1

                  3




                                             0

                                             0

                                              00

                                              00
                       Notification Window (s)

                             CAMPUS       EECS

                                                                 page19
Another Drawback of Notifications

   How many notifications refer to directory objects
    whose data or metadata we are actively accessing?

   CAMPUS:         4%
   EECS:           <1%

   Are we taking sips from a fire-hose?
   Also, knowing the existence of a change may be
    sufficient more often than knowing the change itself.



                                                     page20
Summary

   A simple directory delegation scheme
    without notifications gives immediate
    improvements in latency and caching, in the
    read-only case
   Notifications have the tendency to create
    lots of information that is otherwise unused
    by the client
   Still much more work to be done…



                                              page21
TODO

   Start looking at v4 traces. Of course, I need v4
    traces, especially Windows v4 traces Anyone want to
    help?
   Simulating write directory delegations
   Looking at other lookup cache maintenance
    algorithms (instead of just clock-based)
   Adding more variables to the simulator (cache
    eviction, for example)
   Implementing directory delegations for real



                                                   page22
Questions?




             page23

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:13
posted:3/20/2011
language:English
pages:23