CAVES Configurable Application View Storage System

Document Sample
CAVES Configurable Application View Storage System Powered By Docstoc
					CAVES: Configurable Application
    View Storage System

    Computer Science Department
    Rensselaer Polytechnic Institute
What is CAVES?
• Middleware or client side system connecting
  multiple distributed servers to multiple client
• Goal – reduce overall turnaround time
  between applications and data servers.
• Method – Save and reuse results of prior
  queries in a shared cache controlled by rules
  tailored to specific applications.
   – rules are defined by application programmers
   – the cached queries are called views
CAVES Architecture

        View Use Methods

          Runtime System                Simulation
      Priority    View     Runtime        Model
      Queue      Storage   Statistics

    View Management Methods
View Use Methods
                                                 • Lookup methods –
                                                   determine if a stored
                                                   view can be used to
                                                   answer a new query.
    View Use Methods

                                                 • Filter methods – use a
                                                   stored view to answer a
      Runtime System                Simulation
  Priority    View     Runtime        Model
  Queue      Storage   Statistics
                                                   new more restrictive
View Management Methods                          • Methods are created by
                                                   application developers
                                                   and tailored to specific
View Management Methods
                                                 • Rules define the priority
                                                   of a view based on a
                                                   set of statistics.
    View Use Methods
                                                 • Statistics are updated
      Runtime System
                                                   as views are used.
  Priority    View     Runtime        Model      • Priorities determine
                                                   which views enter the
  Queue      Storage   Statistics

                                                   cache and which views
View Management Methods
                                                   are removed.
                                                 • Statistics and rules are
                                                   defined by application
Runtime System
                                                 • Manages a fixed-size
                                                   disk cache for views.
    View Use Methods                             • A priority queue orders
                                                   the views in the cache
      Runtime System                Simulation     to make removal of the
                                      Model        views with lowest
                                                   priorities efficient.
View Management Methods                          • System statistics are
                                                   used for tuning the
                                                   operation of the cache.
Simulation Model
                                                 • Models the runtime
                                                   system and rules for
    View Use Methods
                                                   computing priorities.
                                                 • Simulates response of
      Runtime System
                                                   the runtime system to
                                                   evaluate effectiveness
  Priority    View     Runtime        Model
  Queue      Storage   Statistics
                                                   of specific rules.
                                                 • Newly discovered rules
View Management Methods
                                                   and rule refinements
                                                   are fed back to improve
                                                   runtime performance.
• CAVES maintains a set of statistics for views
  that are tailored to capture the important
  properties of views for particular applications.
   – each statistic, with initialize and update methods,
     is defined by application developers
• Examples:
   – size of view                 – computation cost
   – average rate of reference    – time admitted
   – total number of hits         – last reference time
Order Formulas
• CAVES computes a priority for a view based
  on the view’s statistics.
  – order formulas define this computation
• Order formulas are tailored to specific
  applications and are specified by application
• Example order formulas:
          computation _ cost                               1
   size * (now  timein  constant )          average network transfer rate

Weight Vector
• Relative weights of order formulas are used for
  computing the priority of a view.
• Let OF = <f1 , f2 , …, fm> be the set of order formula.
• Let W = <w1 , w2 , …, wm> be a weight vector, such
  that w1+ w2 + … + wm = 1.
• Then the priority of view V is:
      Priority (V )  w1 * f1 (V )  w2 * f 2 (V )  ...  wm * f m (V )
• The weight vector is changed to tune the system.
View Management Algorithm
                       New query Q                                            VIEW USE
                     from application
                                                                            Send Q to server
             VIEW USE                                                (network connect + db connect
         Compare with all views                                       + db execute + network send)
           of the same type

                     if found                              VIEW USE                   Entry criteria for V is met
                                        if not        Receive answers V to Q
               VIEW USE                 found
           Find the lowest cost                                                            VIEW MANAGEMENT
         view V that can be used                                                              Write V to file
                                                      Results back to application
                                                                                               Space not
                                                                     Space not available       available

  VIEW USE                 VIEW USE                                      VIEW MANAGEMENT
Read and filter V    Update the statistics for V,                            Remove the lowest
                       remove V from the                                 priority view from storage
                          priority queue                                     and priority queue
                                                                                                Space available

                                           VIEW MANAGEMENT                                    VIEW USE
                                          Insert V into priority queue                Create initial statistics for V
Monitoring and Tuning
• CAVES constantly monitors general system behavior
  and collects and updates statistics to measure the
  effectiveness of the storage management rules.
• This is accomplished through the use of a simulation
  model of the actual system.
• Dynamic tuning is done by making changes to the
  weight vector and/or to the order formulas.
• Weight vector changes are triggered by weight
  change rules:                       Ai is a Boolean expression
                                      defined using the statistics
                           , w, f 
           A1 , ..., Am  *           and relational operators.
                                   
                                     A rule changes the weight
                                            for order formula f.
Potential Applications
• Geographic Information Systems involving zooming
  and panning operations in possibly multi-layer maps.
• Data mining and data cube computations.
• Mobile applications where connections to servers are
• Applications in which prefetching of data is possible
  based on known rules.
• Distributed applications running over slow or highly
  congested networks.
Project Organization
• Participants
   –   Sibel Adali (
   –   Chris Carothers (
   –   David Spooner (
   –   Chi-Nan Chiang (
   –   Garrett Yaun (
   –   Ryan Huebsch (
   –   Jie Zhang (
• Support
   – Funded by the National Science Foundation: Grant Number
     IIS-9876932, September 1999 - August 2002

Shared By: