Dynamic_Adaptive Systems

Document Sample
Dynamic_Adaptive Systems Powered By Docstoc
					                          Dynamic/Adaptive Systems
            Cross-cut Group, NSF Workshop on Directions for Systems Research
                      Writeup: Andrew A. Chien, achien@cs.uiuc.edu
                                    August 1, 1997
Participants:

         Andrew A. Chien, University of Illinois at Urbana-Champaign
         John Chapin, Massachusetts Institute of Technology
         Peter Reiher, University of California at Los Angeles
         Frank Anger, National Science Foundation
         Bobby Blumofe, University of Texas at Austin

Charter:
         The group discussed opportunities for dynamic adaptation to provide more predictable, more
robust, higher performance, more composable systems. The focus of the group was on multiprogrammed
systems and delivering system performance to real applications. Thus, encompassing issues that involved
applications, operating systems, and hardware architecture.

Key conclusion:
          Technology trends in hardware and software continue to reduce the hardware and runtime cost of
building dynamic, adaptive systems. Concurrently, the increasing dynamic performance range across
processors, networks, and storage hierarchies are dramatically increasing the potential benefits of
customization. Therefore, dynamic/adaptive systems, that is systems that include significant capability for
customization beyond programming in the central processing unit (e.g. in the memory system, I/O system,
networking system), are a promising direction for systems research. We believe this area has the potential
to deliver quantum leaps in capabilities. Dynamic/adaptive systems can include the following
characteristics:
         - programmable engines in I/O devices
         - programmable engines in memory controllers
         - programmable engines in cache controllers
         - programmable engines in network devices
         - configurable data paths / logic in any part of the system
         - configurable queueing in scheduling for interconnect, devices, memory
         - performance instrumentation and feedback for subsystem utilization
         - interfaces for information flow amongst application, operating system, and hardware
         - infrastructure for processing performance data, mechanisms for implementing coordinated
              resource management
         - policies for coordinated resource management

Motivating applications/technologies:
        - rapidly decreasing cost of local computation (gates or transistors)
        - rapidly increasing cost of communication, relative to local computation
        - latency-sensitive applications (best effort not good enough)
        - increasing performance fragility of systems (high Q systems)
        - increasingly dynamic/unpredictable workloads (software mobility)
        - excess resources (enable offline analysis and tuning)

Examples where order of magnitude performance benefits are available today:
       (these and other opportunities continue to increase)
       - adaptation of disk layout, based on access pattern
       - adaptation of virtual memory layout, based on cache miss data
       - adaptation of resource scheduling (processor, memory, I/O, network), to reduce end to end
            latency
         -   adaptation of data compression boundaries, based on where data goes into cache
         -   adaptation of memory hierarchy data units, based on data usage, eliminate false sharing
         -   adaptation of application resource usage, based on projection of resource availability
         -   adaptation of application resource acquisition requests, prediction based on past behavior

Examples of Information Flows to support this Adaptation/Configuration:

                                  Information Flow                    Adaptation to Improve System
Application -> OS                 Large scale data access patterns    Data organization/cache/prefetching
                                                                      Memory/resource alloc/reservation
                                  Performance requirements for        Subsystem scheduling
                                  subsystems
                                  Security requirements (as a         Selection of encryption boundaries,
                                  performance tradeoff)               Transfer paths in the system
Application -> Hardware           Cache performance, Cache            Optimized data movement
                                  protocol control
                                  Dynamic data structures             Custom prefetching and cache block
                                                                      size
                                  Application data representation     Application specific compression in
                                                                      memory/disk controllers
                                  Application encryption keys         Application specific encryption in
                                                                      memory/disk controllers
Hardware ->{OS,Application}       Application-specific memory         Data Reorganization/Page Mapping
                                  hierarchy behavior
                                  Queueing and utilization for        Application-level load management
                                  subsystems                          Operating system load management
                                                                      Resource management and
                                                                      scheduling
                                  Customization specific data ->      Application-level adaptation for
                                  application                         resource and load management, data
                                                                      structures, algorithm, etc.
Network -> {OS, Application}      Load/latency/schedule               WAN resource management, Local
                                  information (cost or available      application adaptation (e.g.
                                  performance)                        prefetching)


Solutions are not just interfaces, but rather interfaces, information formats, analysis, mechanisms for
adaptation, and policies for adaptation. All of these components must be melded together into systems that
enable systematic solutions. Information for adaptation can be collected and passed across interfaces
explicitly, or it can be derived implicitly (for example by observing previous behavior of the program).

To get research started in this area, the best first steps would be to:

1.   Define initial interfaces for information flow across systems
2.   Define interfaces and mechanisms for dynamic extension, enabling experimentation
3.   Produce a reference compiler, OS, HW systems that implement these interfaces and enable
     experimentation