DHS Cybersecurity Research Roadmap

Document Sample
DHS Cybersecurity Research Roadmap Powered By Docstoc
					A Roadmap for Cybersecurity Research




                           November 2009
    Contents
Executive Summary ................................................................................................................................................iii
Introduction ..............................................................................................................................................................v
Acknowledgements .................................................................................................................................................ix
Current Hard Problems in INFOSEC Research
       1. Scalable Trustworthy Systems ...................................................................................................................1
       2. Enterprise-Level Metrics (ELMs) ..........................................................................................................13
       3. System Evaluation Life Cycle...................................................................................................................22
       4. Combatting Insider Threats ....................................................................................................................29
       5. Combatting Malware and Botnets ..........................................................................................................38
       6. Global-Scale Identity Management ........................................................................................................50
       7. Survivability of Time-Critical Systems ..................................................................................................57
       8. Situational Understanding and Attack Attribution ..............................................................................65
       9. Provenance .................................................................................................................................................76
       10. Privacy-Aware Security ..........................................................................................................................83
       11. Usable Security ........................................................................................................................................90
Appendices
Appendix A. Interdependencies among Topics ..............................................................................................A1
Appendix B. Technology Transfer .................................................................................................................... B1
Appendix C. List of Participants in the Roadmap Development .................................................................C1
Appendix D. Acronyms ...................................................................................................................................... D1




                                                                                                                                                                         i
Executive Summary
            Executive Summary

            The United States is at a significant decision point. We must continue to defend our
            current systems and networks and at the same time attempt to “get out in front” of
            our adversaries and ensure that future generations of technology will position us to
            better protect our critical infrastructures and respond to attacks from our adversaries.
            The term “system” is used broadly to encompass systems of systems and networks.

            This cybersecurity research roadmap is an attempt to begin to define a national R&D
            agenda that is required to enable us to get ahead of our adversaries and produce
            the technologies that will protect our information systems and networks into the
            future. The research, development, test, evaluation, and other life cycle consider-
            ations required are far reaching—from technologies that secure individuals and
            their information to technologies that will ensure that our critical infrastructures
            are much more resilient. The R&D investments recommended in this roadmap
            must tackle the vulnerabilities of today and envision those of the future.

            The intent of this document is to provide detailed research and development
            agendas for the future relating to 11 hard problem areas in cybersecurity, for use
            by agencies of the U.S. Government and other potential R&D funding sources.
            The 11 hard problems are:

            1. Scalable trustworthy systems (including system architectures and requisite
                development methodology)
            2. Enterprise-level metrics (including measures of overall system trustworthiness)
            3. System evaluation life cycle (including approaches for sufficient assurance)
            4. Combatting insider threats
            5. Combatting malware and botnets
            6. Global-scale identity management
            7. Survivability of time-critical systems
            8. Situational understanding and attack attribution
            9. Provenance (relating to information, systems, and hardware)
            10. Privacy-aware security
            11. Usable security

            For each of these hard problems, the roadmap identifies critical needs, gaps in
            research, and research agenda appropriate for near, medium, and long term
            attention.

            DHS S&T assembled a large team of subject matter experts who provided input
            into the development of this research roadmap. The content was developed over
            the course of 15 months that included three regional multi-day workshops, two
            virtual workshops for each topic, and numerous editing activities by the participants.


                                                                                                 iii
 Introduction
                                    Introduction

                                    Information technology has become pervasive in every way—from our phones and
                                    other small devices to our enterprise networks to the infrastructure that runs our
                                    economy. Improvements to the security of this information technology are essential
                                    for our future. As the critical infrastructures of the United States have become more
                                    and more dependent on public and private networks, the potential for widespread
                                    national impact resulting from disruption or failure of these networks has also
                                    increased. Securing the nation’s critical infrastructures requires protecting not only
                                    their physical systems but, just as important, the cyber portions of the systems on
                                    which they rely. The most significant cyber threats to the nation are fundamentally
                                    different from those posed by the “script kiddies” or virus writers who tradition-
                                    ally have plagued users of the Internet. Today, the Internet has a significant role
                                    in enabling the communications, monitoring, operations, and business systems
                                    underlying many of the nation’s critical infrastructures. Cyberattacks are increas-
                                    ing in frequency and impact. Adversaries seeking to disrupt the nation’s critical
                                    infrastructures are driven by different motives and view cyberspace as a possible
                                    means to have much greater impact, such as causing harm to people or widespread
                                    economic damage. Although to date no cyberattack has had a significant impact on
                                    our nation’s critical infrastructures, previous attacks have demonstrated that exten-
                                    sive vulnerabilities exist in information systems and networks, with the potential for
                                    serious damage. The effects of a successful attack might include serious economic
                                    consequences through impacts on major economic and industrial sectors, threats
                                    to infrastructure elements such as electric power, and disruptions that impede the
                                    response and communication capabilities of first responders in crisis situations.

                                    The United States is at a significant decision point. We must continue to defend our
                                    current systems and networks and at the same time attempt to “get out in front”
                                    of our adversaries and ensure that future generations of technology will position
                                    us to better protect our critical infrastructures and respond to attacks from our
                                    adversaries. It is the opinion of those involved in creating this research roadmap that
                                    government-funded research and development (R&D) must play an increasing role
                                    to enable us to accomplish this goal of national and economic security. The research
                                    topics in this roadmap, however, are relevant not only to the federal government
                                    but also to the private sector and others who are interested in securing the future.

                                    This cybersecurity research roadmap is an attempt to begin to define a national R&D
                                    agenda that is required to enable us to get ahead of our adversaries and produce
                                    the technologies that will protect our information systems and networks into the
                                    future. The research, development, test, evaluation, and other life cycle consider-
                                    ations required are far reaching—from technologies that secure individuals and
                                    their information to technologies that will ensure that our critical infrastructures
“The time is now near at hand...”   are much more resilient. These investments must tackle the vulnerabilities of today
— George Washington, July 2, 1776   and envision those of the future.


                                                                                                                        v
Historical background                       research programs. The original list has     mixes of legacy systems), and the pres-
                                            proven useful in guiding INFOSEC             ence of significant, asymmetric threats.
The INFOSEC Research Council (IRC)          research, and policy makers and planners
is an informal organization of govern-      may find the document useful in evalu-       The area of cybersecurity and the associ-
ment program managers who sponsor           ating the contributions of ongoing and       ated research and development activities
information security research within the    proposed INFOSEC research programs.          have been written about frequently over
U.S. Government. Many organizations         However, the significant evolution of        the past decade. In addition to both
have representatives as regular members     technology and threats between 1999          the original IRC HPL in 1999 and the
of the IRC: Central Intelligence Agency,    and 2005 required an update to the list.     revision in 2005, the following reports
Department of Defense (including the        Therefore, an updated version of the         have discussed the need for investment
Air Force, Army, Defense Advanced           HPL was published in November 2005.          in this critical area:
Research Projects Agency, National          This updated document included the
                                                                                           ƒ Toward a Safer and More Secure
Reconnaissance Office, National Secu-       following technical hard problems from
rity Agency, Navy, and Office of the        the information security perspective:             Cyberspace
Secretary of Defense), Department                                                          ƒ Federal Plan for Cyber Security
                                            1. Global-Scale Identity Management               and Information Assurance
of Energy, Department of Homeland
Security, Federal Aviation Administra-      2. Insider Threat                                 Research and Development
tion, Intelligence Advanced Research        3. Availability of Time-Critical               ƒ Cyber Security: A Crisis of
Projects Activity, National Aeronautics        Systems
                                                                                              Prioritization
and Space Administration, National          4. Building Scalable Secure Systems
                                                                                           ƒ Hardening the Internet
Institutes of Health, National Institute    5. Situational Understanding and
of Standards and Technology, National          Attack Attribution                          ƒ Information Security
Science Foundation, and the Technical       6. Information Provenance                         Governance: A Call to Action
Support Working Group. In addition,                                                        ƒ The National Strategy to Secure
                                            7. Security with Privacy
the IRC is regularly attended by partner                                                      Cyberspace
organizations from Canada and the           8. Enterprise-Level Security Metrics
                                                                                           ƒ Cyber Security Research and
United Kingdom.
                                                                                              Development Agenda
                                            These eight problems were selected
The IRC developed the original Hard         as the hardest and most critical chal-
Problem List (HPL), which was com-          lenges that must be addressed by the         These reports can be found at http://
posed in 1997 and published in draft        INFOSEC research community if trust-         www.cyber.st.dhs.gov/documents.html
form in 1999. The HPL defines desir-        worthy systems envisioned by the U.S.
able research topics by identifying a set   Government are to be built. INFOSEC          Current context
of key problems from the U.S. Govern-       problems may be characterized as “hard”
ment perspective and in the context of      for several reasons. Some problems are       On January 8, 2008, the President
IRC member missions. Solutions to           hard because of the fundamental techni-      issued National Security Presiden-
these problems would remove major           cal challenges of building secure systems,   tial Directive 54/Homeland Security
barriers to effective information secu-     others because of the complexity of          Presidential Directive 23, which for-
rity (INFOSEC). The Hard Problem            information technology (IT) system           malized the Comprehensive National
List was intended to help guide the         applications. Contributing to these          Cybersecurity Initiative (CNCI) and a
research program planning of the IRC        problems are conflicting regulatory and      series of continuous efforts designed to
member organizations. It was also hoped     policy goals, poor understanding of          establish a frontline defense (reducing
that nonmember organizations and            operational needs and user interfaces,       current vulnerabilities and preventing
industrial partners would consider these    rapid changes in technology, large het-      intrusions), defending against the full
problems in the development of their        erogeneous environments (including           spectrum of threats by using intelligence

 vi
and strengthening supply chain security,    influence in networking and IT systems,      interagency coordination to ensure cov-
and shaping the future environment by       components, and standards among U.S.         erage of all the topics.
enhancing our research, development,        competitors. Federal agencies with
and education, as well as investing in      mission-critical needs for increased         Each of the following topic areas is
“leap-ahead” technologies.                  cybersecurity, which includes informa-       treated in detail in a subsequent section
                                            tion assurance as well as network and        of its own, from Section 1 to Section 11.
The vision of the CNCI research com-        system security, can play a direct role
                                                                                         1. Scalable trustworthy systems
munity over the next 10 years is to         in determining research priorities and
                                                                                             (including system architectures and
“transform the cyber-infrastructure so      assessing emerging technology proto-
                                                                                             requisite development methodol-
that critical national interests are pro-   types. Moreover, through technology
                                                                                             ogy)
tected from catastrophic damage and         transfer efforts, the federal government
                                                                                         2. Enterprise-level metrics (including
our society can confidently adopt new       can encourage rapid adoption of the
                                                                                             measures of overall system trust-
technological advances.”                    results of leap-ahead research. Technol-
                                                                                             worthiness)
                                            ogy breakthroughs that can curb or
Two components of the CNCI deal             break the resource-draining cycle of         3. System evaluation life cycle (in-
                                                                                             cluding approaches for sufficient
with cybersecurity research and develop-    security patching will have a high likeli-
                                                                                             assurance)
ment—one focused on the coordination        hood of marketplace implementation.
of federal R&D and the other on the                                                      4. Combatting insider threats
development of leap-ahead technologies.     As stated previously, this Cybersecu-        5. Combatting malware and botnets
                                            rity Research Roadmap is an attempt          6. Global-scale identity management
No single federal agency “owns” the         to begin to address a national R&D           7. Survivability of time-critical
issue of cybersecurity. In fact, the        agenda that is required to enable us to          systems
federal government does not uniquely        get ahead of our adversaries and produce     8. Situational understanding and
own cybersecurity. It is a national and     the technologies that will protect our           attack attribution
global challenge with far-reaching con-     information systems and networks into        9. Provenance (relating to informa-
sequences that requires a cooperative,      the future. The topics contained in this         tion, systems, and hardware)
comprehensive effort across the public      roadmap and the research and develop-
                                                                                         10. Privacy-aware security
and private sectors. However, as it has     ment that would be accomplished if the
                                                                                         11. Usable security
done historically, U.S. Government          roadmap were implemented are, in fact,
R&D in key technologies working in          leap-ahead in nature and address many
close cooperation with private-sector       of the topics that have been identified      Eight of these topics (1, 2, 4, 6, 7, 8,
partners can jump-start the necessary       in the CNCI activities                       9, 10) are adopted from the November
fundamental technical transformation.                                                    2005 IRC Hard Problem List [IRC05]
                                            Document format                              and are still of vital relevance. The
The leap-ahead strategy aligns with the                                                  other three topics (3, 5, 11) represent
consensus of the nation’s networking        The intent of this document is to            additional areas considered to be of
and cybersecurity research communi-         provide detailed research and develop-       particular importance for the future.
ties that the only long-term solution to    ment agendas for the future relating to
the vulnerabilities of today’s network-     11 hard problem areas in cybersecurity,      The order in which the 11 topics are
ing and information technologies is to      for use by agencies of the U.S. Govern-      presented reflects some structural simi-
ensure that future generations of these     ment and anyone else that is funding         larities among subgroups of the topics
technologies are designed with secu-        or doing R&D. It is expected that each       and exhibits clearly some of their major
rity built in from the ground up. The       agency will find certain parts of the        interdependencies. The order proceeds
leap-ahead strategy will help extend        document resonant with its own needs         roughly from overarching system con-
U.S. leadership at a time of growing        and will proceed accordingly with some       cepts to more detailed issues—except

                                                                                                                               vii
for the last topic—and has the following   Background                               ƒ What R&D is evolutionary and
structure:                                                                             what is more basic, higher risk,
                                             ƒ What is the problem being               game changing?
a. Topics 1–3 frame the overarching            addressed?
                                                                                    ƒ Resources
   problems.                                 ƒ What are the potential threats?
                                                                                    ƒ Measures of success
b. Topics 4–5 relate to specific major       ƒ Who are the potential
                                               beneficiaries? What are their        ƒ What needs to be in place for test
   threats and needs.
                                               respective needs?                       and evaluation?
c. Topics 6–10 relate to some of the
                                             ƒ What is the current state of the     ƒ To what extent can we test real
   “ilities” and to system concepts
                                               practice?                               systems?
   required for implementing the
   previous topics.                          ƒ What is the status of current      Following the 11 sections are three
                                               research?                          appendices:
Topic 11, usable security, is different
from the others in its cross-cutting       Future Directions                      Appendix A: Interdependencies among
nature. If taken seriously enough, it                                             Topics
                                             ƒ On what categories can we
can influence the success of almost all
the other topics. However, some sort           subdivide the topics?              Appendix B: Technology Transfer
of transcendent usability requirements       ƒ What are the major research
need to be embedded pervasively in all         gaps?                              Appendix C: List of Participants in the
the other topics.                            ƒ What are some exemplary            Roadmap Development
                                               problems for R&D on this topic?
Each of the 11 sections follows a
                                             ƒ What are the challenges that
similar format. To get a full picture of
                                               must be addressed?
the problem, where we are, and where
we need to go, we ask the following          ƒ What approaches might be
questions:                                     desirable?


References
[IRC2005]        INFOSEC Research Council Hard Problem List, November 2005
                 http://www.cyber.st.dhs.gov/docs/IRC_Hard_Problem_List.pdf.

[USAF-SAB07] United States Air Force Scientific Advisory Board, Report on Implications of Cyber Warfare. Volume 1:
             Executive Summary and Annotated Brief; Volume 2: Final Report, August 2007. For Official Use Only.

Additional background documents (including the two most recent National Research Council study reports on cybersecurity)
can be found online. (http://www.cyber.st.dhs.gov/documents.html).




viii
Acknowledgements
           Acknowledgements

           The content of this research roadmap was developed over the course of 15 months
           that included three workshops, two phone sessions for each topic, and numer-
           ous editing activities by the participants. Appendix C lists all the participants.
           The Cyber Security program of the Department of Homeland Security (DHS)
           Science and Technology (S&T) Directorate would like to express its appre-
           ciation for the considerable amount of time they dedicated to this effort.

           DHS S&T would also like to acknowledge the support provided by the staff of SRI
           International in Menlo Park, CA, and Washington, DC. SRI is under contract with
           DHS S&T to provide technical, management, and subject matter expert support for
           the DHS S&T Cyber Security program. Those involved in this effort include Gary
           Bridges, Steve Dawson, Drew Dean, Jeremy Epstein, Pat Lincoln, Ulf Lindqvist,
           Jenny McNeill, Peter Neumann, Robin Roy, Zach Tudor, and Alfonso Valdes.

           Of particular note is the work of Jenny McNeill and Peter Neumann. Jenny
           has been responsible for the organization of each of the workshops and phone
           sessions and has worked with SRI staff members Klaus Krause, Roxanne Jones,
           and Ascencion Villanueva to produce the final document. Peter Neumann
           has been relentless in his efforts to ensure that this research roadmap rep-
           resents the real needs of the community and has worked with roadmap
           participants and government sponsors to produce a high-quality product.




                                                                                          ix
Current Hard Problems in INFOSEC Research
             1. Scalable Trustworthy Systems
             BACKGROUND

             What is the problem being addressed?
             Trustworthiness is a multidimensional measure of the extent to which a system is
             likely to satisfy each of multiple aspects of each stated requirement for some desired
             combination of system integrity, system availability and survivability, data confi-
             dentiality, guaranteed real-time performance, accountability, attribution, usability,
             and other critical needs. Precise definitions of what trustworthiness means for these
             requirements and well-defined measures against which trustworthiness can be evalu-
             ated are fundamental precursors to developing and operating trustworthy systems.
             These precursors cut across everything related to scalable trustworthy systems. If
             what must be depended on does not perform according to its expectations, then
             whatever must depend on it may itself not be trustworthy. A trusted system is
             one that must be assumed to satisfy its requirements—whether or not it is actu-
             ally trustworthy; indeed, it is a system whose failure in any way may compromise
             those requirements. Unfortunately, today’s systems are typically not well suited for
             applications with critical trustworthiness requirements.

             Scalability is the ability to satisfy given requirements as systems, networks, and
             systems of systems expand in functionality, capacity, complexity, and scope of trust-
             worthiness requirements security, reliability, survivability, and improved real-time
             performance. Scalability must typically be addressed from the outset; experience
             shows that scalability usually cannot be retrofitted into systems for which it was
             not an original design goal. Scalable trustworthiness will be essential for many
             national- and world-scale systems, including those supporting critical infrastructures.
             Current methodologies for creating high-assurance systems do not scale to the size
             of today’s—let alone tomorrow’s—critical systems.

             Composability is the ability to create systems and applications with predictably
             satisfactory behavior from components, subsystems, and other systems. To enhance
             scalability in complex distributed applications that must be trustworthy, high-
             assurance systems should be developed from a set of composable components and
             subsystems, each of which is itself suitably trustworthy, within a system architecture
             that inherently supports facile composability. Composition includes the ability to
             run software compatibly on different hardware, aided considerably by abstraction,
             operating systems, and suitable programming languages. However, we do not yet
             have a suitable set of trustworthy building blocks, composition methodologies,
             and analytic tools that would ensure that trustworthy systems could be developed
             as systems of other systems. In addition, requirements and evaluations should
             also compose accordingly. In the future, it will be vital that new systems can be
             incrementally added to a system of systems with some predictable confidence that
             the trustworthiness of the resulting systems of systems will not be weakened—or
             indeed that it may be strengthened.

                                                                                                 1
Growing interconnectedness among            follows: (1) trustworthiness, (2) com-        computing base that would provide a
existing systems results, in effect, in     posability, and (3) scalability. Thus, the    suitable foundation for such computing.
new composite systems at increasingly       challenge addressed here is threefold:        However, this assumption has not been
large scales. Existing hardware, operat-    (a) to provide a sound basis for compos-      justified. In the future, we must be able
ing system, networking, and application     ability that can scale to the development     to develop scalable trustworthy systems
architectures do not adequately account     of large and complex trustworthy              effectively.
for combined requirements for security,     systems; (b) to stimulate the develop-
performance, and usability—confound-        ment of the components, analysis tools,       Who are the potential
ing attempts to build trustworthy           and testbeds required for that effort;
                                                                                          beneficiaries? What are their
systems on them. As a result, today the     and (c) to ensure that trustworthiness
                                                                                          respective needs?
security of a system of systems may be      evaluations themselves can be composed.
drastically less than that of most of its                                                 Large organizations in all sectors—for
components.                                 What are the potential                        example, government, military, com-
                                                                                          mercial, financial, and energy—suffer
                                            threats?
In certain cases, it may be possible                                                      the consequences of using large-scale
to build systems that are more trust-       Threats to a system in operation include      computing systems whose trustworthi-
worthy than some (or even most)             everything that can prevent critical appli-   ness either is not assured or is potentially
of their components—for example,            cations from satisfying their intended        compromised because of costs that
through constructive system design and      requirements, including insider and out-      outweigh the perceived benefits. All
meticulous attention to good software       sider misuse, malware and other system        stakeholders have requirements for
engineering practices. Techniques for       subversions, software flaws, hardware         confidentiality, integrity, and availabil-
building more trustworthy systems out       malfunctions, human failures, physical        ity in their computing infrastructures,
of less trustworthy components have         damage, and environmental disruptions.        although the relative importance of
long been known and used in practice        Indeed, systems sometimes fail without        these requirements varies by application.
(e.g., summarized in [Neu2004], in the      any external provocation, as a result         Achieving scalability and evolvability of
context of composability). For example,     of design flaws, implementation bugs,         systems without compromising trust-
error-correcting codes can overcome         misconfiguration, and system aging.           worthiness is a major need. Typical
unreliable communications and storage       Additional threats arise in the system        customers include the following:
media, and encryption can be used to        acquisition and code distribution pro-
                                                                                             ƒƒ Large-system developers (e.g., of
increase confidentiality and integrity      cesses. Serious security problems have
despite insecure communication chan-        also resulted from discarded or stolen             operating systems, database
nels. These techniques are incomplete by    systems. For large-scale systems consist-          management systems, national
themselves and generally ignore many        ing of many independent installations              infrastructures such as the power
security threats. They typically depend     (such as the Domain Name System,                   grid)
on the existence of some combination        DNS), security updates must reach and            ƒƒ Application developers
of trustworthy developers, trustwor-        be installed in all relevant components          ƒƒ Microelectronics developers
thy systems, trustworthy users, and         throughout the entire life cycle of the
                                                                                             ƒƒ System integrators
trustworthy administrators, and their       systems. This scope of updating has proven
trustworthy embedding in those systems.     to be difficult to achieve.                      ƒƒ Large- and small-scale users
                                                                                       ƒƒ Purveyors of potential exemplar
The primary focus of this topic area is     Critical systems and their operating envi-    applications for scalable
scalability that preserves and enhances     ronments must be trustworthy despite a        trustworthiness
trustworthiness in real systems. The per-   very wide range of adversities and adver-
ceived order of importance for research     saries. Historically, many system uses Several types of systems suggest the
and development in this topic area is as    assumed the existence of a trustworthy importance of being able to develop

 2    SCALABLE TRUSTWORTHY SYSTEMS
scalable trustworthy systems. Examples     What is the current state of              insufficient in the long run. Research
include the following:                     the practice?                             is needed to establish the repertoire of
                                                                                     architected hardware protections that
   ƒƒ Air traffic control systems          Hardware developers have recently made are essential for system trustworthiness.
   ƒƒ Power grids                          significant investments in specification, It is unlikely that software alone can ever
   ƒƒ Worldwide funds transfer systems formal methods, configuration control, compensate fully for the lack of such
                                           modeling, and prediction, partly in hardware protections.
   ƒƒ Cellphone networks
                                           response to recognized problems, such
Such systems need to be robust and as the Intel floating point flaw, and A possible implication is that the com-
capable of satisfying the perceived trust- partly as a result of increased demon- mercial off-the-shelf (COTS) systems in
worthiness requirements. Outages in strations of the effectiveness of those pervasive use today will never become
these systems can be extremely costly techniques.                                    sufficiently trustworthy. If that is indeed
and dangerous. However, the extent to                                                true, testing that implication should be
which the underlying concepts used to The foundation for trustworthy scalable identified as an activity and milestone
build these existing systems can continue systems is established by the underly- in the recommended research agenda.
to scale and also be responsive to more ing hardware architecture. Adequate
exacting trustworthiness requirements hardware protections are essential, and Convincing hardware manufacturers
is unknown—especially in the face of nearly all extant hardware architectures and software developers to provide and
increasing cyberthreats. The R&D must lack needed capabilities. Examples support needed hardware capabilities,
provide convincing arguments that they include fine-grain memory protec- of course, is a fundamental obstacle.
will scale appropriately. Exemplars of tion, inaccessible program control state, The manufacturers’ main motivations
potential component systems might unmodifiable executable codes, fully are least change and time to market.
include the following:                     granular access protections, and virtu- Until compelling research findings, legal
                                           ally mapped bus access by I/O and consequences (e.g., financial liability
   ƒƒ Trustworthy handheld                 other adapter boards.                     for customer damages), and economic
      multipurpose devices and other                                                 forces (e.g., purchase policies mandat-
      end-user devices                     Although it might be appealing to try ing the needed capabilities) are brought
   ƒƒ Trustworthy special-purpose          to apply those approaches to software, to bear, it seems unlikely that goals for
      servers                              the issues of scalability suggest that the securing COTS and open source
   ƒƒ Embedded control systems             additional approaches may be necessary. products can be realized.
      that can be composed and used        Numerous software-related failures
      effectively                          have occurred (e.g., see [Neu1995]). What is the status of current
                                           In addition, techniques are needed to
   ƒƒ Trustworthy networks                                                           research?
                                           address how software/hardware inter-
   ƒƒ Navigation systems, such as          actions affect the overall trust level. Over the past decade, significant com-
      the Global Positioning Systems       Unfortunately, there is no existing puter security investments have been
      (GPS)                                mandate for significant investment made in attempts to create greater
                                           during software system development to assurance for existing applications
One or more such systems should be ensure scalable trustworthiness. Conse- and computer-based enterprises that
chosen for deeper study to develop quently, such efforts are generally not are based predominantly on COTS
better understanding of the approaches adequately addressed.                         components. Despite some progress,
to scalable security developed in this                                               there are severe limits to this approach,
program. In turn, the results of ongoing Diagnostic tools to detect software and success has been meager at best,
work on scalable trustworthiness should flaws on today’s hardware architectures particularly with respect to trustwor-
be applied to those and other exemplars. may be useful in the short run but are thiness, composability, and scalability.

                                                                         SCALABLE TRUSTWORTHY SYSTEMS                        3
The assurance attainable by incremental   example of how trustworthy com-            In recent years, research has advanced
improvements on COTS products is          puting systems can be designed and         significantly in formal methods appli-
fundamentally inadequate for critical     built. It will make all elements of the    cable to software trustworthiness. That
applications.                             constructive security process openly       research is generally more applicable to
                                          available. Recent advances in cryptog-     new systems rather than to being retro-
Various research projects over the past raphy can also help, although some           fitted into existing systems. However, it
half-century have been aimed at the composability issues remain to be                needs to focus on attributes and subsys-
challenge of designing and evaluating resolved as to how to embed those              tems for which it can be most effective,
scalable trustworthy systems and net- advances securely into marginally              and must deal with complexity, scal-
works, with some important research secure computer systems. Also, public            ability, hardware and software, and
contributions with respect to both key infrastructures (PKIs) are becom-             practical issues such as device drivers
hardware and software. Some of these ing more widely used and embedded               and excessive root privileges.
date back to the 1960s and 1970s, such in applications. However, many gaps
as Multics, PSOS (the Provably Secure remain in reusable requirements for
Operating System) and its formally trustworthiness, system architectures,            FUTURE DIRECTIONS
based Hierarchical Development Meth- software engineering practices, sound
odology (HDM), the Blacker system programming languages that avoid                   On what categories can we
as an early example of a virtual private many of the characteristic flaws, and
                                                                                     subdivide this topic?
network, the CLInc (Computational analysis tools that scale up to entire
Logic, Inc.) stack, Gypsy, InaJo, Euclid, systems. Thoroughly worked examples        For present purposes, different
ML and other functional programming of trustworthy systems are needed that           approaches to development of trustwor-
languages, and the verifying compiler, can clearly demonstrate that well-con-        thy scalable systems are associated with
to name just a few. However, very few ceived composability can enhance both          the following three roadmap categories.
systems available today have taken trustworthiness and scalability. For              These categories are distinguished from
serious advantage of such potentially example, each of the exemplars noted           one another roughly based on the extent
far-reaching research efforts, or even above would benefit greatly from the          to which they are able to reuse existing
the rather minimal guidance of Security incorporation of scalable trustworthy        components.
Level 4 in FIPS 140-1. Also, the valued systems.
but inadequately observed 1975 secu-                                                 1. Improving trustworthiness in
rity principles of Saltzer and Schroeder At present, even for small systems, there   existing systems. This incremental
have recently been updated by Saltzer exist very few examples of requirements,       approach could entail augmenting rela-
and Kaashoek [Sal+2009].                  trustworthiness metrics, and opera-        tively untrustworthy systems with some
                                          tional systems that encompass a broad      trustworthy components and enforcing
Some more recent efforts can also be spectrum of trustworthiness with any            operational constraints in attempts to
cited here. For example, architectures generality. Furthermore, such require-        achieve either trustworthy functions or
exist or are contemplated for robust ments, metrics, and systems need to             systems with more clearly understood
hardware that would inherently be composable and scalable into trust-                trust properties. Can we make existing
increase system trustworthiness worthy systems of systems. However,                  systems significantly more trustworthy
by avoiding common vulnerabilities, a few examples exist for dedicated               without wholesale replacement?
including modernized capability- special-purpose systems, such as data
based architectures. In addition, the diodes enforcing one-way communi-              2. Clean-slate approaches. This entails
Trusted Computing Exemplar Project cation paths and the Naval Research               building trustworthy primitives, com-
at the Naval Postgraduate School Laboratory Pump enabling trustworthy                posing them into trustworthy functions,
(http://cisr.nps.edu/projects/tcx.html) reading of information at lower levels       and then verifying the overall trust level
is intended to provide a working of multilevel security.                             of the composite system. How much

 4    SCALABLE TRUSTWORTHY SYSTEMS
better would this be? Would this enable    controlled. A clean-slate approach tol-     practices that can yield greater trust-
solutions of problems that cannot be       erating an ongoing level of continuous      worthiness. See also [Can2001], which
adequately addressed today, and for        compromise in its system components         represents the beginning of work on
what requirements? Under what circum-      might also be viewed as a hybrid of         the notion of universal composability
stances and for what requirements might    categories 2 and 3. Further R&D is          applied to cryptography.
this be possible? What new technologies,   clearly required to determine the trade-
system architectures, and tools might      offs in cost-effectiveness, practicality,   However, there are gaps in our under-
be needed?                                 performance, usability, and relative        standing of composability as it relates
                                           trustworthiness attainable for any par-     to security, and to trustworthiness more
3. Operating successfully for given        ticular set of requirements. DARPA’s        generally, primarily because we lack
requirements despite the presence          IAMANET is a step in that direction.        precise specifications of most of the
of partially untrusted environments.                                                   important requirements and desired
For example, existing computing        An urgent need exists for R&D on                properties. For example, we are often
systems might be viewed as “enemy      incremental, clean-slate, and hybrids           good at developing specific solutions
territory” because they have been      approaches. Trustworthiness issues may          to specific security problems, but we
subject to unknown influences within   affect the development process and the          do not understand how to apply and
the commercial supply chain and the    resulting system performance. Adding            combine these specific solutions to
overall life cycle (design, implementa-functionality and concomitant com-              produce trustworthy systems. We lack
tion, operations, maintenance, and     plexity to achieve trustworthiness may          methods for analyzing how even small
decommissioning).                      be counterproductive, if not done con-          changes to systems affect their trust-
                                       structively; it typically merely introduces     worthiness. More broadly, we lack a
It is inherently impossible to control new vulnerabilities. Trustworthiness            good understanding of how to develop
every aspect of the entire life cycle must be designed in from the outset              and maintain trustworthy systems com-
and the surrounding operational envi- with complete specified requirements.            prehensively throughout the entire life
ronments. For example, end-to-end Functionality and trustworthiness are                cycle. We lack methods and tools for
cryptography enables communications inherently in conflict in the design               decomposing high-level trustworthiness
over untrustworthy media—but does process, and this conflict must be                   goals into specific design requirements,
not address denial-of-service attacks resolved before any implementation.              capturing and specifying security require-
en route or insider subversion at the                                                  ments, analyzing security requirements,
endpoints.                             What are the major research                     mapping higher-layer requirements into
                                                                                       lower-layer ones, and verifying system
                                           gaps?
The three categories are not intended                                                  trustworthiness properties. We do not
to be mutually exclusive. For example,     Research relating to composability has      understand how to combine systems in
hybrid approaches can combine legacy       addressed some of the fundamental           ways that ensure that the combination is
systems from category 1 with incremen-     problems and underlying theory. For         more, rather than less, secure and resil-
tal changes and significant advances       example, see [Neu2004] for a recent         ient than its weakest components. We
from category 2. Indeed, hybrids among     consideration of past work, current prac-   lack a detailed case history of past suc-
these three categories are not merely      tice, and R&D directions that might be      cesses and failures in the development
possible but quite likely. For example,    useful in the future. It contains numer-    of trustworthy systems that could help
approaches that begin with a clean-slate   ous references to papers and reports        us to elucidate principles and properties
architecture could also incorporate some   on composability. It also considers a       of trustworthy systems, both in an over-
improvements of existing systems, and      variety of techniques for compositions      arching sense and in specific application
even allow some operations to take place   of subsystems that can increase trustwor-   areas. We lack development tools and
in untrusted environments—if suitably      thiness, as well as system and network      languages that could enable separation
encapsulated, confined, or otherwise       architectures and system development        of functionality and trustworthiness

                                                                        SCALABLE TRUSTWORTHY SYSTEMS                          5
concerns for developers. For small              for composing trustworthy               ƒƒ More extensive detailed worked
systems, ad hoc solutions seldom suffice        systems                                   examples.
if they do not reflect such fundamental
                                             ƒƒ Well-defined composable
understanding of the problems. For the                                               Several threads could run through this
                                                specifications for requirements
large-scale, highly complex systems of                                               timeline—for example, R&D relating
                                                and components
the future, we cannot expect to achieve                                              to trustworthy isolation, separation,
adequate trustworthiness without             ƒƒ Realistic provable security          and virtualization in hardware and
deeper understanding, better tools, and         properties for small-scale systems   software; composability of designs and
more reliable evaluation methods—as          ƒƒ Urgent need for detailed worked      implementations; analyses that could
well as composable building blocks and          examples                             greatly simplify evaluation of trustwor-
well-documented, worked examples of                                                  thiness before putting applications into
less complex systems.                        ƒƒ Better understanding of the          operation; robust architectures that
                                                security properties of existing      provide self-testing, self-diagnosing,
The research directions can be parti-           major components.                    self-reconfiguring, compromise resil-
tioned into near-term, medium-term,                                                  ient, and automated remediation; and
                                           Medium term
and long-term opportunities. In general,                                             architectures that break the current
                                            ƒƒ New hardware with well-
the near-term approaches fall into the                                               asymmetric advantage for attackers
                                               understood trustworthiness
incremental category, and the long-                                                  (offense is cheaper than defense, at
                                               properties
term approaches fall into clean-slate                                                present). The emphasis needs to be
and hybrid categories. However, the         ƒƒ Better operating systems and          on realistic, practical approaches to
long-term approaches often have staged         networking                            developing systems that are scalable,
efforts that begin with near-term efforts.  ƒƒ Better application architectures
                                                                                     composable, and trustworthy.
Also, the hybrid efforts tend to require
                                               for trustworthy systems
longer-term schedules because some of                                                The gaps in practice and R&D,
them rely on near- and medium-term          ƒƒ Isolation of legacy systems           approaches, and potential benefits are
efforts.                                       in trustworthy virtualization         summarized in Table 1.1. The research
                                               environments                          directions for scalable trustworthy
Near term                                    ƒƒ Continued research in                systems are intended to address these
  ƒƒ Development of prototype                   composability, techniques for        gaps. Table 1.2 also provides a summary
     trustworthy systems in selected            verifying the security properties    of this section.
     application and infrastructure             of composed systems in terms of
                                                                                     This topic area interacts strongly with
     domains                                    their specifications
                                                                                     enterprise-level metrics (Section 2) and
     ƒƒ Exploitation of cloud                ƒƒ Urgent need for detailed realistic   evaluation methodology (Section 3) to
       architectures and web-based              and practical worked examples.       provide assurance of trustworthiness.
       applications                        Long term                                 In the absence of such metrics and suit-
     ƒƒ Development of simulation            ƒƒ Tools for verifying                  able evaluation methodologies, security
       environments for testing                                                      would be difficult to comprehend, and
                                                trustworthiness of composite
       approaches to development of                                                  the cost-benefit trade-offs would be
                                                systems
       scalable trustworthy systems                                                  difficult to evaluate. In addition, all the
                                             ƒƒ Techniques and tools for             other topic areas can benefit from scal-
     ƒƒ Intensive further research in           developing and maintaining           able trustworthy systems, as discussed
       composability                            trustworthy systems throughout       in Appendix A.
     ƒƒ Development of building blocks          the life cycle


 6      SCALABLE TRUSTWORTHY SYSTEMS
                         TABLE 1.1: Summary of Gaps, Approaches, and Benefits

 Concept           GapsƒinƒPractice                GapsƒinƒR&D                   Approaches                  PotentialƒBenefits
 Requirements      Nonexistent, inconsistent,      Orange Book/Common            Canonical, composable,      Relevant developments;
                   incomplete nonscalable          Criteria have inherent        scalable trustworthiness    Simplified procurement
                   requirements                    limitations                   requirements                process
 System            Inflexibility; Constraints of   Evolvable architectures,      Scalably composable         Long-term scalable
 architectures     flawed legacy systems           scalable theory of            components and              evolvability maintaining
                                                   composability are needed      trustworthy architectures   trustworthy operation

 Development       Unprincipled systems,           Principles not                Built-in assured            Fewer flaws and risks;
 methodologies     unsafe languages, sloppy        experientially                scalably composable         Simplified evaluations
 and software      programming practices           demonstrated; Good            trustworthiness
 engineering                                       programming language
                                                   theory widely ignored

 Analytic tools    Ad-hoc, piecemeal tools with    Tools need sounder bases      Rigorously based            Eliminating many flaws
                   limited usefulness                                            composable tools

 Whole-system      Impossible today for large      Top-to-bottom, end-to-        Formal methods,             Scalable incremental
 evaluations       systems                         end analyses needed           hierarchical staged         evaluations
                                                                                 reasoning
 Operational       Enormous burdens on             User and administrator        Dynamic self-diagnosis      Simplified, scalable
 practices         administrators                  usability are often ignored   and self-healing            operational management




What are the challenges that             of high assurance information technol-              could compromise the trustworthi-
must be addressed?                       ogy. Time-consuming evaluations of                  ness of the entire system. Designing
                                         trustworthy systems today create long               complex secure systems from the ground
The absence of sound systemwide delays when compared with conven-                            up is an exceptionally hard problem,
architectures designed for trustworthi- tional system developments with weaker               particularly since large systems may
ness and the relatively large costs of evaluations. Consequently, development                have catastrophic flaws in their design
full verification and validation (V&V) of trustworthy systems can be expected                and implementation that are not dis-
have kept any secure computing base to take longer than is typically planned                 covered until late in development, or
from economically providing the req- for COTS systems. In addition, the                      even after deployment. Catastrophic
uisite assurance and functionality. (The performance of trustworthy systems                  software flaws may occur even in just
sole exception is provided by “high- typically lags the performance of COTS                  a few lines of mission-critical code,
consequence” government applications, systems with comparable functions.                     and are almost inevitable in the tens
in which cost is a secondary concern                                                         of millions of lines of code in today’s
to national security.) This situation is One of the most pressing challenges                 systems. Given the relatively minuscule
exacerbated by the scale and complexity involves designing system architectures              size of programs and systems that have
often needed to provide required func- that minimize how much of the system                  been extensively verified and the huge
tionality. In addition, the length of must be trustworthy—i.e., minimiz-                     size of modern systems and applica-
the evaluation process can exceed the ing the size and extent of the trusted                 tions, scaling up formal approaches to
time available for patches and system computing base (TCB). In contrast, for                 production and verification of bug-free
upgrades and retarded the incorporation a poorly designed system, any failure                systems seems like a Herculean task. Yet,

                                                                              SCALABLE TRUSTWORTHY SYSTEMS                              7
                                 TABLE 1.2: Scalable Trustworthy Systems Overview

 Vision:ƒMake the development of trustworthy systems of systems (TSoS) practical; ensure that even very large and complex systems
 can be built with predictable scalability and demonstrable trustworthiness, using well-understood composable architectures and well-
 designed, soundly developed, assuredly trustworthy components.

 Challenges: Most of today’s systems are built out of untrustworthy legacy systems using inadequate architectures, development
 practices, and tools. We lack appropriate theory, metrics of trustworthiness and scalability, sound composable architectures, synthesis and
 analysis tools, and trustworthy building blocks.

 Goals: Sound foundations and supporting tools that can relate mechanisms to policies, attacks to mechanisms, and systems to
 requirements, enabling facile development of composable TSoS systematically enhancing trustworthiness (i.e., making them more
 trustworthy than their weakest components); documented TSoS developments, from specifications to prototypes to deployed systems.


 MILESTONES

 IncrementalƒSystems                         Clean-SlateƒSystems                         HybridƒSystems
 Near-termƒmilestones:                       Near-termƒmilestones:                       Near-termƒmilestones:
 Sound analytic tools                        Alternative architectures                   Mix-and-match systems
 Secure bootloading                          Well-specified requirements                 Integration tools
 Trusted platforms                           Sound kernels/VMMs                          Evaluation strategies

 Medium-termƒmilestones:                     Medium-termƒmilestones:                     Medium-termƒmilestones:
 Systematic use of tools                     Provably sound prototypes                   Use in infrastructures
 More tool development                       Proven architectures                        Integration experiments

 Long-termƒmilestones:                       Long-termƒmilestones:                       Long-termƒmilestones:
 Extensively evaluated systems               Top-to-bottom formal evaluations            Seamless integration of COTS/open-source
                                                                                         components
 Test/evaluation: Identify measures of trustworthiness, composability, and scalability, and apply them to real systems.

 Techƒtransfer: Publish composition methodologies for developing TSoS with mix-and-match components. Release open-source tools
 for creating, configuring, and maintaining TSoS. Release open-source composable, trustworthy components. Publish successful, well-
 documented TSoS developments. Develop profitable business models for public-private TSoS development partnerships for critical
 applications, and pursue them in selected areas.




formally inspired approaches may be             components is almost certainly an even          of the executable code has not been
more promising than any of the less             harder problem.                                 compromised and (b) that the code
formal approaches attempted to date.                                                            resides in memory in a manner that it
In addition, considerable progress is          As one example, securing the bootload            can be neither read nor altered, but only
being made in analyzing system behav-          process would be very valuable, but the          executed. Firmware residing in ROM,
ior across multiple layers of abstraction.     underlying general principle is that every       when ROM updating is cryptographi-
On the other hand, designing complex           module of executable software within             cally protected for integrity, meets these
trustworthy systems and “compromise-           a system should be backed by a chain             criteria. Software that is cryptographi-
resilient” systems on top of insecure          of trust, assuring (a) that the integrity        cally protected for integrity, validated




 8     SCALABLE TRUSTWORTHY SYSTEMS
when loaded, and protected by hardware       there are no accepted methodologies for      languages; and corresponding analysis
so it can only be executed also meets        design, implementation, operation, and       techniques. System design and analysis,
these criteria.                              evaluation that adequately characterize      of course, must also anticipate desired
                                             the trade-offs among trustworthiness,        operational practice and human usabil-
One of the most relevant challenges for      functionality, cost, and so on.              ity. It must also encompass the entire
this topic area is how to achieve highly                                                  system life cycle and consider both
principled system development pro-           What approaches might be                     environmental adversaries and other
cesses based on detailed and farsighted      desirable?                                   adverse influences.
requirements and sound architectures         Currently, searching for flaws in micro-
that can be composed out of demon-           processor design makes effective use         Recent years have seen considerable
strably trustworthy components and           of formal verification tools to evaluate     progress in model checking and theorem
subsystems, and subjected to rigor-          a chip’s logic design, in addition to        proving. In particular, significant prog-
ous software, hardware, and system           other forms of testing and simulation.       ress has been made in the past decade
engineering disciplines for its imple-       This technology is now becoming very         on static and dynamic analysis of source
mentation. The tools currently being         cost-effective. However, it is not likely    code. This progress needs to be extended,
used do not even ensure that a com-          to scale up by itself to the evaluation      with particular emphasis on realistic
posed system is at least as trustworthy      of entire hardware/software systems,         scalability that would be applicable to
as its components.                           including their applications. Also, it       large-scale systems and their applications.
                                             is unclear whether existing hardware
Measuring confidentiality and integrity      verification tools are robust against        Verification of a poorly built system after
flaws in trustworthy system construc-        nation-state types of adversaries. Formal    the fact has never been accomplished,
tion requires the ability to identify and    verification and other analytic tools that   and is never likely to work. However,
measure the channels through which           can scale will be critical to building       because we cannot afford to scrap our
information can leak out of a system.        systems with significantly higher assur-     existing systems, we must seek an evo-
Covert channels have been well studied       ance than today’s systems. Better tools      lutionary strategy that composes new
in the constrained, older, local sense       are needed for incorporating assurance       systems out of combinations of old and
of the term. In an increasingly con-         in the development process and for auto-     new subsystems, while minimizing the
nected world of cross-domain traffic,        mating formal verification. These tools      risks from the old systems. A first step
distributed covert channels become           may provide the functionality to build       might involve a more formal under-
increasingly available. For more distrib-    a secure computing base to meet many         standing of the security limitations
uted forms of covert channels or other       of users’ needs for assurance and func-      and deficiencies of important exist-
out-of-band signaling channels, we lack      tionality. They should be available for      ing components, which would at least
the science, mathematics, fundamental        pervasive use in military systems, as well   allow us to know the risks being taken
theory, tools for risk assessment, and the   as to commercial providers of process        by using such components in trustwor-
ability to seal off such adverse channels.   control systems, real-time operating         thy composable systems. The ultimate
                                             systems, and application environments.       goal is to replace old systems gradually
Legacy constraints on COTS soft-             Tools that can scale up to entire systems    and piecewise over time, to increase
ware, lack of networking support, and        (such as national-scale infrastructures)     trustworthiness for progressively more
serious interoperability constraints have    will require rethinking how we design,       complex systems.
retarded progress. Meaningful security       build, analyze, operate, and maintain
has not been seen as a competitive           systems; addressing requirements;            Verification is expensive. Most COTS
advantage in the mainstream. Even if         system architectures; software engi-         systems are built around functional-
trustworthiness were seen in that light,     neering; programming and specification       ity rather than trustworthiness, and




                                                                           SCALABLE TRUSTWORTHY SYSTEMS                           9
are optimized on cost of development       forever remain stuck in the intractable   composability of function enables scal-
and time to deployment—generally to        position of starting from scratch each    ability of system development today).
the detriment of trustworthiness and       time. This foundation must include        Fundamental research in writing security
often resulting in undetected vulner-      verified and validated hardware, soft-    specifications that are precise enough to
abilities. An alternative approach is to   ware, compilers, and libraries with       be verified, strict enough to be trusted,
start from a specification and check the   easily composable models that include     and flexible enough to be implemented
soundness of the system as it is being     responses to environmental stimuli,       will be crucial to major advances in
built. The success of such an approach     misconfigurations and other human         this area.
would depend on new languages, envi-       errors, and adversarial influences, as
ronments that enable piecewise formal      well as means of verifying composi-       Resources
verification, and more scalable proof-     tions of those components.
generation technology that requires                                                  As noted above, this topic is absolutely
less user input for proof-carrying code.   What R&D is evolutionary and              fundamental to the other topics. The
A computer automated secure software       what is more basic, higher                costs of not being able to develop scal-
engineering environment could greatly                                                able trustworthy systems have already
                                           risk, game changing?
facilitate the construction of secure                                                proven to be enormous and will con-
systems. Better yet, it should encompass   Evolutionary R&D might include incre-     tinue to escalate. Unfortunately, the
hardware and total system trustworthi-     mental improvements of large-scale        costs of developing high-assurance
ness as well.                              systems for certain critical national     systems in the past have been consider-
                                           infrastructures and specific applica-     able. Thus, we must reduce those costs
Another critical element is the creation tion domains, such as DNS and               without compromising the effective-
of comprehensible models of logic and DNSSEC, routing and securing the               ness of the development and evaluation
behavior, with comprehensible inter- Border Gateway Protocol (BGP), vir-             processes and the trustworthiness of
faces so that developers can maintain tualization and hypervisors, network           the resulting systems. Although it is
an understanding of systems even as file systems and other dedicated servers,        difficult to assess the costs of develop-
they increase in size and scale. Such exploitation of multicore architectures,       ing trustworthy systems in the absence
models and interfaces should help and web environments (e.g., browsers,              of soundly conceived building blocks,
developers avoid situations where cata- web servers, and application servers         we are concerned here with the costs
strophic bugs lurk in the complexity such as WebSphere and WebLogic).                of the research and prototype devel-
of incomprehensible systems or in the However, approaches such as harden-            opments that would demonstrate the
complexity of the interactions among ing particularly vulnerable components          efficacy and scalability of the desired
systems. Creation of a language for or starkly subsetting functionality are          approaches. This may seem to be a
effectively specifying a policy involving inherently limited, and belief in their    rather open-ended challenge. However,
many components is a hard problem. effectiveness is full of risks. Goals of          incisive approaches that can increase
Problems that emerge from interac- this line of R&D include identifying              composability, scalability, and trust-
tions between components underscore needs, principles, methodologies, tools,         worthiness are urgently needed, and
the need for verifying behavior not and reusable building blocks for scalable        even relatively small steps forward can
only in the lab, but in the field as well. trustworthy systems development.          have significant benefits.

Finally, efficiently creating provably     More basic, higher-risk, game-changing    To this end, many resources will be
trustworthy systems will require           R&D broadly includes various topics       essential. The most precious resource is
creation of secure but flexible com-       under the umbrella of composability,      undoubtedly the diverse collection of
ponents, and theories and tools for        because it is believed that only effec-   people who could contribute. Also vital
combining them. Without a secure           tive composability for trustworthiness    are suitable languages for requirements,
computing foundation, developers will      can achieve true scalability (just as     specification, programming, and so on,

10    SCALABLE TRUSTWORTHY SYSTEMS
along with suitable development tools.          computer automated secure software           could proceed for any systems in the
In particular, theories are needed to           engineering environment (including its       context of the exemplars noted above,
support analytic tools that can facili-         generalization to hardware and systems)      initially with respect to prototypes and
tate the prediction of trustworthiness,         should be measured in the reduction of       potentially scaling upward to enterprises.
inclusion modeling, simulation, and             person-hours required to construct and
formal methods.                                 verify systems of comparable assurance       To what extent can we test
                                                levels and security. The reuse and size
                                                                                             real systems?
Measures of success                             of components being reused should be
                                                measured, since the most commonly         In general, it may be more cost-effective
Overall, the most important measure             used components in mission-critical       to carry out R&D on components, com-
of success would be the demonstrable            systems should be verified components.    posability, and scalability in trustworthy
avoidance of the characteristic system          Evaluation methodologies need to be       environments at the subsystem level
failures that have been so common in            developed to systematically exploit the   than in general system environments.
the past (e.g., see [Neu1995]), just a few      metrics. The measures of success for scal-However, composition still requires test
of which are noted earlier in this section.     able trustworthy systems also themselves  and evaluation of the entire system. In
                                                need to be composable into enterprise-    that it is clearly undesirable to experi-
Properties that are important to the            level measures of success, along with the ment with critical systems such as
designers of systems should be measured         measures contained in the sections on     power grids, although owners of these
in terms of the scale of systems that can       the other topic areas that follow.        systems have realistic but limited-scale
be shown to have achieved a specified                                                     test environments. There is consider-
level of trustworthiness. As noted at the       What needs to be in place for able need for better analytic tools and
beginning of this section, trustworthi-                                                   testbeds that closely represent reality.
                                                test and evaluation?
ness typically encompasses requirements                                                   Furthermore, if applicable principles,
for security, reliability, survivability, and   Significant improvements are necessary techniques, and system architectures
many other system properties. Each              in system architectures, development can be demonstrated for less critical
system will need to have its own set            methodologies, evaluation methodolo- systems, successful system developments
of metrics for evaluation of trustwor-          gies, composable subsystems, scalability, would give insights and inspiration that
thiness, composability, and scalability.        and carefully documented, successful would be applicable to the more critical
Those metrics should mirror generic             worked examples of scalable prototypes. systems without having to test them
requirements, as well as any require-           Production of a reasonable number of initially in more difficult environments.
ments that are specific to the intended         examples will typically require that will
applications. The effectiveness of any          not all succeed. Test and evaluation


References
[Can2001]          Ran Canetti. Universally composable security: A new paradigm for cryptographic protocols
                   (http://eprint.iacr.org/2000/067), 2005. An extended version of the paper from the 42nd
                   Symposium on Foundations of Computer Science (FOCS’01) began a series of papers
                   applying the notion of universal composability to cryptography. Much can be learned
                   from this work regarding the more general problems of system composability.

[Neu1995]          Peter G. Neumann. Computer-Related Risks, Addison-Wesley/ACM Press, New York, 1995. See also an
                   annotated index to online sources for the incidents noted here, as well as many more recent cases
                   (http://www.csl.sri.com/neumann/illustrative.html).


                                                                              SCALABLE TRUSTWORTHY SYSTEMS                         11
[Neu2004]    Peter G. Neumann. Principled assuredly trustworthy composable architectures. DARPA-CHATS Final
             Report, SRI International, Menlo Park, California, December 2004
             (http://www.csl.sri.com/neumann/chats4.html). This report characterizes many of the
             obstacles that must be overcome in achieving composability with predictable results.

[Sal+2009]   J.H. Saltzer and F. Kaashoek. Principles of computer design. Morgan Kauffman, 2009. (Chapters
             1-6; Chapters 7-11 are online at: http://ocw.mit.edu/ans7870/resources/system/index.htm).




12   SCALABLE TRUSTWORTHY SYSTEMS
Current Hard Problems in INFOSEC Research
             2. Enterprise-Level Metrics (ELMs)

             BACKGROUND

             What is the problem being addressed?
             Defining effective metrics for information security (and for trustworthiness more
             generally) has proven very difficult, even though there is general agreement that such
             metrics could allow measurement of progress in security measures and at least rough
             comparisons between systems for security. Metrics underlie and quantify progress
             in all other roadmap topic areas. We cannot manage what we cannot measure, as
             the saying goes. However, general community agreement on meaningful metrics
             has been hard to achieve, partly because of the rapid evolution of information
             technology (IT), as well as the shifting locus of adversarial action.

             Along with the systems- and component-level metrics that are discussed elsewhere
             in this document and the technology-specific metrics that are continuing to emerge
             with new technologies year after year, it is essential to have a macro-level view of
             security within an organization. A successful research program in metrics should
             define a security-relevant science of measurement. The goals should be to develop
             metrics to allow us to answer questions such as the following:

                ƒƒ How secure is my organization?
                ƒƒ Has our security posture improved over the last year?
                ƒƒ To what degree has security improved in response to changing threats and
                  technology?
                ƒƒ How do we compare with our peers with respect to security?
                ƒƒ How secure is this product or software that we are purchasing or deploying?
                ƒƒ How does that product or software fit into the existing systems and
                  networks?
                ƒƒ What is the marginal change in our security (for better or for worse), given
                  the use of a new tool or practice?
                ƒƒ How should we invest our resources to maximize security and minimize
                  risks?
                ƒƒ What combination of requirement specification, up-front architecture,
                  formal modeling, detailed analysis, tool building, code reviews, programmer
                  training, and so on, would be most effective for a given situation?
                ƒƒ How much security is enough, given the current and projected threats?
                ƒƒ How robust are our systems against cyber threats, misconfiguration,
                  environmental effects, and other problems? This question is especially
                  important for critical infrastructures, national security, and many other
                  large-scale computer-related applications.


                                                                                                13
Enterprise-level metrics (ELMs) address       environment. Note that this definition        quantifiable, feasible to measure, and
the security posture of an organization       incorporates a specification of system        repeatable. They provide relevant trends
and complement the component-level            objectives and a specification of the         over time and are useful in tracking
metrics examined elsewhere in the             system environment, which would               performance and directing resources
roadmap topics. “Enterprise” is a term        include some notion of a threat model.        to initiate performance improvement
that encompasses a wide range. It could       Although this type of probability metric      actions.” [http://www.itl.nist.gov/lab/
in principle apply to the Internet as a       has been computed for system reliability      bulletns/bltnaug03.htm]
whole, but realistically it is intended       and for certain system risk assessments,
here to scale in scope from a large cor-      the potential accuracy of such assess-        Most organizations view the answers to
poration or department of the federal         ments with respect to security seems          the questions listed above in the short
government down to the small office/          to be extremely questionable, given the       term from a financial mind-set and
home office (SOHO). For our purposes,         rapidly changing threat environment for       attempt to make cost-benefit trade-
an enterprise has a centralized decision      IT systems. For example, a presumed           off analyses. However, in the absence
making authority to ensure the use of         high probability of meeting security          of good metrics, it is unclear whether
ELMs to rationally select among alterna-      objectives essentially goes to zero at the    those analyses are addressing the right
tives to improve the security posture of      instant security exploits are announced       problems. Decisions resulting from
that enterprise. ELMs can support deci-       and immediately perpetrated.                  such analyses will frequently be detri-
sions such as whether adoption of one                                                       mental to making significant security
technology or another might improve           Security metrics are difficult to develop     improvements in the long term and
enterprise security. ELMs also provide        because they typically try to measure         thus eventually require costly new
the basis for accurate situational aware-     the absence of something negative (e.g.,      developments.
ness of the enterprise’s security posture.    lack of any unknown vulnerabilities in
                                              systems and lack of adversary capabilities    What are the potential
In this discussion, we define metrics rel-    to exploit both known and unknown
                                                                                            threats?
evant to systems and networking within        vulnerabilities). This task is difficult
an enterprise, and consider composing         because there are always unknowns in          Lack of effective ELMs leaves one in the
host-level and other lower-layer mea-         the system and the landscape is dynamic       dark about cyberthreats in general. With
surements up to an enterprise level. In       and adversarial. We need better defini-       respect to enterprises as a whole, cyber-
other words, the goals of ELMs are to         tions of the environment and attacker         security has been without meaningful
understand the security of a large-scale      models to guide risk-based determi-           measurements and metrics throughout
system—enabling us to understand              nation. These are difficult areas, but        the history of information technol-
enterprise security as a whole, with a        progress is achievable.                       ogy. (Some success has been achieved
goal of using these measurements to                                                         with specific attributes at the compo-
guide rational investments in security.       The following definition from NIST            nent level.) This lack seriously impedes
If these ELM goals are met, then exten-       may provide useful insights.                  the ability to make enterprise-wide
sions to other related cases, such as                                                       informed decisions of how to effectively
Internet service providers (ISPs) and         “IT security metrics provide a practical      avoid or control innumerable known
their customers, should be feasible.          approach to measuring information             and unknown threats and risks at every
                                              security. Evaluating security at the system   stage of development and operation.
Security itself is typically poorly defined   level, IT security metrics are tools that
in real systems, or is merely implicit.       facilitate decision making and account-       Who are the potential
One view might be to define it as the         ability through collection, analysis, and
                                                                                            beneficiaries? What are their
probability that a system under attack        reporting of relevant performance data.
                                                                                            respective needs?
will meet its specified objectives for a      Based on IT security performance goals
specified period of time in a specified       and objectives, IT security metrics are       In short, everyone who is affected by an

14     ENTERPRISE-LEVEL METRICS
automated IT system has the potential        caused by cyber attacks, which might           short-term economic losses caused by
to benefit from better security metrics,     be enhanced with the existence of mean-        system outages. Potential beneficiaries,
especially at the enterprise level. Spon-    ingful metrics. However, that market           challenges, and needs are summarized
sors of security R&D require such            is perhaps undercut not by the lack            in Table 2.1.
metrics to measure progress. With such       of suitable metrics, but more by the
metrics, decision makers, acquisition        prevalence of insecure systems and their       What is the current state of
managers and investors in security tech-     exploitations and by a historical lack of
                                                                                            the practice?
nology could make a better business case     consistent actuarial data.
for such technology, and guide intel-                                                       At present, the practice of measuring
ligent investment in such technology.        Metrics defined relative to a mission          security is very ad hoc. Many of the
This demand of course would guide            threat model are necessary to understand       processes for measurement and metric
the market for development of mea-           the components of risk, to make risk           selection are mostly or completely sub-
surably more secure systems. Metrics         calculations, and to improve decision          jective or procedural, as in evaluation
can be applied not just to technol-          making in response to perceived risk.          of compliance with Sarbanes-Oxley,
ogy, but to practices as well, and can       A risk model must incorporate threat           HIPAA, and so on. New approaches
provide management with an incentive         information, the value of the enterprise       are introduced continually as the old
structure oriented toward security per-      information being protected, poten-            approaches prove to be ineffective. There
formance improvement. Robust metrics         tial consequences of system failure,           are measurements such as size and scope
would enhance the certification and          operational practices, and technology.         of botnets, number of infections in a
accreditation process, moving toward         More specifically, risk assessment needs a     set of networks, number of break-ins,
quantitative rather than qualitative pro-    threat model (encompassing intent and          antivirus detection rates over time, and
cesses. Metrics also can be used to assess   capabilities), a model of actual protective    numbers of warrants served, crimi-
the relative security implications of        measures, a model of the probability that      nal convictions obtained, and national
alternative security measures, practices,    the adversary will defeat those protective     security letters issued (enforcement).
or policies.                                 measures, and identification of the con-       These are not related to fundamental
                                             sequences of concern or adversary goals.       characteristics of systems, but are more
Administrators require metrics to guide      These consequences of concern are typi-        about what can be measured about
the development of optimal network           cally specific to each enterprise, although    adversaries. Examples include websites
configurations that explicitly consider      many commonalities exist. For critical         that attempt to categorize the current
security, usability, cost, and perfor-       infrastructures, loss of system availability   state of the Internet’s health, the current
mance. There seems to be a potential         may be the key concern. For commercial         state of virus infections world wide, or
market in insurance and underwriting         enterprises, loss of proprietary infor-        the number and sizes of botnets cur-
for predicting and reducing damages          mation may be a greater concern than           rently active.


                                TABLE 2.1: Beneficiaries, Challenges, and Needs

 Beneficiaries                                Challenges                                      Needs
 Developers                                   Establishing meaningful ELMs                    Specification languages, analysis tools
                                              (comprehensive, feasibly implementable,         for feasibility, hierarchical evaluation,
                                              realistic)                                      and incremental change
 System procurers                             Insisting on the use of meaningful ELMs         Certified evaluations
 User communities                             Having access to the evaluations of             Detailed evaluations spanning all
                                              meaningful ELMs                                 relevant aspects of trustworthiness




                                                                                        ENTERPRISE-LEVEL METRICS                          15
Numerous initiatives and projects are               on some sort of thermometer).          ƒƒ Measures of effectiveness. The
being undertaken to improve or develop              However, password strength is a          Institute for Defense Analyses
metrics for all or a specific portion of the        rather vacuous concept in systems        (IDA) developed a methodology
security domain. Included in these are              with inherently weak security in         for determining the effectiveness
the following:                                      other areas.                             of cybersecurity controls based on
                                                 ƒƒ Security implementation                  its well-used and -documented
     ƒƒ Several government documents                                                         methodology for determining the
       and efforts (for example, NIST               metrics, which might be used
                                                    to assess how many systems in            effectiveness of physical security
       SP800-55) that describe an                                                            controls. Using a modified
       approach to defining and                     an enterprise install a newly
                                                    announced patch, and how                 Delphi technique, the measures
       implementing IT security                                                              of effectiveness of various
       metrics. Although some of the                quickly.
                                                                                             components and configurations
       measures and metrics are useful,          ƒƒ Initiatives in security processes,       were determined, which then
       they are not sufficient to answer            which might define metrics               allowed for a security “ranking”
       the security questions identified            relating to the adoption of those        of the potential effectiveness
       earlier in this section.                     processes and require extensive          of various architectures and
                                                    documentation. However,                  operating modes against different
     ƒƒ Methods that assess security
                                                    such approaches typically are            classes of adversaries [IDA2006].
       based on system complexity
                                                    about process and not actual
       (code complexity, number                                                            ƒƒ Ideal-based metrics. The Idaho
                                                    performance improvement with
       of entry points, etc.). These                                                         National Laboratory (INL) took
                                                    respect to security.
       may give some indication of                                                           a vastly different approach to
       vulnerability, but in the absence       This section focuses on metrics for           developing metrics. It chose to
       of data on attack success rates or      cybersecurity issues. However, it is also     specify several best-case outcomes
       the efficacy of mitigation efforts,     useful to consider existing metrics and
                                                                                             of security and then attempt to
       these methods prove very little.        design techniques for physical security
                                                                                             develop real-world measures of
                                               systems, and the known limitations
     ƒƒ Red Teaming, which provides                                                          those “ideals.” The resulting set of
                                               of those techniques. This informa-
       some measure of adversary work                                                        10 system measurements covering
                                               tion would help advance cybersecurity
       factor and is currently done                                                          7 ideals is being tested in the
                                               research. It will also be required as
       in security assessments and                                                           field to determine how well they
                                               our logical and physical cybersecurity
       penetration testing. One can                                                          can predict actual network or
                                               systems become ever more intertwined
       apply penetration testing, using                                                      system security performance
                                               and interdependent. Similarly, tech-
       a variety of available tools and/                                                     [McQ2008].
                                               niques for financial risk management
       or hiring a number of firms that                                                    ƒƒ Goal-oriented metrics. Used
                                               may also be applicable to cybersecurity.
       provide this as a service. For                                                        primarily in the software
       example, this can provide metrics                                                     development domain, the
       on adversary work factor and            What is the status of current                 goal-oriented paradigm seeks to
       residual vulnerabilities before and     research?                                     establish explicit measurement
       after implementation of a security                                                    goals, define sets of questions
                                               There are initiatives aimed at developing
       plan.                                                                                 that relate to achieving the goals,
                                               new paradigms for identifying measures
     ƒƒ Heuristic approaches to provide        and metrics. Some of them attempt to          and identify metrics that help to
       metrics in a number of security-        apply tools and techniques from other         answer those questions.
       related areas. For example,             disciplines; others attempt to approach     ƒƒ Quality of Protection (QoP).
       systems often report a measure          the problem from new directions. These        This is a recent approach that
       of “password strength” (usually         initiatives include the following:            is in early stages of maturity. It

16
        ENTERPRISE-LEVEL METRICS
     has been the subject of several        answering questions such as the degree       Analysis
     workshops but is still relatively      to which one system is more secure than      Analysis focuses on determining how
     qualitative [QoP2008].                 another or the degree to which adop-         effectively the metrics describe and
  ƒƒ Adversary-based metrics. MIT           tion of security technology or practice      predict the performance of the system.
     Lincoln Laboratory chose to            makes a system more secure. However,         The prediction should include both
     explore the feasibility and effort     as noted above, these measurements are       current and postulated adversary
     required for an attacker to break      relative to assumed models for adver-        capabilities. There has been relatively
     into network components, by            sary capabilities and goals, and to our      little work on enterprise-level analy-
     examining reachability of those        knowledge of our systems’ vulnera-           ses, because a foundation of credible
     components and vulnerabilities         bilities—and therefore are potentially       metrics and foundational approaches
     present or hypothesized to be          limited by shortcomings in the models,       for deriving enterprise-level evaluations
     present. It and others have built      requirements, knowledge, assumptions,        from more local evaluations have been
     tools employing attack graphs to       and other factors.                           lacking.
     model the security of networks.                                                     Composition
                                            While this section is focused on enter-
                                            prise-level metrics (ELMs), we must          Since security properties are often best
FUTURE DIRECTIONS                           also consider definitions of metrics for     viewed as total-system or enterprise-level
                                            interconnected infrastructure systems,       emergent properties, research is required
                                            as well as for non-enterprise devices.       in the composability of lower-level
On what categories can we
                                            We must also anticipate the nature of        metrics (for components and subsys-
subdivide this topic?
                                            the enterprise of the future; for example,   tems) to derive higher-level metrics
For the purposes of this section, we        technology trends imply that we should       for the entire system. This “compos-
divide the topic of enterprise-level        consider smart phones as part of the         able metrics” issue is a key concern for
metrics into five categories: definition,   enterprise. Infrastructure systems may       developing scalable trustworthy systems.
collection, analysis, composition, and      be thought of as a particular class of       In addition, the composability of enter-
adoption.                                   enterprise-level systems. However, the       prise-level metrics into meta-enterprise
                                            interrelationships among the differ-         metrics and the composability of the
Definition                                  ent infrastructures also suggest that        resulting evaluations present challenges
Definition identifies and develops the      we must eventually be able to consider       for the long-term future.
models and measures to create a set         meta-enterprises.
of security primitives (e.g., for confi-                                                 Adoption
dentiality, integrity, availability, and    Collection                                   Adoption refers to those activities that
others). NIST SP 800-55 provides a          Collection requirements may inspire          transform ELM results into a useful
useful framework for metrics definition.    new research in hardware and software        form (such as a measurement paradigm
This publication proposes development       for systems that enable the collection of    or methodology) that can be broadly
of metrics along the dimensions of          data through meaningful metrics, ideally     used—taking systems, processes, organi-
implementation (of a security policy),      in ways that cannot be compromised by        zational constraints, and human factors
effectiveness/efficiency, and mission       adversaries. This includes conditioning      into account. Monetary and financial
impact.                                     the data via normalization, categoriza-      considerations may suggest adoption of
                                            tion, prioritization, and valuation. It      metrics such as the number of records
Ideally, metrics would be defined to        might also include system developments       in a customer database and a cost per
quantify security, but such definitions     with built-in auditability and embedded      record if those records are disclosed.
have been difficult to achieve in prac-     forensics support, as well as other topic    We may also consider financial metrics
tice. At the basic level, we would like     areas, such as malware defense and situ-     retrospectively (the cost of a particular
to quantify the security of systems,        ational understanding.                       compromise, in terms of direct loss,

                                                                                    ENTERPRISE-LEVEL METRICS                    17
reputation, remediation costs, etc.). This      ƒƒ Composition models of metrics          as system survivability under threats
retrospection would be useful for system           to determine enterprise values         that are not addressed, human safety,
designers and for the insurance under-             from subsystem metrics                 and so on.
writing concept mentioned previously.           ƒƒ Scalability of sets of metrics
                                                                                          Adapting approaches to metrics from
                                                ƒƒ Developing or identifying metric
What are the major research                                                               other disciplines is appropriate, but the
                                                   hierarchies                            result is not complete and often not
gaps?
                                                ƒƒ Measures and metrics for security      sufficiently applicable (as in the case of
In spite of considerable efforts in                primitives                             probability metrics for component and
the past, we do not have any univer-            ƒƒ Appropriate uses of metrics            system reliability). We should consider
sally agreed-upon methodologies to                 (operations, evaluation, risk          connections with other fields, while
address the fundamental question of                management, decision making)           remaining aware that their techniques
how to quantify system security. At a                                                     may not be directly applicable to cyber-
                                                ƒƒ Ability to measure operational
minimum, an evaluation methodol-                                                          security because of intelligent adversaries
                                                   security values
ogy would support hypothesis testing,                                                     and the fluid nature of the attack space.
benchmarking, and adversary models.             ƒƒ Measuring human-system
Hypothesis testing of various degrees of           interaction (HSI)                      Many disciplines (such as financial
formality, from simple engagements to           ƒƒ Tools to enhance and automate          metrics and risk management practices;
formal, well-instrumented experiments,             the above areas in large               balanced scorecard, six-sigma, and insur-
is needed to determine the viability of            enterprises                            ance models; complexity theory; and
proposed security measures. Bench-                                                        data mining) operate in environments of
marking is needed to establish a system                                                   decision making under uncertainty, but
effectiveness baseline, which permits the
                                              What R&D is evolutionary,                   most have proven methods to determine
progress of the system to be tracked as       and what is more basic,                     risk. For example, the field of finance
changes are made and the threat envi-         higher risk, game changing?                 has various metrics that help decision
ronment evolves. Finally, evaluation          Composability advances (for multi-          makers understand what is transpiring
must include well-developed adver-            ple metrics) could be game-changing         in their organizations. Such metrics
sary models that predict how a specific       advances. Hierarchical composition          can provide insight into liquidity, asset
adversary might act in a given context        of metrics should support frameworks        management, debt management, prof-
as systems react to that adversary’s intru-   such as argument trees and security cases   itability, and market value of a firm.
sions or other exploits.                      (analogous to safety cases in complex       Capital budgeting tools determining
                                              mechanical systems, such as aircraft).      net present-values and internal rates of
What are some exemplary                                                                   return allow insights into the returns
                                              Identifying comprehensive metrics, or       that can be expected from investments
problems for R&D on this
                                              a different set of measurement dimen-       in different projects. In addition, there
topic?
                                              sions, might provide a leap forward. The    are decision-making approaches, such
The range of requirements for metrics in      well-known and well-used confidential-      as the Capital Asset Pricing Model
security is broad. R&D may be focused         ity, integrity, availability (CIA) model    and options pricing models, that link
in any of the following areas:                is good for discussing security, but may    risk and return to provide a perspec-
                                              not be easily or directly measured in       tive of the entire financial portfolio
     ƒƒ Choosing appropriate metrics          large enterprises. It is also inherently    under a wide range of potential market
     ƒƒ Methods for validating metrics        incomplete. For example, it ignores         conditions. These methodologies have
                                              requirements relating to accountability,    demonstrated some usefulness and have
     ƒƒ Methods for metric computation        auditing, real-time monitoring, and         been applied across industries to support
       and collection                         other aspects of trustworthiness, such      decision making. A possible analog for

18      ENTERPRISE-LEVEL METRICS
IT security would be sound systems            ƒƒ Economic or market analysis of        metrics and evaluation methodologies
development frameworks that support              adversary actions may provide         for security of the information domain
enterprise-level views of an organiza-           an indirect metric for security       with the metrics and evaluation meth-
tion’s security. Research is needed to           effectiveness. If the cost to         odologies for physical, cognitive, and
identify system design elements that             exploit a vulnerability on a          social domains.
enable meaningful metrics definition             critical and widely used server
and data collection. Research is also            system increases significantly, we
                                                 might surmise that the system         Resources
needed on issues in collection logistics,
such as the cost of collection and its           is becoming more secure over          Industry trends such as exposure to data
impact on the metric being used (e.g.,           time or that the system has           breaches are leading to the development
whether the collection compromises               become more valuable to its           of tools to measure the effectiveness
                                                 adversaries. This approach can be
security).                                                                             of system implementations. Industry
                                                 confounded by, for example, the
                                                                                       mandates and government regulations
                                                 monetary assets accessible to the
Research on metrics related to adversary                                               such as the Federal Information Secu-
                                                 adversary by compromising the
behaviors and capabilities needs to be                                                 rity Management Act (FISMA) and
                                                 service. (A very secure system not
conducted in several key areas, such as                                                Sarbanes-Oxley require the govern-
                                                 widely used in an attractive target
the following:                                   space may discourage a market         ment and private-sector firms to become
                                                 for high-priced vulnerabilities.)     accountable in the area of IT security.
  ƒƒ The extent of an adversary’s                                                      These factors will lead industry and
     opportunity to affect hardware              It is also not obvious that this
                                                 is an enterprise-level metric.        government to seek solutions for the
     and software needs to be studied.                                                 improvement of security metrics.
                                                 Nonetheless, the assembled
     This may lead to research into,
                                                 experts considered market
     for example, global supply-chain                                                  Government investment in R&D is still
                                                 analysis a novel and interesting
     metrics that account for potential          avenue of research.                   required to address the foundational
     adversarial influence during                                                      questions that have been discussed,
     acquisition, update, and remote          ƒƒ Metrics relating to the impact of
                                                                                       such as adversary capabilities and threat
     management cycles.                          cybersecurity recommendations
                                                                                       measurements.
                                                 on public- and private-sector
  ƒƒ Metrics in the broad area of                enterprise-level systems.
     adversary work factor have                                                        Measures of success
     been considered for some time.
     The simple example is the              Metrics can guide root-cause analysis in   The ability to accurately and confi-
     increase in the recommended            the case of security incidents. Research   dently predict the security performance
     length of cryptographic keys           using existing events should compile a     of a component, network, or enter-
     as computational power has             list of metrics that might have avoided    prise is the ultimate measure of success
     increased. This work should            the incident if they had been known        for metrics R&D. Interim milestones
     continue, but there is a question      before the incident.                       include better inputs for risk calculation
     as to the repeatability of the                                                    and security investment decisions. The
                                            A stretch objective in the long term is    extent to which the evaluation of local
     obtained metric.
                                            the development of metrics and data        metrics (e.g., see the other sections)
  ƒƒ Research related to an adversary’s     collection schemes that can provide        can be combined into enterprise-level
     propensity to attempt a                actuarial-quality data with respect to     metrics would be a significant measure
     particular attack, in response         security. This is needed for a robust      of success.
     to a defensive posture adopted         market for insurance against cybersecu-
     by the enterprise, needs to be         rity-related risks. Another long-range
     conducted.                             stretch goal would be to unify the

                                                                                     ENTERPRISE-LEVEL METRICS                 19
What needs to be in place for               assessment of “time to compromise”           To what extent can we test
test and evaluation?                        experimental metrics, possibly consider-     real systems?
                                            ing systems that are identical except for
Testbeds and tools within the testbeds      some security enhancement.                   An enterprise is a testbed of sorts to
are needed to evaluate the descriptive                                                   glean insights on usability, organiza-
and predictive value and effectiveness of   Evaluation and experimentation are           tional behavior, and response to security
proposed measures and models, particu-      essential to measure something that is       practices. Much of the initial collection
larly for potentially destructive events.   relevant to security. Evaluation method-     and verification must be done on real
Repositories of measurement “baselines”     ology goes hand in hand with metrics,        systems to ensure applicability of the
to compare new metric methods and           and tools that accurately measure and        measurements and derived metrics.
models will also be required. Virtualiza-   do not distort quantities of interest also
tion and honeynet environments permit       have direct influence on metrics.


References
[And2008]        R. Anderson. Security Engineering: A Guide to Building Dependable Distributed Systems. Wiley, Indianapolis,
                 Indiana, 2008.

[Avi+2004]       A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr. Basic concepts and taxonomy of dependable and
                 secure computing. IEEE Transactions on Dependable and Secure Computing, 1(1):11-33, January-March 2004.

[Che2006]        E. Chew, A. Clay, J. Hash, N. Bartol, and A. Brown. Guide for Developing Performance Metrics for Information
                 Security. NIST Special Publication 800-80, National Institute of Standards and Technology, Gaithersburg,
                 Maryland, May 2006.

[CRA2003]        Four Grand Challenges in Trustworthy Computing: Second in a Series of Conferences on Grand Research
                 Challenges in Computer Science and Engineering. Computing Research Association, Washington, D.C.,
                 2006 (http://www.cra.org/reports/trustworthy.computing.pdf ).

[Jaq2007]        A. Jaquith. Security Metrics. Addison Wesley Professional, Upper Saddle River, New Jersey, 2007.

[IDA2006]        Institute for Defense Analysis. National Comparative Risk Assessment Pilot Project. Draft Final,
                 September 2006, IDA Document D-3309.

[McQ2008]        M.A. McQueen, W.F. Boyer, S. McBride, M. Farrar, and Z. Tudor. Measurable control system security through
                 ideal driven technical metrics. In Proceedings of the SCADA Scientific Security Symposium, January 2008.

[Met2008]        Metricon 3.0, July 29, 2008, with copious URLs (http://www.securitymetrics.org/content/Wiki.
                 jsp?page=Metricon3.0).

[NIS2009]        Information Security Training Requirements: A Role- and Performance-Based Model. NIST Special
                 Publication 800-16 Revision 1, National Institute of Standards and Technology, Gaithersburg, Maryland,
                 March 20, 2009 (http://csrc.nist.gov/publications/PubsDrafts.html).




20
      ENTERPRISE-LEVEL METRICS
[QoP2008]    4th Workshop on Quality of Protection (Workshop co-located with CCS-2008), October 2008 (http://
             qop-workshop.org/)

[Swa+2003]   M. Swanson, N. Bartol, J. Sabato, J. Hash, and L. Graffo. Security Metrics Guide for Information Technology
             Systems. NIST Special Publication 800-55, National Institute of Standards and Technology, Gaithersburg,
             Maryland, July 2003.




                                                                                                                     21
Current Hard Problems in INFOSEC Research
              3. System Evaluation Life Cycle

              BACKGROUND

              What is the problem being addressed?
              The security field lacks methods to systematically and cost-effectively evaluate its
              products in a timely fashion. Without realistic, precise evaluations, the field cannot
              gauge its progress toward handling security threats, and system procurement is
              seriously impeded. Evaluations that take longer than the existence of a particular
              system version are of minimal use. A suitable life cycle methodology would allow
              us to allocate resources in a more informed manner and enable consistent results
              across multiple developments and applications.

              System evaluation encompasses any testing or evaluation method, including
              testing environments and tools, deployed to evaluate the ability of a system or a
              security “artifact” to satisfy its specified critical requirements. A security artifact
              may be a protocol, device, architecture, or, indeed, an entire system or application
              environment. Its security depends on the security of the environments in which the
              artifact will be deployed (e.g., an enterprise or the Internet), and must be reflected
              throughout the system development life cycle (SDLC). Such a product must meet
              its specification with respect to a security policy that it is supposed to enforce, and
              not be vulnerable to attack or exploitation that causes it to perform incorrectly or
              maliciously. Secondary but also important performance goals can be expressed as “do
              no harm.” The proposed artifact should not inflict collateral damage on legitimate
              actors or traffic in the Internet, and it should not create additional security problems.
              The system evaluation life cycle thus denotes continuous evaluation throughout the
              system life cycle (requirements, design, development and implementation, testing,
              deployment and operations, and decommissioning and disposal). See [NIS2008].

              Security evaluation in the SDLC involves four major areas in addressing
              potential threats:

                 ƒƒ Developing explicit requirements and specifications for systems, including
                   security features, processes, and performance requirements for each
                   development phase in sufficient detail.
                 ƒƒ Understanding whether a product meets its specification with respect to a
                   security policy that it is suppose to enforce. A part of this is understanding
                   how well the product meets the specification and ensures that there
                   are no exploitable flaws. In the case of systems enforcing mandatory
                   confidentiality or integrity policies, this includes demonstration of the
                   limits to adversarial exploitation of covert channels.
                 ƒƒ Understanding whether a product can be successfully attacked or bypassed
                   by testing it in each phase of its development life cycle, either in a testbed
                   or through a mathematical model or simulation.


22
   ƒƒ Developing system evaluation            Such understanding is needed to evalu-         of security products (because they need
     processes whereby incremental            ate the likelihood of human acceptance         reliable means to evaluate what they
     changes can be tracked and               of proposed security artifacts and to          buy); the creators of these products, such
     rapidly reevaluated without              simulate human actions during evalu-           as software and hardware companies;
     having to repeat the entire              ation (e.g., browsing patterns during          and researchers (because they need to
     process.                                 evaluation of a web server defense).           measure their success). Having effective
                                                                                             evaluation methods opens the door to
In each case, independent assessment          What are the potential                         the possibility of standardization of
of a product could reduce reliance on                                                        security and to formation of attestation
                                              threats?
vendor claims that might mask serious                                                        agencies that independently evaluate
problems. On the other hand, embed-           Threats against information and infor-         and rank security products. The poten-
ded self-assurance techniques (such as        mation systems are at the heart of the         tial beneficiaries, challenges, and needs
proof-carrying code) could also be used       need for robust system evaluation. In          are summarized in Table 3.1.
to demonstrate that certain properties        addition to the threats to operational
were satisfied.                               systems, adversaries have the potential to     What is the current state of
                                              affect the security of artifacts at numer-
                                                                                             the practice?
Systematic, realistic, easy-to-use and        ous points within the development life
standardized evaluation methods are           cycle. The complexity of systems, modi-        Evaluation of security artifacts is ad hoc.
needed to objectively quantify perfor-        fications, constant changes to supply          Current methodologies, such as these
mance of any security artifacts and the       chains, remote upgrades and patches,           discussed in NIST SP800-64 (Security
security of environments where these          and other factors give rise to numerous        Considerations in the System Development
artifacts are to be deployed, before and      new threat vectors.                            Life Cycle) [NIS2008] and Microsoft’s
after deployment, as well as the per-                                                        The Security Development Life cycle
formance of proposed solutions. The           Who are the potential                          [How+2006], merely reorder or reem-
evaluation techniques should objectively                                                     phasize many of the tools and methods
                                              beneficiaries? What are their
quantify security posture throughout the                                                     that have been unsuccessful in creating
                                              respective needs?
critical system life cycle. This evaluation                                                  security development paradigms. There
should support research, development,         With regard to the system life cycle,          are neither standards nor metrics for
and operational decisions, and maximize       system architects, engineers, develop-         security evaluation. Product developers
the impact of the investment.                 ers, and evaluators will benefit from          and vendors evaluate their merchandise
                                              enhanced methods of evaluation. Bene-          in-house, before release, via different
Finally, evaluation must occur in a           ficiaries of improved security evaluations     tests that are not disclosed to the public.
realistic environment. The research           range from large and small enterprises         Often, real evaluation takes place in
community lacks data about realis-            to end users of systems. Although              customer environments by product
tic adversarial behavior, including the       beneficiaries’ needs are generally the         vendors collecting periodic statistics
tactics and techniques that adversar-         same—to prevent security incidents and         about threats detected and prevented
ies use to disrupt and deny normal            to respond quickly to those that evade         during live operation. Although this is
operations, as well as normal system          prevention and minimize damage, while          the ultimate measure of success—how a
use patterns and business relation-           protecting privacy—environments that           product performs in the real world—it
ships, to create a realistic environment      they seek to protect may be very different,    does not offer security guarantees to
for evaluation that resembles current         as are their needs for reliability, correct-   customers prior to purchase. There have
environments in which systems are             ness of operation, and confidentiality.        been many incidents when known secu-
deployed. We also lack understanding          Direct beneficiaries of better evaluation      rity devices have failed (e.g., the Witty
of human behavior as users interact with      methods are system developers, system          worm infected security products from a
the system and with security artifacts.       users and administrators; the customers        well-known security product vendor). In

                                                                                 SYSTEM EVALUATION LIFE CYCLE                        23
                                  TABLE 3.1: Beneficiaries, Challenges, and Needs

 Beneficiaries                                 Challenges                                    Needs
 System developers                             Integrate components into systems with        Robust methods to compare components
                                               predictable and dependable security           to be used in new systems. Tools,
                                               properties; effectively track changes from    techniques, and standards of system
                                               one version to another.                       evaluation to enable certification of security
                                                                                             properties of products developed.
 System owners and administrators              Understand the risk to their information      Suites of tools that can be used throughout
                                               operations and assets.                        the operational phases of the system life
                                                                                             cycle to evaluate the current system state
                                               Operate and maintain information systems      and the requirements and impacts of
                                               in a secure manner.                           system upgrades or changes.
 End users                                     Operate confidently in cyberspace.            Recognized and implemented life cycle
                                                                                             system evaluation methods that provide
                                                                                             high confidence in the safety and security of
                                                                                             using online tools and environments.



addition, past efforts such as evaluations                                                          cause exploitable vulnerabilities.
                                               FUTURE DIRECTIONS
of the Trusted System Security Evalua-                                                           ƒƒ Develop realistic traffic,
tion Criteria and the Common Criteria          On what categories can we                            adversary, and environment
[ISO1999] suffer from inadequate incre-        subdivide this topic?                                models that span all four domains
mental methods to rapidly reevaluate                                                                of conflict (physical, information,
new versions of systems.                       We initially discuss this topic relative to          cognitive, and social).
                                               a nominal life cycle model. The SDLC
                                               phases represented in our nominal                 ƒƒ Develop security test cases,
What is the status of current                                                                       procedures, and models to
                                               model are: requirements, design, devel-
research?                                                                                           evaluate the artifact in each life
                                               opment and implementation, testing,
Relatively little research has been done       deployment and operations, and decom-                cycle phase.
on system evaluation methods. The              missioning. System evaluation has to be           ƒƒ More effectively perform speedy
research community still values such           done throughout the entire life cycle,               reevaluations of successive
topics much less than research on novel        with continuous feedback and reevalu-                versions resulting from changes
defenses and attacks. The metrics and          ation against previous stages.                       in requirements, designs,
measures needed to describe security                                                                implementation, and experience
properties during the evaluation life          Potential R&D directions that might                  gained from uses of system
cycle must be developed. (See Section 2).      be pursued at multiple life cycle phases             applications.
The lack of metrics results in security        include the following:
products that cannot be compared and                                                         The following discussion considers the
                                                  ƒƒ Develop cost-effective methods
in solving past problems instead of antic-                                                   individual phases.
ipating and preventing future threats.              to specify security features for
Because the necessary metrics are likely            succeeding life cycle phases.            Requirements
to depend on the nature of the threat             ƒƒ Develop adversarial assessment            ƒƒ Establish a sounder basis for
a security artifact aims to address, it is          techniques that identify and                  how security requirements get
likely that the set of metrics will be large        test for abnormal or unintended               specified, evaluated, and updated
and complex.                                        operating conditions that may                 at each phase in the life cycle.



24
       SYSTEM EVALUATION LIFE CYCLE
  ƒƒ Incorporate relevant (current and         specifications. Concerns about           that are ill-documented, are time-
     anticipated) threats models in            insider threats inside the               dependent, and occur only when
     the requirements phase so that            development process also need to         all of the subsystems have been
     the final specification can be            be addressed.                            integrated.
     evaluated against those threats.       ƒƒ Pursue verification that a system     ƒƒ Conduct Red Team exercises in
  ƒƒ Specify what constitutes secure           is implemented in a way that             a structured way on testbeds to
     operation of systems and                  security claims can be tested.           bring realism. Expand the Red
     environments.                          ƒƒ Consider new programming                 Team concept to include all
  ƒƒ Establish requirement                     languages, constraints on or             phases of the life cycle.
     specification languages that              subsets of existing languages,        ƒƒ Establish evolvable testbeds
     express security properties, so           and hardware design techniques           that are easily upgradeable as
     that automated code analysis              that express security properties,        technology, threat, and adversary
     can be used to extract what the           enforce mandatory access                 models change.
     code means to do and what its             controls, and specify interfaces,
                                                                                     ƒƒ Improve techniques for
     assumptions are.                          so that automated code analysis
                                                                                        combined performance, usability,
Design                                         can be used to extract what the
                                                                                        and security testing. This
  ƒƒ Be able to share data with                code means to do and what its
                                                                                        includes abnormal environments
     adequate privacy, including data          assumptions are.
                                                                                        (e.g., extreme temperatures) and
     on attacks, and with emphasis on     Testing                                       operating conditions (e.g., misuse
     economics of data sharing.             ƒƒ Select and evaluate metrics for          by insiders) that are relevant for
  ƒƒ Develop a richer process to               evaluation of trustworthiness            security testing but may exceed
     develop data used to validate             requirements.                            the system’s intended range of
     security claims.                       ƒƒ Select and use evaluation                operation.
  ƒƒ Develop frameworks for threat             methods that are well suited to     Deployment and Operations
     prediction based on data about            the anticipated ranges of threats     ƒƒ Establish and use evaluation
     current attacks and trends.               and operational environments.            methods that can compare actual
  ƒƒ Develop simulations of (unusual        ƒƒ Develop automated techniques             operational measurements with
     or unanticipated) system states           for identifying all accessible           design specifications to provide
     that are critical for security, as        system interfaces (intentional,          feedback to all life cycle phases.
     opposed to simulation of steady           unintentional, and adversary-         ƒƒ Develop methods to identify
     states.                                   induced) and system                      system, threat, or environment
Development and Implementation                 dependencies. For example,               changes that require reevaluation
  ƒƒ Pursue evaluation methods able            exploitation of a buffer overflow        to validate compliance with
     to verify that an implementation          might be considered a simple             evolving security requirements.
                                               example of an unintended system
     follows requirements precisely                                                  ƒƒ Define and consistently deploy
                                               interface.
     and does not introduce anything                                                    certification and accreditation
     not intended by requirements.          ƒƒ Develop and apply automated              methods that provide
     If specifications exist, this can         tools for testing all system             realistic values regarding the
     be done in two steps: verifying           dependencies under a wide range          trustworthiness of a system with
     consistency of specifications             of conditions. As an example,            respect to its given requirements.
     with requirements and then                some adversaries may exploit        Decommissioning
     consistency of software with              hardware-software interactions        ƒƒ Develop end-of-life evaluation



                                                                           SYSTEM EVALUATION LIFE CYCLE                   25
       methods to verify that security      needs real hosts but can emulate or               methodologies and supporting
       requirements have been achieved      simulate the network interconnections.)           tools that can result in timely
       during the entire life cycle.        Also relevant here is the DETERlab                evaluations and can rapidly
       This includes ensuring that an       testbed (cyber-DEfense Technology                 track the effects of incremental
       adversary can not extract useful     Experimental Research laboratory                  changes.
       information or design knowledge      testbed (http://www.isi.edu/deter). The         ƒƒ Enable creation of attack data
       from a decommissioned or             DETERlab testbed is a general-purpose             repositories under government
       discarded security artifact.         experimental infrastructure for use in
                                                                                              management, similar to the
     ƒƒ Inform threat models from           research (http://www.deterlab.net).
                                                                                              PREDICT repository
       product or system end-of-life                                                          (http://www.predict.org),
       analysis.                            Understanding of which evaluation
                                                                                              for legitimate data. Develop
                                            methods work for which threats is also
                                                                                              approaches to bring realism into
                                            lacking. For example, formal reason-
What are the major research                 ing and model checking may work for
                                                                                              simulations and testbeds.
gaps?                                       software, but simulation may work               ƒƒ Develop research about when
A major gap is lack of the knowledge        better for routing threats. Finally, there        scalability matters, and in what
and understanding of the threat domain      is no peer review mechanism to review             way. Develop research about
that is needed to develop realistic secu-   and validate evaluation mechanisms or             when realism (or simulation)
rity requirements. One reason for this      proposals.                                        matters, and what type of
gap is the lack of widely available data                                                      realism. Develop research about
on legitimate and attack traffic, for                                                         what type of testing works for
various threats and at various levels.      What are some exemplary
                                                                                              which threats and environments.
Another large challenge is the lack of      problems for R&D on this
                                                                                              Develop simple metrics for
reliable methods to measure success of      topic?
                                                                                              system and network health and
various attacks, and inversely to measure   Possible directions to solve current              for attack success.
the success of defensive actions against    problems in security evaluation are:
attacks.                                    (a) system architectures that enhance           ƒƒ Develop detailed metrics for
                                            evaluation throughout the development             system and network health
Yet another challenge lies in not under-    cycle; (b)  development of security               and for attack success. Develop
standing how much realism matters for       metrics and benchmarks for compo-                 benchmarks and standardize
testing and evaluation. For example,        nents, subsystems, and entire enterprises;        testing.
can tests in a 100-node topology            (c) development of tools for easy replica-
with realistic traffic predict behavior     tion of realistic environments in testbeds   What R&D is evolutionary,
in a 10,000-node topology, and for          and simulations; (d) realistic adversary     and what is more basic,
which threats? Some large “hybrid”          models, including how those adversaries      higher risk, game changing?
testbeds may need mixtures of real,         might react to changes in the defensive
emulated, and simulated entities to         security posture; and (e) the encom-         The development over time of system
provide flexible tradeoffs between test     passing methodologies that bring these       evaluation tools, methodologies, mea-
accuracy and testbed cost/scalabil-         components together.                         sures, and metrics will require iterations
ity. If so, then workload estimation                                                     and refinements of the successes of
and workload partitioning tools are         Projects envisioned in this area include     short-term projects, as well as long-term
needed to design experiments for large      the following:                               research. There are short- and long-term
testbeds. (A simple example is that                                                      implications in many of the projects and
a malware research testbed typically           ƒƒ Develop cost-effective                 challenges noted.


26
        SYSTEM EVALUATION LIFE CYCLE
Evolutionary, relatively short-term           success and for security based on     metrics for evaluation, including joint
R&D challenges include the following:         the models of correct operation.      design of realism criteria for evaluation
                                            ƒƒ Developing benchmarks to             environments.
  ƒƒ Defining verifiable parametric
    sets of requirements for                  standardize testing.
                                                                                    Government should help in mandat-
    trustworthiness and improved            ƒƒ Developing understanding about
                                                                                    ing, regulating, and promoting this
    models for assessing                      advantages and limitations of         collaboration, especially with regard
    requirements.                             various evaluation methods            to data sharing. Legal barriers to data
  ƒƒ Devising methods to recreate             (simulation, emulation, pilot         sharing must be addressed. Some
    realism in testbeds and                   deployment, model checking,           industry sectors may be reluctant to
    simulations while providing               etc.) when related to specific        share vulnerability data because of legal
    flexible trade-offs between cost,         threats.                              liability concerns. There may also be
    scalability, and accuracy. (These       ƒƒ Managing risky test                  privacy and customer relations concerns.
    include better methods for                environments (such as those           An example would be data sharing by
    designing experiments for large           containing malware).                  common carriers where the shared data
    testbeds).                              ƒƒ Developing better techniques for     uniquely identify individual customers.
  ƒƒ Developing methods and                   security testing across all domains   The government should also provide
    representations such as                   of conflict.                          more complete threat and adversary
    abstraction models to describe                                                  capability models for use in developing
                                            ƒƒ Developing integrated, cost-
    threats, so that designers can                                                  evaluation and testing criteria.
                                              effective methodologies and
    develop detailed specifications.          tools that systemically address
                                                                                    Other potential government activities
  ƒƒ Developing user interfaces, tools,       all of the above desiderata,
                                                                                    include the following:
    and capabilities to allow complex         including facilitation of scalable
    evaluations to be conducted.              trustworthiness (Section 1),            ƒƒ Propose evaluation methods that
                                              survivability (Section 7),                 are proven correct as national
  ƒƒ Developing tool sets that
                                              resistance to tampering and                or international standards for
    can grow with technology
                                              other forms of insider misuse              tech transfer. They also should
    (e.g., 64-bit words, IPv6).
                                              by developers (Section 4), rapid           be implemented in current
  ƒƒ Creating better techniques for           reevaluation after incremental             popular simulations and testbeds.
    testing combined performance,             changes, and suitable uses of              Industry should be encouraged
    usability, and security.                  formal methods where most                  to use these methods, perhaps via
  ƒƒ Developing understanding of              usefully applicable—among                  market incentives.
    how much realism matters and              other needs. The potential              ƒƒ Form attestation agencies that
    what type of realism is possible          utility of formal methods has              would evaluate products on the
    and useful.                               increased significantly in the past        market, using evaluation methods
                                              four decades and needs to be               that are ready for tech transfer,
Long-term, high-risk R&D challenges
                                              considered whenever it can be              and rank those products publicly.
include the following:
                                              demonstrably effective.                 ƒƒ Create a National CyberSecurity
  ƒƒ Developing models of correct                                                        and Safety Board that would
    operation for various network         Resources
                                                                                         collect attack reports from
    elements and networks at and          Academia and industry should col-              organizations and share them in
    across all levels of protocol         laborate to share data about traffic,          a privacy-safe manner. The board
    models.                               attacks, and network environments              could also mandate sharing.
  ƒƒ Developing metrics for attack        and to jointly define standards and            Another way is establishing

                                                                          SYSTEM EVALUATION LIFE CYCLE                    27
       a PREDICT-like repository                 developed by projects in other         well-established evaluation methods.
       for attack data sharing. Yet a            areas (e.g., solutions for critical    Direct comparisons of vendor products
       third way is developing market            system availability). Thus, various    will be possible, based on measures of
       incentives for data sharing.              evaluation methods could be            performance in standard tests.
     ƒƒ Fund joint academic/industry             compared with real-deployment
                                                 evaluations. Without this ground       What needs to be in place for
       partnerships in a novel way.
                                                 truth comparison, it is impossible     test and evaluation?
       Academics have a hard time
       finding industry partners that            to develop good evaluation             A flexible, scalable, and secure large-scale
       are willing to commit to tech             methods because evaluation must        testbed would enable high-fidelity tests
       transfer. A novel way would               correctly predict ground truth.        of products using new development and
       have government find several                                                     evaluation methods.
       partners from various fields:        Measures of success
       enterprises, ISPs, government        One key milestone as a measure of           To what extent can we test
       networks, SCADA facilities,          success will be the eventual adoption
                                                                                        real systems?
       security device manufacturers,       by standards bodies such as NIST or
       etc. These partners would            ISO of consistent frameworks, method-       Because system evaluation must occur
       pledge to provide data to            ologies, and tools for system evaluation.   at all phases of the life cycle, there
       researchers in the evaluation        System developers will be able to choose    should be opportunities to test new
       area and to provide small pilot      components from vendors based on            tools and methodologies on real systems
       deployments of technologies          results obtained from well-known and        inobtrusively.


References
[Ade2008]          S. Adee. The hunt for the kill switch. IEEE Spectrum, 45(5):32-37, May 2008
                   (http://www.spectrum.ieee.org/may08/6171).

[DSB2005]          Defense Science Board Task Force on High Performance Microchip Supply, February 2005
                   (http://www.acq.osd.mil/dsb/reports/2005-02-HPMS_Report_Final.pdf ).

[How+2006]         M. Howard and S. Lipner. The Security Development Life Cycle. Microsoft Press, Redmond, Washington, 2006.

[ISO1999]          International Organization for Standardization / International Electrotechnical Commission (ISO/IEC)
                   International Standard 15408:1999 (parts 1 through 3), Common Criteria for Information Technology Security
                   Evaluation, August 1999.

[NIS2008]          Security Considerations in the System Development Lhife Cycle. NIST Special Publication 800-64 Revision 2
                   (Draft), National Institute of Standards and Technology, Gaithersburg, Maryland, March 2008.




28      SYSTEM EVALUATION LIFE CYCLE
Current Hard Problems in INFOSEC Research
              4. Combatting Insider Threats
              BACKGROUND

              What is the problem being addressed?

              Cybersecurity measures are often focused on threats from outside an organization,
              rather than threats posed by untrustworthy individuals inside an organization.
              Experience has shown that insiders pose significant threats:

                 ƒƒ Trusted insiders are among the primary sources of many losses in the
                   commercial banking industry.
                 ƒƒ Well-publicized intelligence community moles, such as Aldrich Ames,
                   Robert Hanssen, and Jonathan Pollard, have caused enormous and
                   irreparable harm to national interests.
                 ƒƒ Many insiders involved in misuses were hired as system
                   administrators, became executives, or held other kinds of privileges
                   [Cap2008.1, Cap2008.2].

              This section focuses on insider threats to cyber systems and presents a roadmap for
              high-impact research that could aggressively curtail some aspects of this problem. At
              a high level, opportunities exist to mitigate insider threats through aggressive profil-
              ing and monitoring of users of critical systems, “fishbowling” suspects, “chaffing”
              data and services users who are not entitled to access, and finally “quarantining”
              confirmed malevolent actors to contain damage and leaks while collecting action-
              able counter-intelligence and legally acceptable evidence.

              There are many proposed definitions of the insider threat. For the purposes of this
              discussion, an insider threat is one that is attributable to individuals who abuse
              granted privileges. The scope of consideration here includes individuals masquerad-
              ing as other individuals, traitors abusing their own privileges, and innocents fooled
              by malevolent entities into taking adverse actions. Inadvertent and intentional
              misuse by privileged users are both within the scope of the definition. Although an
              insider can have software and hardware acting on his or her behalf, it is the indi-
              vidual’s actions that are of primary concern here. Software proxies and other forms
              of malevolent software or hardware—that is, electronic insiders—are considered in
              Section 5 on combatting malware and botnets.

              The insider threat is context dependent in time and space. It is potentially relevant
              at each layer of abstraction. For example, a user may be a physical insider or a
              logical insider, or both. The threat model must be policy driven, in that no one
              description will fit all situations.

              Unlike unauthorized outsiders and insiders who must overcome security controls to
              access system resources, authorized insiders have legitimate and (depending on their
              positions) minimally constrained access to computing resources. In addition, highly

                                                                                                   29
trusted insiders who design, maintain,        [Noo+2008], integrity, availability, and     brought forward by the research commu-
or manage critical information systems        total system survivability are of highest    nity are multilevel security (MLS), an
are of particular concern because they        priority and can be compromised by           example of mandatory access controls
possess the skills and access necessary to    insiders.                                    (MAC) that prevents highly sensitive
engage in serious abuse or harm. Typical                                                   information from being accessed by less
trusted insiders are system administra-       Beneficiary needs may include tools          privileged users. Some work has also
tors, system programmers, and security        and techniques to prevent and detect         been done on multilevel integrity (MLI
administrators, although ordinary users       malicious insider activity throughout        [Bib1977]), which prevents less trusted
may have or acquire those privileges          the entire system life cycle, approaches     entities from affecting more trusted
(sometimes as a result of design flaws        to minimize the negative impact of           entities. However, these are typically
and implementation bugs). Thus, there         malicious insider actions, education and     too cumbersome to be usable in all but
are different categories of insiders.         training for safe computing technology       the most extreme environments; even
                                              and human peer detection of insider          in such environments, the necessary
What are the potential                        abuses, and systems that are resilient       systems are not readily available. Access
                                              and can effectively remediate detected       controls that are used in typical business
threats?
                                              insider exploits. Of particular interest     environments tend to be discretionary,
The insider threat is often discussed         will be the ability to deal with multiple    meaning that the individual or group of
in terms of threats to confidentiality        colluding insiders—including detect-         individuals who are designated as owners
and privacy (such as data exfiltration).      ing potential abuses and responding to       of an object can arbitrarily grant or deny
However, other trustworthiness require-       them.                                        others access to the object. Discretion-
ments, such as integrity, availability,                                                    ary access controls (DAC) typically do
and accountability, can also be com-          What is the current state of                 not prevent anyone with read access to
promised by insiders. The threats span                                                     an object from copying it and sharing
                                              the practice?
the entire system life cycle, including                                                    the copy outside the reach of that user’s
not only design and development but           The insider threat today is addressed        access control system. They also do not
also operation and decommissioning            mostly with procedures such as aware-        ensure sufficient protection for system
(e.g., where a new owner or discov-           ness training, background checks, good       and data integrity. Further background
erer can implicitly become a de facto         labor practices, identity management         on these and other security-related issues
insider).                                     and user authentication, limited audits      can be found in [And08,Bis02,Pfl03].
                                              and network monitoring, two-person
Who are the potential                         controls, application-level profiling and    File and disk encryption may have some
                                              monitoring, and general access con-          relevance to the insider threat, to the
beneficiaries? What are their
                                              trols. However, these procedures are         extent that privileged insiders might
respective needs?
                                              not consistently and stringently applied     not be able to access the encrypted data
The beneficiaries of this research range      because of high cost, low motivation,        of other privileged users. Also of pos-
from the national security bodies operat-     and limited effectiveness. For example,      sible relevance might be secret splitting,
ing the most sensitive classified systems     large-scale identity management can          k-out-of-n authorizations, and possibly
to homeland security officials who            accomplish a degree of nonrepudiation        zero-knowledge proofs. However, these
need to share Sensitive But Unclassified      and deterrence but does not actually         would need considerable improvement
(SBU) information/Controlled Unclas-          prevent an insider from abusing granted      if they were to be effective in commercial
sified Information (CUI), and to health       privileges.                                  products.
care, finance, and many other sectors
where sensitive and valuable informa-         Technical access controls can be applied
tion is managed. In many systems, such        to reduce the insider threat but not elim-
as those operating critical infrastructures   inate it. The technologies traditionally

30     COMBATTING INSIDER THREATS
What is the status of current                [HDJ2006], various Columbia Univer-          ƒƒ Response strategy and privacy
research?                                    sity papers and a book on insider threats      protection for falsely accused
                                             (e.g., [Sto+2008]), and an NSA/ARDA            insider abuses. In particular,
Several studies of the insider threat have   (IARPA) report on classifications and          privacy-enhanced sharing of
been produced in the past 10 to 15 years,    insider threats [Bra2004] are relevant.        behavior models and advanced
although some of these rely on research      Also, the Schonlau data set for user           fishbowling techniques to enable
on access controls dating back as many       command modeling may be of interest            detailed monitoring and limit
as 40 years. These need to be compiled       (www.schonlau.net).                            damage by a suspected inside
and serve as input to a taxonomy of the                                                     threat. (See Section 10.)
threats and possible violations. Ongoing                                                  ƒƒ Behavior-based access control.
                                             FUTURE DIRECTIONS
and emerging research efforts include
                                                                                          ƒƒ Decoys, deception, tripwires in
the following:                               On what categories can we                      the open.
   ƒƒ The 2008 Dagstuhl summer               subdivide this topic?
                                                                                          ƒƒ Beacons in decoy (and real)
     seminar on Countering Insider
                                             Approaches for coping with insider             documents. Adobe and other
     Threats [Dag08] included
                                             misuse can be categorized as collect and       modern platforms perform a
     position papers that are being
                                             analyze (monitoring), detect (provide          great deal of network activity at
     considered for publication as
                                             incentives and data), deter (prevention        startup and during document
     a book. It represented a broad
                                             should be an important goal), protect          opening, potentially enabling
     assessment of a wide variety of
                                             (maintain operations and economics),           significant beaconing.
     considerations.
                                             predict (anticipate threats and attacks),    ƒƒ More pervasive monitoring
   ƒƒ Ongoing insider and identity           and react (reduce opportunity, capabil-        and profiling, coupled with
     management projects under               ity, and motivation and morale for the         remediation in the presence of
     the aegis of The Institute for          insider). For present purposes, these six      detected potential misuses.
     Information Infrastructure              categories are grouped pairwise into
     Protection (I3P)—for example,                                                        ƒƒ Controlled watermarking of
                                             three bins: collect and analyze, detect;
     decoy networking and honeypots,         deter, protect; and predict, react.            documents and services to trace
     correlating host and network                                                           sources.
     indicators of insider threats, and                                                   ƒƒ Useful data. The research
                                             What are the major research
     behavior-based access control.                                                         community needs much more
     Three papers from the I3P               gaps?
                                                                                            data and more realistic data sets
     identity management projects            Many gaps relating to insider threats need     for experimentation.
     were presented at IDtrust 2009.         to be better understood and remediated.
     See the references in Section 6.                                                     ƒƒ Procedures and technology for
                                                ƒƒ Checking. Better mechanisms              emergency overrides are needed
   ƒƒ Two Carnegie Mellon University
                                                  are needed for policy specification       in almost every imaginable
     reports on insider threats in
                                                  and automated checking (e.g.,             application, but must typically
     government [Cap2008.1] and in
                                                  role-based access control [RBAC]          be specific to each application.
     information technology generally,
                                                  and other techniques). However,           They are particularly important
     with emphasis on the financial
                                                  any such mechanism must have              in health care, military, and other
     sector [Cap2008.2]; see also
                                                  precise and sound semantics if it         situations where human lives
     [Ran2004] and [FSS2008].
                                                  is to be useful. (Some past work          depend on urgent access. The
In addition, a DoD Insider Threat                 on digital rights management              existing limitations are in part
to Information Systems report                     may be of some indirect interest          related to lack of motivation for
[IAT2008], a study of best practices              here.)                                    developing and using fine-grained

                                                                                 COMBATTING INSIDER THREATS                  31
       access controls. In addition,               with respect to preventing insider           to be established. Very few such
       emergency overrides can be                  misuse. In addition, even the                data sets on insider behavior are
       abused by insiders who feign or             existing controls are not used               available today, in part because
       exploit crises. Overall, approaches         to their full extent. Moreover,              victims are reluctant to divulge
       must be closely connected to                better mechanisms are needed                 details and in part because many
       policy specifications.                      for both active monitoring (for              cases remain unknown beyond
     ƒƒ Lessons may be learned from                detection and response) and                  local confines. What data should
       safety systems. For example,                passive monitoring (for later                be collected and how it should
       in process control applications,            analysis and forensics). Note                be made available (openly
       separate safety systems are used to         that the prevention/monitoring/              or otherwise), perhaps via
       ensure that a process is safely shut        recording/archiving mechanisms               trustworthy third parties, need to
       down when certain parameters                must themselves be able to                   be considered. Privacy concerns
       are exceeded because of failure             withstand threats, especially                must be addressed.
       of the control system or for any            when the defenders are also the
                                                                                             ƒƒ Systems need to be designed to
       unanticipated reasons. Analogous            attackers. Also, collection of
                                                                                                be auditable in ways sufficient to
       protection mechanisms for                   evidence that will stand up in
                                                                                                allow collection and analysis of
       an information system might                 court is an important part of
                                                                                                forensic-quality data.
       ensure that certain operations are          deterrence. To this end, forensic
                                                   mechanisms and information                ƒƒ Models are needed to represent
       never allowed, regardless of the
       privileges of the users attempting          must be separated from the                   both normal and abnormal
       them. Similarly, the principles of          systems themselves.                          insider activity. However, past
       least common mechanism and                                                               experience with pitfalls of such
       least privilege should be applied                                                        models needs to be respected.
                                              Advanced fine-grained differential access
       more consistently. Also relevant       controls; role-based access controls;          ƒƒ Methodologies are needed
       would be “safe booting” for self-      serious observance of separation of roles,        for measuring and comparing
       protected monitors.                                                                      techniques and tools meant to
                                              duties, and functionality; and the prin-
     ƒƒ From a user perspective,              ciple of least privilege also need to be          handle insider threats.
       security and usability must            integrated with functional cryptography      Detect
       generally be integrally aligned,       techniques, such as identity-based and         ƒƒ Detection of insider abuse and
       but especially with respect to         attribute-based encryption, and with              suspected anomalies must be
       insider misuse. For example,           fine-grained policies for the use of all          timely and reliable.
       users should not feel threatened       the above concepts.
                                                                                             ƒƒ Data mining, modeling, and
       unless they are actually threats
                                                                                                profiling techniques are needed
       to system integrity and to other       What are some exemplary
       users. (Interactions with the                                                            for detection of malicious insider
                                              problems for R&D on this                          activity.
       usability topic in Section 11 are
                                              topic?
       particularly relevant here.)                                                          ƒƒ Better techniques are needed
                                              The categories noted above and some               to determine user intent from
     ƒƒ Privacy is an important
                                              potential approaches are summarized               strict observation, as opposed to
       consideration, although it
                                              in Table 4.1.                                     merely detecting deviations from
       typically depends on the specific
       policies of each organization.         Collect and Analyze                               expected policies.
     ƒƒ Existing access controls tend           ƒƒ Data sets relating to insider             ƒƒ Prediction and detection need to
       to be inadequately fine-grained             behavior and insider misuse need             be effectively integrated.


32      COMBATTING INSIDER THREATS
                       TABLE 4.1: Potential Approaches to Combatting Insider Threats

  Category                                   Definition                                   PotentialƒApproaches
  Collect and Analyze, Detect                Understanding and identifying threats and    Broad-based misuse detection oriented to
                                             potential risks                              insiders
  Deter, Protect                             Trustworthy systems with specific policies   Inherently secure systems with differential
                                             to hinder insider misuse                     access controls
  Predict, React                             Remediation when insider misuse is           Intelligent interpretation of likely
                                             detected but not prevented                   consequences and risks


Deter                                              and particular policies, and to               more invisible might be useful
  ƒƒ Fine-grained access controls                  identify all relevant insiders                in addressing the insider threat.
     and correspondingly detailed                  therein. (See the section on                  Decoys must be conspicuous,
     accountability need to have                   System Evaluation Life Cycle.)                believable, differentiable (by
     adequate assurance. Audit logs                Note that in many cases there are             good guys), noninterfering, and
     must be reduced to be correctly                                                             dynamically changing.
                                                   no specific boundaries between
     interpreted, without themselves
                                                   inside and outside.                         ƒƒ New research is especially
     leaking information.
                                                                                                 needed in countering multiple
                                                ƒƒ Continuous user authentication
  ƒƒ Deterrence policies need to be                                                              colluding insiders. For example,
                                                   and reauthentication may be
     explored and improved. Training                                                             the development of defensive
                                                   desirable to address insider
     should include use of decoys.                                                               mechanisms that systematically
                                                   masquerading.
  ƒƒ Incentives need to be developed,                                                            necessitate multiple colluders
                                                ƒƒ System architectures need to                  would be a considerable
     such as increased risks of being
                                                   pervasively enforce the principle             improvement.
     caught, greater consequences
                                                   of least privilege, which is
     if caught, lessened payoffs                                                               ƒƒ Anti-tamper technologies are
                                                   particularly relevant against
     if successful, and decreased                                                                needed for situations where
                                                   insider threats. The principle
     opportunities for user                                                                      insiders have physical access to
                                                   of least common mechanism                     systems. Similar technologies may
     disgruntlement. The role of an
                                                   could also be useful, restricting             be desirable for logical insiders.
     ombudsperson should also be
                                                   functionality and limiting                    Inspiration from nuclear safety
     considered in this context.
                                                   potential damage. Access control              controls can illuminate some of
  ƒƒ Increased incentives for                      mechanisms must move beyond                   the concerns.
     anonymous whistle-blowing,                    the concept of too-powerful
     engendering an atmosphere of                                                              ƒƒ Protections are needed for
                                                   superuser mechanisms, by
     peer-level misuse detection and                                                             both system integrity and
                                                   splitting up the privileges as
     monitoring.                                                                                 data integrity, perhaps with
                                                   was done in Trusted Xenix.                    finer-grained controls than
  ƒƒ Social, ethical, and legal issues, as         Mechanisms such as k-out-                     for outsiders. In addition,
     well as human factors, need to be             of-n authorizations might also                operational auditing and
     addressed in a multidisciplinary              be useful. New access control                 rollback mechanisms are
     fashion.                                      mechanisms that permit some                   needed subsequent to integrity
Protect                                            of the discipline of multilevel               violations. Note that physical
  ƒƒ Using a life cycle view could                 security might also help.                     means (e.g.,write-once media)
     be helpful to establish security           ƒƒ Deception, diversity, and making              and logical means (log-structured
     perimeters for specific purposes              certain protection mechanisms                 file systems) are both relevant.

                                                                                   COMBATTING INSIDER THREATS                           33
     ƒƒ Mechanisms are needed that               communications, user behavior,            credentials, perfect forward
       exhaustively describe and enforce         and content. Prediction must              secrecy built into systems, and
       the privileges that a user is             address users and surrogates, as          other approaches that could
       actually granted. In particular,          well as their actions and targets.        simplify timely reactions.
       visualization tools are needed for   React                                       ƒƒ Note that these categories are
       understanding the implications         ƒƒ Automated mechanisms are                  somewhat interrelated. Any
       of both requested and granted             needed that can intercede when            research program related to
       privileges, relative to each user         misuse is suspected, without              coping with the insider threats
       and each object. This approach            jeopardizing system missions and          needs to keep this in mind.
       needs to include not just logical         without interfering with other            Table 4.2 summarizes some of the
       privileges, but also physical             users. For example, some sort of          research gaps, research initiatives,
       privileges.                               graceful degradation or system            benefits, and time-frame.
                                                 recovery may be needed, either
     ƒƒ Mechanisms are needed to
                                                 before misuse has been correctly     What are the near-term, mid-
       prevent overescalation of
                                                 identified or afterwards.            term, long-term capabilities
       privileges on a systemwide basis
       (e.g., chained access that allows      ƒƒ Mechanisms and policies are          that need to be developed?
       unintended access to a sensitive          needed to react appropriately        Near Term
       piece of data). However, note             to the detection of potentially        ƒƒ Compile and compare existing
       that neither trust nor delegation         actively colluding insiders.              studies relating to the insider
       is a transitive operation.             ƒƒ Architecturally integrated                threat. (Detect)
Predict                                          defense and response strategies        ƒƒ Develop data collection
  ƒƒ Various predictive models                   might mitigate the effects of             mechanisms and collect data.
     are needed—for example, for                 insider attacks—for example, an           (Detect)
     indicators of risks of insider              insider who is able to override        ƒƒ Evaluate suitability of existing
     misuse, dynamic precursor                   existing policies. One strategy of        RBAC R&D to address insider
     indicators for such misuse, and             considerable interest would be            threats. (Protect)
     determining what is operationally           unalterable (e.g., once-writable)
                                                                                        ƒƒ Develop anti-tampering
     relevant (such as the potentially           and non-bypassable audit trails
                                                                                           approaches. (Protect)
     likely outcomes).                           that cannot be compromised.
                                                 Another strategy would be              ƒƒ Explore the possible relevance
     ƒƒ Dynamic analysis techniques
       are needed to predict a system            mechanisms that cannot be                 of digital rights management
       component’s susceptibility to             altered without physical access,          (DRM) approaches. (Protect)
       a certain insider attack, based           such as overriding safety            Medium Term
       on system operations and                  interlocks.                           ƒƒ Develop feature extraction and
       configuration changes.                 ƒƒ Architecturally integrated               machine learning mechanisms to
     ƒƒ Profiles of expected good                response strategies might also be        find outliers. (Detect)
       behavior and profiles of possible         invoked when misuse is detected,       ƒƒ Develop tools to exhaustively and
       bad behavior are generally both           gathering forensics-worthy                accurately understand granted
       useful, but neither approach is           evidence of the potential network         privileges as roles and system
       sufficient. Additional approaches         of inside threats, adversary              configurations change. (Detect)
       are needed.                               sources and methods, to enable
                                                                                        ƒƒ Develop procedures to evaluate
     ƒƒ Better technologies are                  law-enforcement use of evidence.          insider threat protection methods
       needed to achieve meaningful           ƒƒ Research is needed on scalable            in reliable and comparable ways.
       prediction, including analysis of         mechanisms for revocable                  (Detect)

34      COMBATTING INSIDER THREATS
                                       TABLE 4.2: Gaps and Research Initiatives

IdentifiedƒGap                      ResearchƒInitiatives             Benefit                            TimeƒFrame
Inadequately fine-grained           Better mechanisms, policies,     Better detection and prevention    Near- to long term
access controls                     monitoring                       of insider misuse
Absence of insider-misuse aware Better detection tools               More precise detection of          Near term
detection                                                            insider misuse
Difficulties in remediation         Mixed strategies for finer-      Flexible response to detected      Longer term
                                    grained, continuous monitoring   misuses
                                    and action



  ƒƒ Develop better methods to                       advances in natural language            might be able to hinder insider misuse.
     combat insiders acting alone.                   understanding). (Protect)               For example, what might be the rela-
     (Protect)                                                                               tive merits of cryptographically based
                                                  ƒƒ Develop insider prediction
                                                                                             authentication, biometrics, and so on,
  ƒƒ Pursue the relevance and                        techniques for users, agents, and
                                                                                             with respect to misuse, usability, and
     effectiveness of deception                      actions. (React)
                                                                                             effectiveness? To what extent would
     techniques. (Protect)                                                                   various approaches to differential
  ƒƒ Incorporate integrity protection          What R&D is evolutionary and                  access controls hinder insider misuse?
     into authorization and system             what is more basic, higher                    Detectability of insider misuse and the
     architectures. (Protect)                  risk, game changing?                          inviolability of audit trails would also be
                                                                                             amenable to useful metrics.
  ƒƒ Develop behavior-based security,
                                               Intelligent uses of authentication, exist-
     for example, advanced decoy               ing access-control and accountability         The extent to which such localized
     networking. (Protect)                     mechanisms, and behavior monitor-             metrics might be composable into
  ƒƒ Develop and apply various risk            ing would generally be incremental            enterprise-level metrics is a challenge of
     indicators. (React)                       improvements. However, in the long            particular interest here.
                                               term, significantly new approaches are
Long Term
                                               desirable.                                    To what extent can we test
  ƒƒ Establish effective methods
     to apply the principle of least                                                         real systems?
     privilege. (Protect)
                                               Resources
                                                                                                 ƒƒ There is a strong need for
  ƒƒ Develop methods to address                Research, experimental testbeds, and                  realistic data for evaluation of
     multiple colluding insiders.              evaluations will be essential.                        technologies and policies that
     (Protect)                                                                                       counter insider threats. This
                                               Measures of success                                   must be done operationally in
  ƒƒ Pursue the architecture of insider-
                                                                                                     a relatively noninvasive way.
     resilient systems. (Protect)              Various metrics are needed with respect               Testbeds are needed, as well
  ƒƒ Pursue applications of                    to the ability of systems to cope with                as exportable databases of
     cryptography that might limit             insiders. Some will be generic: others                anonymized data (anonymization
     insider threats. (Protect)                will be specific to given applications and
                                                                                                     is generally a complicated
                                               given systems. Metrics might consider
  ƒƒ Develop automated decoy                                                                         problem).
                                               the extent to which various approaches
     generation (may require                   to authentication and authorization               ƒƒ Red teaming is needed to identify


                                                                                   COMBATTING INSIDER THREATS                        35
       potential attack vectors available        misuse can be expected to be               but rarely detected or reported. If
       to insiders and to test the               unique in their motivation                 budgets are limited, choices may
       relevance of potential solutions.         and execution, although there              have to be made regarding the
     ƒƒ Some effort should be devoted to         will be common modalities.                 relative importance of improving
       reliably simulating insider attacks       Thus, special care must be                 positive and negative detection
       and their system consequences.            devoted to understanding and               rates, and for which types of
                                                                                            misuse cases.
     ƒƒ Cases of insider misuse may              accommodating the implications
       represent statistically rare              of rare events. Alternatively,           ƒƒ Tests involving decoys might be
       events. Many cases of insider             insider misuse may be common               useful in training exercises.


References
[And2008]          R. Anderson. Security Engineering: A Guide to Building Dependable
                   Distributed Systems. Wiley, Indianapolis, Indiana, 2008.

[Bib1977]          K.J. Biba. Integrity Considerations for Secure Computer Systems. Technical Report MTR 3153,
                   The MITRE Corporation, Bedford, Massachusetts, June 1975. Also available from USAF
                   Electronic Systems Division, Bedford, Massachusetts, as ESD-TR-76-372, April 1977.

[Bis2002]          M. Bishop. Computer Security: Art and Science. Addison-Wesley Professional, Boston, Massachusetts, 2002.

[Bra2004]          Richard D. Brackney and Robert H. Anderson. Understanding the Insider Threat: Proceedings of a
                   March 2004 Workshop. RAND Corporation, Santa Monica, California, 2004
                   (http://www.rand.org/pubs/conf_proceedings/2005/RAND_CF196.pdf ).

[Cap2008.1]        D. Capelli, T. Conway, S. Keverline, E. Kowalski, A. Moore, B. Willke, and M. Williams. Insider Threat
                   Study: Illicit Cyber Activity in the Government Sector, Carnegie Mellon University, January 2008
                   (http://www.cert.org/archive/pdf/insiderthreat_gov2008.pdf ).

[Cap2008.2]        D. Capelli, E. Kowalski, and A. Moore. Insider Threat Study: Illicit Cyber Activity in the Information
                   Technology and Telecommunications Sector. Carnegie Mellon University, January 2008
                   (http://www.cert.org/archive/pdf/insiderthreat_it2008.pdf ).

[Dag2008]          Dagstuhl Workshop on Insider Threats, July 2008 (http://www.dagstuhl.de).

[FSS2008]          Financial Services Sector Coordinating Council for Critical Infrastructure Protection and
                   Homeland Security, Research and Development Committee. Research Agenda for the Banking
                   and Finance Sector. September 2008 (https://www.fsscc.org/fsscc/reports/2008/RD_Agenda-
                   FINAL.pdf ). Challenge 4 of this report is Understanding the Human Insider Threat.

[HDJ2006]          IT Security: Best Practices for Defending Against Insider Threats to Proprietary Data, National Defense
                   Journal Training Conference, Arlington, Virginia. Homeland Defense Journal, 19 July 2006




36      COMBATTING INSIDER THREATS
[IAT2008]    Information Assurance Technical Analysis Center (IATAC). The Insider Threat to Information
             Systems: A State-of-the-Art Report. IATAC, Herndon, Virginia, February 18, 2008.

[Kee2005]    M. Keeney, D. Cappelli, E. Kowalski, A. Moore, T. Shimeali, and St. Rogers. Insider Threat
             Study: Computer System Sabotage in Critical Infrastructure Sectors. Carnegie Mellon
             University, May 2005 (http://www.cert.org/archive/pdf/insidercross051105.pdf ).

[Moo2008]    Andrew P. Moore, Dawn M. Cappelli, and Randall F. Trzeciak. The “Big Picture” of IT Insider Sabotage
             Across U.S. Critical Infrastructures. Technical Report CMU/SEI-2008-TR-009, Carnegie Mellon
             University, 2008 (http://www.cert.org/archive/pdf/08tr009.pdf ). This report describes the MERIT model.

[Neu2008]    Peter G. Neumann. Combatting insider misuse with relevance to integrity and accountability in elections
             and other applications. Dagstuhl Workshop on Insider Threats, July 2008
             (http://www.csl.sri.com/neumann/dagstuhl-neumann.pdf ). This position paper expands on the
             fuzziness of trustworthiness perimeters and the context-dependent nature of the concept of insiders.

[Noo+2008]   Thomas Noonan and Edmund Archuleta. The Insider Threat to Critical Infrastructures. National
             Infrastructure Advisory Council, April 2008
             (http://www.dhs.gov/xlibrary/assets/niac/niac_insider_threat_to_critical_infrastructures_study.pdf ).

[Pfl2003]    Charles P. Pfleeger and Shari L. Pfleeger. Security in Computing, Third
             Edition. Prentice Hall, Upper Saddle River, New Jersey, 2003.

[Ran04]      M.R. Randazzo, D. Cappelli, M. Keeney, and A. Moore. Insider Threat Study: Illicit Cyber Activity in the
             Banking and Finance Sector, Carnegie Mellon University, August 2004
             (http://www.cert.org/archive/pdf/bankfin040820.pdf ).

[Sto+08]     Salvatore Stolfo, Steven Bellovin, Shlomo Hershkop, Angelos Keromytis, Sara Sinclair, and Sean
             Smith (editors). Insider Attack and Cyber Security: Beyond the Hacker. Springer, New York, 2008.




                                                                         COMBATTING INSIDER THREATS                  37
Current Hard Problems in INFOSEC Research
              5. Combatting Malware and Botnets
              BACKGROUND

              What is the problem being addressed?
              Malware refers to a broad class of attack software or hardware that is loaded on
              machines, typically without the knowledge of the legitimate owner, that compro-
              mises the machine to the benefit of an adversary. Present classes of malware include
              viruses, worms, Trojan horses, spyware, and bot executables. Spyware is a class of
              malware used to surreptitiously track and/or transmit data to an unauthorized third
              party. Bots (short for “robots”) are malware programs that are covertly installed
              on a targeted system, allowing an unauthorized user to remotely control the com-
              promised computer for a variety of malicious purposes [GAO2007]. Botnets are
              networks of machines that have been compromised by bot malware so that they
              are under the control of an adversary.

              Malware infects systems via many vectors, including propagation from infected
              machines, tricking users to open tainted files, or getting users to visit malware-
              propagating websites. Malware may load itself onto a USB drive inserted into
              an infected device and then infect every other system into which that device is
              subsequently inserted. Malware may propagate from devices and equipment that
              contain embedded systems and computational logic. An example would be infected
              test equipment at a factory that infects the units under test. In short, malware can
              be inserted at any point in the system life cycle. The World Wide Web has become
              a major vector for malware propagation. In particular, malware can be remotely
              injected into otherwise legitimate websites, where it can subsequently infect visitors
              to those supposedly “trusted” sites.

              There are numerous examples of malware that is not specific to a particular operat-
              ing system or even class of device. Malware has been found on external devices (for
              example, digital picture frames and hard drives) and may be deliberately coded into
              systems (life cycle attacks). Increasingly intelligent household appliances are vulner-
              able, as exemplified by news of a potential attack on a high-end espresso machine
              [Thu2008]. Patching of these appliances may be difficult or impossible. Table 5.1
              summarizes malware propagation mechanisms.

              Potentially victimized systems include end user systems, servers, network infra-
              structure devices such as routers and switches, and process control systems such as
              Supervisory Control and Data Acquisition (SCADA).

              A related policy issue is that reasonable people may disagree on what is legitimate
              commercial activity versus malware. In addition, ostensibly legal software utilities
              (for example, for digital rights management [DRM]) may have unintended conse-
              quences that mimic the effects of malware [Sch2005, Hal2006].


38
It is likely that miscreants will develop         systems until attribution can be         consequences of botnets and malware
new infection mechanisms in the future,           accomplished. Honeypots can              include spam, distributed denials of
either through discovery of new security          also be useful in this regard.)          service (DDoSs), eavesdropping on
gaps in current systems or through new                                                     traffic (sniffing), click fraud, loss of
exploits that arise as new communi-          The NSA/ODNI Workshop on                      system stability, loss of confidentiality,
cation and computation paradigms             Computational Cyberdefense in                 loss of data integrity, and loss of access
emerge.                                      Compromised Environments, Santa               to network resources (for example,
                                             Fe, NM, August  2009, was an                  being identified as a bot node and
The technical challenges are, wherever       example of a step in this direction           then blocked by one’s ISP or network
possible, to do the following:               (http://www.c3e.info).                        administrator, effectively a DoS inflicted
                                                                                           by one victim on another). An increas-
  ƒƒ Avoid allowing malware onto a
                                             What are the potential                        ing number of websites (such as popular
      platform.
                                                                                           social networking systems, web forums,
  ƒƒ Detect malware that has been            threats?
                                                                                           and mashups) permit user-generated
      installed.                             Malware has significant impact in many        content, which, if not properly checked,
  ƒƒ Limit the damage malware can            aspects of the information age and            can allow attackers to insert rogue
      do once it has installed itself on a   underlies many of the topics discussed        content that is then potentially down-
      platform.                              elsewhere in this document. Impacts           loaded by many users.
  ƒƒ Operate securely and effectively        can be single-host to networkwide, nui-
      in the presence of malware.            sance to costly to catastrophic. Negative     Beyond its nuisance impact, malware
  ƒƒ Determine the level of risk             consequences include degraded system          can have serious economic and national
      based on indications of detected       performance and data destruction or           security consequences. Malware can
      malware.                               modification. Spyware permits adver-          enable adversary control of critical com-
  ƒƒ Remove malware once it has              saries to log user actions (to steal user     puting resources, which in turn may
      been installed (remediation), and      credentials and facilitate identity theft,    lead, for example, to information com-
      monitor and identify its source        for example), while bot malware enables       promise, disruption and destabilization
      (attribution). (Remediation            an adversary to build large networks of       of infrastructure systems (“denial of
      may sometimes be purposefully          compromised machines and amplify an           control”), and manipulation of financial
      delayed on carefully monitored         adversary’s digital firepower. Negative       markets.


                                  TABLE 5.1: Malware Propagation Mechanisms

MalwareƒPropagationƒMechanism                                      Examples
Life cycle                                                         From the developer, either deliberate or through the use of infected
                                                                   development kits.
Scan and Exploit                                                   Numerous propagating worms. May propagate without requiring
                                                                   action on the part of the user.
Compromised Devices                                                Infected USB tokens, CDs/DVDs, picture frames, etc.
Tainted File                                                       E-mail attachment
Web                                                                Rogue website induces user to download tainted files. (Note: Newer
                                                                   malware may infect victims’ systems when they merely visit the
                                                                   rogue site, or by redirecting them to an infected site via cross-site
                                                                   scripting, for example)




                                                                       COMBATTING MALWARE AND BOTNETS                                 39
Malware can be particularly damaging          in 2003, even though these systems         The potential of malware to compromise
to elements of the network infrastruc-        were supposedly immune to such an          confidentiality, integrity, and availability
ture. Attacks against the Domain Name         attack (the plant was not online at the    of the Internet and other critical infor-
System (DNS), for example, could              time) [SF2003]. Propagating malware        mation infrastructures is another serious
direct traffic to rogue sites and enable      may have exacerbated the impact of         concern. A real-world example would
a wide variety of man-in-the-middle           the 2003 blackout in the northeastern      be the attacks on Estonia’s cyber infra-
and denial-of-service attacks. Successful     United States and slowed the recovery      structure via a distributed botnet in the
attacks against DNS allow an adversary        from it. It is reasonable to assume that   spring of 2007 [IW2007]. That incident
to intercept and redirect traffic, for        malware authors will target embedded       raised the issue of whether “cyberwar” is
example to rogue or spoofed servers. In       systems and emerging initiatives, such     covered under NATO’s collective self-
addition to redirection to rogue servers,     as the Advanced Metering Infrastructure    defense mission. In the absence of robust
there is also the opportunity for selective   (AMI) for electric power.                  attribution, the question remains moot.
or timed denial-of-service attacks; it may                                               There were reports of a cyber dimension
be easier to drop a site from DNS than        There is also the impact associated with   in the August 2008 conflict in the nation
to deny availability by flooding its con-     remediating compromised machines.          of Georgia, but the cyber attacks were
nection. These concerns underlay the          From an ISP’s point of view, the biggest   apparently limited to denials of service
recent mandate to implement DNSSEC            impacts include dealing with customer      against Georgian government websites
for the .gov domain and recommenda-           support calls, purchasing and distrib-     and did not target cyberinfrastructure
tions to implement DNSSEC for DNS             uting antivirus (A/V) software, and        [Ant2008]. A recent malware-du-jour is
root servers.                                 minimizing customer churn. For some        Conficker, which spread initially primar-
                                              high-consequence government applica-       ily through systems that had not been
Adversaries buy and sell exploits and         tions, an infection may even necessitate   upgraded with security patches, and has
lease botnets in an active adversary          replacement of system components/          subsequently reappeared periodically in
market [Fra2007]. These botnets can be        hardware.                                  increasingly sophisticated versions.
used for massive distributed attacks,
spam distribution, and theft of sensi-        Who are the potential                      The law enforcement and DoD com-
tive data, such as security credentials,                                                 munities are particularly interested in
                                              beneficiaries? What are their
financial information, and company                                                       attribution, which, as noted above, is
                                              respective needs?
proprietary information, through                                                         currently difficult.
sophisticated phishing attacks. The use       Malware potentially affects anyone who
of botnets makes attribution to the           uses a computer or other information       What is the current state of
ultimate perpetrator extremely difficult.     system. Malware remediation (clean-
                                                                                         the practice?
Botnets provide the adversary with vast       ing infected machines, for example) is
resources of digital firepower and the        difficult in the case of professionally    Deployed solutions by commercial anti-
potential to carry out surveillance on        administered systems and beyond the        virus and intrusion detection system/
sensitive systems, among other threats.       technical capability of many private       intrusion prevention system (IDS/IPS)
                                              citizens and small office/home office      vendors, as well as the open-source com-
Malware propagation is usually dis-           (SOHO) users. Rapid, scalable, usable,     munity, attempt to detect or prevent
cussed in the context of enterprise and       and inexpensive remediation may be         an incoming infection via a variety
home computing. However, it also has          the most important near-term need          of vectors. A/V removal of detected
the potential to affect control systems       in this topic area. Improved detection     malware and system reboot are currently
and other infrastructure systems. For         and quarantine of infected systems are     the primary cleanup mechanisms. The
example, the alarm systems at the             also needed, as discussed below. Ben-      fundamental challenge to this approach
Davis-Besse nuclear plant in Ohio             eficiaries, challenges, and needs are      is that miscreants can release repacked
were infected by the Slammer worm             summarized in Table 5.2.                   and/or modified malware continually,

40     COMBATTING MALWARE AND BOTNETS
                               TABLE 5.2: Beneficiaries, Challenges, and Needs

Beneficiaries                               Challenges                                       Needs
Users                                       Under attack from multiple malware               User-friendly prevention, detection,
                                            vectors; Systems not professionally              containment, and remediation of malware
                                            administered
Administrators                              Protect critical systems, maintain continuity,   New detection paradigms, robust
                                            enterprise-scale remediation in face of          remediation, robust distribution of
                                            explosive growth in malware variants             prevention and patches
Infrastructure Systems                      Prevent accidental infection [SF 2003],          Similar to administrator needs, but often
                                            address the growing challenge of targeted        with special constraints of legacy systems
                                            infection                                        and the inability to patch and reboot at
                                                                                             arbitrary times
ISPs                                        Provide continuity of service, deal with         Defenses against propagating attacks and
                                            malware on more massive scale than               botnets; progress in the malware area has
                                            administrators face                              potential immediate impact in alleviation of
                                                                                             these consequences
Law Enforcement                             Counter growing use of malware and               Robust attribution, advances in forensics
                                            botnets for criminal fraud and data and
                                            identity theft
Government and DoD                          Growing infection of defense systems,            Share the needs of administrators, ISPs, and
                                            such as the Welchia intrusion into the Navy      law enforcement
                                            Marine Corps Intranet (NMCI) [Messmer
                                            2003]. More recently, there have been
                                            reports of malware engineered specifically
                                            to target defense systems [LATimes08]



while new A/V signatures take time          Web-based A/V services have entered               Vendors of operating systems and
to produce, test, and distribute. In        the market, some offering a service               applications have developed mecha-
addition, it takes time for the user com-   whereby a security professional can               nisms for online updating and patching
munity to develop, test, and deploy         submit a suspicious executable to see             software for bugs, including bugs that
patches for the underlying vulnerability    whether it is identified as malicious by          affect security. Other defenses include
that the malware is exploiting. Further-    current tools. This mechanism most                antispyware, whitelists of trusted web-
more, the malware developers can test       likely functions also as a testbed for            sites and machines, and reputation
their software against the latest A/V       malware developers (VirusTotal).                  mechanisms.
versions.                                   [Vir].

Research in malware detection and           The U.S. National Institute of Stan-              Current detection and remediation
prevention is ongoing. For example,         dards and Technology (NIST) Security              approaches are losing ground, because
see the Cyber-Threat Analytics project      Content Automation Protocol (SCAP)                it is relatively easy for an adversary
(http://www.cyber-ta.org). Also worth       is a method for using specific stan-              (whether sophisticated or not) to
noting is the Anti-Phishing Working         dards to enable automated vulnerability           alter malware to evade most existing
Group (APWG):                               management, measurement, and policy               detection approaches. Given trends in
http://www.antiphishing.org.                compliance evaluation.                            malware evolution, existing approaches

                                                                        COMBATTING MALWARE AND BOTNETS                                    41
(such as A/V software and system patch-        a virtualized environment on a par-         of, malware infection. For example,
ing) are becoming less effective. For          ticular host) [Vra2005] and honeynets       DNS zone changes may predict a spam
example, malware writers have evolved          (network environments, partially            attack. Fast flux of DNS registrations (as
strategies such as polymorphism,               virtual, deployed on unused address         in Conficker) may indicate that particu-
packing, and encryption to hide their          space, that interact with malware in        lar hosts are part of the command and
signature from existing A/V software.          such a way as to capture a copy to enable   control (C2) network for a large botnet.
There is also a window of vulnerabil-          further analysis) [SRI2009]. Malware        Encrypted traffic on some network ports
ity between the discovery of a new             is increasingly engineered to detect        may indicate C2 traffic to a botnet client
malware variant and subsequent system          virtual and honeynet environments           on a given host.
patches and A/V updates. Further,              and change its behavior in response.
malware authors also strive to disable         There is industry research advancing        Virtualization and honeynets still
or subvert existing A/V software once          virtual machines to the Trusted Platform    provide much potential in malware
their malware has a foothold on the            Module (TPM) and hypervisor technol-        detection, analysis, and response, at
target system. (This is the case with a        ogy in hardware and software, as well       least for the near and medium terms.
later version of Conficker, for example.)      as in cleanup/remediation (technically      For honeynets to continue to be useful,
A/V software may itself be vulnerable          possible to do remotely in some cases,      research must address issues such as:
to life cycle attacks that subvert it prior    but with unclear legal and policy impli-
to installation. Patching is a necessary       cations if the system owner has not given      ƒƒ What features of honeynets do
system defense that also has drawbacks.        prior permission). The Department of             adversaries look for to identify
For example, the patch can be reverse          Homeland Security has funded ongoing             them as honeynets?
engineered by the adversary to find            research in cross-domain attack correla-       ƒƒ What is the ratio of “enter and
the original vulnerability. This may           tion and botnet detection and mitigation         retreat” to “enter and attack” in
allow the malware writers to refine            [CAT2009]. Analysis techniques include           honeynets?
their attacks against the unpatched            static and dynamic analysis methods
                                                                                              ƒƒ How does what is actually
systems. Much can be learned from              from traditional computer science.
                                                                                                observed in a honeynet compare
recent experiences with successive ver-
                                                                                                with known “script kiddie”
sions of Conficker.                            There is considerable research into open-
                                               source IDS (SNORT and Bro) along the             attacks and targeted malware
Specifically with respect to identity theft,   lines of expanding the signature base            activity in the real world?
which is one potential consequence             and defending these systems against
of malware but may be perpetrated              adversarial intentions. Recent research     DARPA’s Self-Regenerative Systems
by other means, there is an emerging           has considered automatic signature gen-     (SRS) program developed some technol-
commercial market in identity theft            eration from common byte sequences in       ogy around these techniques.
insurance and remediation. This implies        suspicious packet payloads [Kim2004]
that some firms believe they have ade-         as a countermeasure to polymorphic          Artificial diversity is transparent to
quate metrics to quantify risk in this         malware.                                    correct system use but diverse from the
case.                                                                                      point of view of some exploits. This has
                                               Significant research has been done into     been an elusive goal, but some modest
What is the status of current                  analysis of execution traces and similar    progress has been made in the com-
                                               characteristics of malware on an infected   mercial and research sectors. Address
research?
                                               host, but we have a poor understand-        space randomization is now included
There is considerable activity in malware      ing of the network dimensions of the        in many operating systems; and there
detection, capture, analysis, and defense.     malware problem. Certain network            has been some work in the general area
Major approaches include virtualiza-           behaviors have been observed to be          of system obfuscation (equivalent func-
tion (detect/contain/capture within            important precursors to, or indicators      tionality with diverse implementation)

42     COMBATTING MALWARE AND BOTNETS
[Sha2004], although it has some funda-       IT experts in order to develop effective       agility and polymorphism of malware.
mental limitations.                          defenses. Reaction is supported by cost-       Automatic detection of the command
                                             effective, secure remediation that can be      and control structure of a malware
Emerging approaches such as behavior-        implemented by non-IT professionals.           sample is a significant challenge.
based detection and semantic malware
descriptions have shown promise and are      What are the major research                    We do not have an adequate taxon-
deployed in commercial A/V software.                                                        omy of malware and botnets. It has
                                             gaps?
However, new techniques must be devel-                                                      been observed that many examples
oped to keep pace with the development       A/V and IDS/IPS approaches are                 of malware are derived from earlier
of malware.                                  becoming less effective because malware        examples, but this avenue has not been
                                             is becoming increasingly sophisticated,        explored as far as necessary. Progress
                                             and at any rate the user base (particularly    in this area may enable, for example,
FUTURE DIRECTIONS                            consumer systems) does not keep A/V            defenses against general classes of
                                             up to date. Malware polymorphism               malware, including as-yet unseen
On what categories can we                    is outpacing signature generation and          variants of current exemplars. A well-
subdivide this topic?                        distribution in A/V and IDS/IPS.               understood taxonomy may also support
For this malware and botnets topic,                                                         and improve attribution.
prevent/protect/detect/analyze/react         Current research initiatives do not
provides a reasonable framework (see         adequately address the increasing              The attacker-defender relation is cur-
Table 5.3). Protection and detection         sophistication and stealth of malware,         rently asymmetric. An attacker who
are supported by instrumented virtual-       including the encryption and packing           develops an exploit for a particular
ization and sandboxing environments to       of the malicious code itself, as well          system type will find large numbers of
combat inherently secure systems, appli-     as encrypted command and control               nearly identical exemplars of that type.
cations, and protocols. Analysis consists    channels and fast-flux DNS for botnets         Thus, it is desirable to force the adver-
of examination of captured malware (for      [Sha2008, Hol2008]. Broadly speaking,          sary to handcraft exploits to individual
example, harvested on a honeynet) by         research should better understand the          hosts, so that the cost of developing


                                            TABLE 5.3: Potential Approaches

 Category                                    Definition                                    PotentialƒApproaches
 Prevent                                     Prevent the production and propagation of     IDS/IPS, A/V, Virtualization, Inherently secure
                                             malware                                       systems
 Protect                                     Protect systems from infection when           IPS, A/V, Inherently secure systems
                                             malware is in the system’s environment
 Detect                                      Detect malware as it propagates on            IDS/IPS, A/V, Virtualization, Deceptive
                                             networks, detect malware infections on        environments
                                             specific systems
 Analyze                                     Analyze malware’s infection, propagation,     Static and dynamic analysis,
                                             and destructive mechanisms                    Experimentation in large-scale secured
                                                                                           environments
 React                                       Remediate a malware infection and identify    Updated IDS/IPS and A/V, Inherently
                                             mechanisms to prevent future outbreaks        secure systems, Thin client, Secure cloud
                                             (links to the prevent category)               computing paradigm




                                                                       COMBATTING MALWARE AND BOTNETS                                   43
malware to compromise a large number          What are some exemplary                           complexity of security controls, and rogue
of machines is raised significantly. Arti-    problems for R&D on this                          content injection, users can be tricked
ficial diversity can address the growing      topic?                                            into interacting with adversary systems
asymmetry of the attacker-defender                                                              while thinking they are performing valid
relation.                                     Robust Security Against OS Exploits:              transactions, such as online banking.
                                              Although binary-exploit malware target-           Research in this area should advance user
For hosts, the defenses against malware       ing the OS is still important and worthy          education and awareness and make secu-
(e.g., A/V software, Windows Update,          of incremental near-term investment,              rity controls more usable, particularly in
and so on) are typically part of or exten-    malware increasingly targets browsers             browsers. Search engine manipulation
sions to operating systems (OSs). This        and e-mail through social engineering             causes the victim to go to the malware
fact allows malware to easily target and      and other mechanisms.                             (e.g., at an infected website) rather than
disable those host-based defenses. A                                                            the malware’s targeting the user (e.g., via
summary of the gaps are outlined in           Protect Users from Deceptive Infections:          phishing e-mail). Server-side attacks in
Table 5.4.                                    At present, through social engineering,           the form of Structured Query Language


                                      TABLE 5.4: Gaps and Research Initiatives

 IdentifiedƒGap                   ResearchƒInitiatives                Benefit                             TimeƒFrame
 Inadequate defenses against      Human factors analysis to           More secure present and future      Near
 e-mail and web malware           resist social engineering (tools,   e-commerce
                                  interfaces, education), Robust
                                  whitelisting
 Escape from virtual machines     TPM low in the hardware/            Prolongs usefulness of              Near
                                  software stack                      virtualization as a defensive
                                                                      strategy
 Difficulty of remediation        Thin client, Automatic              Fast, cost-effective recovery       Near
                                  remediation                         from attack
 Inadequate test environments     Internet-scale emulation            Safe observation of malware         Near
                                                                      spread dynamics, better
                                                                      containment strategies
 Attacker/defender asymmetry      Intentional diversity, Inherently   Attacker must craft attack for a    Medium/Long
                                  monitorable systems                 large number of platforms
 No attack tolerance              Attack containment, Safe          Correct operation in the              Medium
                                  sandboxing, Intentional diversity presence of “subclinical”
                                                                    malware infection
 Detection approaches losing the Inherently monitorable systems,      Less space for attacker to          Medium/Long
 battle of scale                 Robust software whitelisting,        conceal activity
                                 Model-based monitoring of
                                                                      Detection that is generalized
                                 correct software behavior
                                                                      and scalable
 Inadequately understood threat   Analysis of adversary markets,      Strategic view enables defensive Long
                                  Penetration of adversary            community to take the upper
                                  communities, Containing             hand
                                  damage of botnets while
                                  observing




44     COMBATTING MALWARE AND BOTNETS
(SQL) injection, cross-site scripting, and   useful. The general research question is      that is difficult to define. Moreover,
other methods are increasingly common        how “deception” can be best leveraged         sharing malware may be illegal, depend-
ways to infect clients accessing compro-     by defenders.                                 ing on the business of the entity.
mised website.
                                             There are concerns about the limitations      Collaborative detection supports an
Internet-scale emulation could               of these approaches. Even a correctly         identified need in the situational under-
provide game-changing breakthroughs          functioning hypervisor is inadequate          standing topic area. In particular, the
in malware research. Being able to           in case of some flaws in the guest OS,        detection, quarantine, and remedia-
observe malware (specifically botnets        for example. Also, highly sophisticated       tion of botnet assets is a major overlap
and worms) at Internet scales without        malware is likely to be able to escape cur-   between the research needs for malware
placing the real Internet in jeopardy may    rent-generation virtual environments.         and those of situational understanding
help identify weaknesses in the malware      Improved hardware architecture access         (Section 8). Network-level defenses
code and how it spreads or reacts to         mechanisms will maintain the effective-       must come online to supplement host-
outside stimuli. Additionally, charac-       ness of these approaches to some degree.      level defenses. For example, we require
teristics observed at the macro level may    However, additional research is needed        better identification of bad traffic at the
give us clues as to how to detect and        on techniques that seize the strategic low    carrier level. This presents challenges in
respond to malware at the micro level.       ground within our computing systems           scale and speed.
High-fidelity large-scale emulation is an    and also separate the security func-
important enabling capability for many       tions from other functionality. The key       Thin-client technology has been pro-
of the other initiatives discussed below.    insight is that our detection methods         posed in the past. In this model, the
                                             and instrumentation must reside lower         user’s machine is stateless, and all files
The broad area of virtualization and         in the hardware/software stack than           and applications are distributed on some
honeynets will provide much value            the malware. Otherwise, the malware           network (the terminology “in the cloud”
in the near and medium terms, with           controls the defenders’ situational aware-    is occasionally used, although there are
respect to protection and detection          ness, and the defenders have no chance.       also parallels with traditional main-
approaches. Malware is becoming more         Recent research injecting vulnerabilities     frame computing). If we can make the
adaptive, in terms of polymorphism and       into hardware designs suggests disturb-       distributed resources secure, and that is
evasion techniques. The latter might         ing possibilities for the future on this      itself a big question, the attacker options
be used to a defensive advantage. If         front.                                        against user asset are greatly reduced,
malware is designed to be dormant if                                                       and remediation is merely a question
it detects that it is in a virtual machine   Collaborative detection may involve           of restarting. The long-term research
or in a honeynet environment, active         privacy-preserving security information       challenges toward this secure cloud
deception on the part of the defender        sharing across independent domains            computing paradigm are securing the
(making production systems look like         that may not have an established trust        distributed resource base and making
virtual systems and production net-          relationship. We may share malware            this base available to the authenticated
works look like honeynets, and vice          samples, metadata of a sample, and            and authorized user from any location,
versa; changing virtual and real systems     experiences. A repository of active           supported by a dedicated, integrated
very rapidly; or even the use of an analog   malware may accelerate research               infrastructure.
to a “screen saver” that toggles a com-      advances but raises security concerns in
puter from real to honeynet when the         its own right, and access must be care-       Remediation of infected systems is
user is not actively using it) may prove     fully controlled according to a policy        extremely difficult, and it is arguably




                                                                       COMBATTING MALWARE AND BOTNETS                              45
impossible to assert that a previously        research area. Short-term work in trusted    are required.
infected system has in fact been thor-        paths to all devices may reduce the risk
oughly cleansed. In particular, systems       of, for example, key logging software.       Not enough is being done in threat
may be infected with rootkits, which          In the short term, we require advances       analysis. In any case, the nature of
come in many forms, from user level           in authenticated updates, eventually         the threat changes over time. One
to kernel level rootkits. More recently,      evolving systems that are immune to          interesting avenue of research is eco-
hardware virtual machine (HVM) root-          malware. Advances in this area relate to     nomic analysis of adversary markets.
kits have been proposed, which load           the scalable trustworthy systems topic       Attackers sell malware exploits (and
themselves into an existing operating         in Section 1.                                also networks of infected machines, or
system, transforming it into a guest OS                                                    botnets). The price fluctuations may
controlled by the rootkit [Dai2006].          A longer-term research challenge is to       permit analysis of adversary trends and
We require advances in remediation,           develop systems, applications, and pro-      may also enable definition of metrics as
built-in diagnostic instrumentation,          tocols that are inherently more secure       to the effectiveness of defenses. Related
and VM introspection that provides            against malware infection and also easier    to the economic approach is research
embedded digital forensics to deal with       to monitor in a verifiable way (in effect,   into making malware economically less
these threats.                                to reduce the space in which malware         attractive to adversaries (for example,
                                              can hide within systems). In particu-        by much better damage containment,
Containment technology (which                 lar, hardware-based instrumentation          increasing the effectiveness of attribu-
includes TPM approaches mentioned             that provides unbiased introspection         tion, limiting the number of systems
previously) is promising but needs            for and unimpeded control of COTS            that can be targeted with a given exploit,
further work. An interesting goal is to       computing devices, while being unob-         and changing existing laws/policies so
tolerate malware (for example, safely         servable by the malware, may help            that the punishments reflect the true
doing a trusted transaction from a            enable embedded forensics and intrinsi-      societal cost of cybercrime).
potentially untrusted system). Another        cally auditable systems.
goal is to have a “safe sandbox” for criti-
cal transactions (in contrast to current      Artificial diversity can take many           What R&D is evolutionary,
sandboxing environments that typi-            forms: the code is different at each site,   and what is more basic,
cally seek to contain the malware in the      the location of code is different, system    higher risk, game changing?
sandbox). A final issue is whether large      calls are randomized, or other data is       In the near term, we are in a defensive
systems can achieve their goal while          changed. It may be worth researching         struggle, and R&D should continue
tolerating a residual level of ongoing        (both in terms of practicality and eco-      in the promising areas of virtualization
compromise within their components            nomics) how to randomize instruction         and honeynets. We require near-term
and subsystems. Generally, the research       sets, operating systems, and libraries       advances in remediation to address
agenda should recognize that malware          that are loaded from different system        the serious and increasing difficulty
is part of the environment, and secure        reboots. A difficult end goal would be       of malware cleanup, particularly on
operation in the presence of malware is       to develop systems that function equiva-     end-user systems. Research in the area
essential.                                    lently for correct usage but are unique      of attack attribution in the near and
                                              from an attack standpoint, so an adver-      medium terms can aid the policing that
Development of inherently secure,             sary must craft attacks for individual       is necessary on the Internet. Mecha-
monitorable, and auditable systems            machines. Artificial diversity is just one   nisms to share data from various kinds of
has presented a significant challenge. In     approach to changing the attacker-           malware attacks are currently lacking, as
general, this is a medium- to long-term       defender asymmetry, and novel ideas          well. The problems faced by researchers




46
       COMBATTING MALWARE AND BOTNETS
in this domain range from privacy con-      that must be detected in order to claim    It would be beneficial to have reliable
cerns, legal aspects of data sharing, and   effectiveness at some level.               metrics that estimate the vulnerability
the sheer volume of data itself. Research                                              of particular systems to corruption by
in generating adequate metadata and         We can define measures of success at       malware, and how well they are able
provenance is required to overcome          a high level by answering the follow-      to withstand other kinds of malware-
these hurdles.                              ing questions and tracking the answers     enabled attacks, such as DDoS attacks.
                                            over time:                                 Similarly, metrics that suggest the ben-
Techniques to capture and analyze                                                      efits that will accrue with the use of
                                              ƒƒ How many machines do we
malware and propagate defenses faster                                                  particular malware prevention or reme-
are essential in order to contain epidem-        know about that serve malware?        diation strategies would be helpful.
ics. Longer-term research should focus        ƒƒ What is the rate of emergence of
on inherently secure, monitorable, and           new malware?
auditable systems. Threat analysis and        ƒƒ Since spam is a primary botnet        What needs to be in place for
economic analysis of adversary markets           output, what fraction of e-mail is    test and evaluation?
should be undertaken in pilot form in            spam?
the near term, and pursued more vigor-                                                 Beyond reverse engineering of malware,
                                              ƒƒ What is the industry estimate of
ously if they are shown to be useful.                                                  the most effective studies of malicious
                                                 hosts serving malware?                code have taken place on network test-
                                              ƒƒ What is the trend in malware          beds. These testbeds have included
Measures of success                              severity (on a notional               simple virtual machines “networked”
We require baseline measurements of              continuum, say from nuisance to       on an analyst’s computer, testbeds
the fraction of infected machines at any         adware, spyware, bot capture)?        consisting of tens or hundreds of real
time; success would be a reduction in         ƒƒ What fraction of known attacks        (nonvirtualized) nodes, such as DETER
this fraction over time.                         is successful, and what fraction is   [DET], and simulated networks created
                                                 thwarted?                             within network simulation tools. The
Some researchers currently track the                                                   research community has yet to approach
emergence of malware. In this way, they     We may also consider cost-based mea-       studies of malware in Internet-scale
are able to identify trends (for example,   sures (from the defender point of view),   emulated environments. The infrastruc-
the number of new malware samples per       such as:                                   ture and tools do not currently exist to
month). A reversal of the upward trend                                                 build emulation environments on the
                                              ƒƒ What is our cost of searching for
in malware emergence would indicate                                                    order of 10,000,000 nodes or more.
success.                                         malware propagators?
                                              ƒƒ What is the cost to identify          As malware sophistication improves to
Time between malware capture and                 botnets and their bot command         include detection of virtual environ-
propagation of defense (or, perhaps              and control infrastructures?          ments, the realism of the virtualization
more appropriately, implementation            ƒƒ What is the cost to increase          environment (for example, virtual
of the defense on formerly vulnerable            sharing of malware host lists?        machine or honeynet) testbed presents
systems) tracks progress in human and                                                  a challenge.
automated response time.                    Economic analysis of adversary markets
                                            may allow definition of metrics as to      Tools and environments to study
With reference to the repository, we        effectiveness of particular defenses.      malware need to evolve as the malware
may define a minimal set of exemplars                                                  evolves. In particular, the community




                                                                     COMBATTING MALWARE AND BOTNETS                         47
currently does not have testbeds for           to the research community. Another        To what extent can we test
hardware/firmware-based malware.               desirable resource would be a shared      real systems?
                                               honeynet, which would allow learning      It is possible to test defenses for efficacy
The tools and infrastructure required to       malware behavior. Current honeynets       on real systems. Experiments can be
adequately harden a test environment           are run mostly on an ad hoc basis by      conceived in which real and emulation
are research problems in their own right.      individual groups. Legal and regula-      networks are exposed to public networks,
Testbeds to study malware are specific         tory issues inhibit meaningful sharing,   with and without particular defenses.
to this application. The testbed should        however.                                  However, rapid automated configura-
not be discernible as a test environment,                                                tion and propagation of defenses must
even to sophisticated malware.                 Internet-scale emulation would permit     first be thoroughly demonstrated on
                                               realistic testing of defenses and their   emulated systems.
The community requires an up-to-date,          dynamic interaction with malware out-
reliably curated malware repository for        breaks. Observation at this level would
research purposes. Limited repositories        provide a view of worm and botnet
exist at present, but they are not available   spread and operation never seen before.

References

[Ant2008]          A.M. Antonopoulos. Georgia cyberwar overblown. Network World, August 19, 2008
                   (http://www.pcworld.com/businesscenter/article/150021/georgia_cyberwar_overblown.html).

[CAT2009]          Conference for Homeland Security 2009 (CATCH ’09), Cybersecurity Applications and Technology,
                   March 3–4, 2009. The IEEE proceedings of this conference include relevant papers on detection and
                   mitigation of botnets, as well as correlation and collaboration in cross-domain attacks, from the University
                   of Michigan and Georgia Tech, as well as Endeavor, HBGary, Milcord, and Sonalyst (among others).

[Dai2006]          Dino Dai Zovi, Vitriol: Hardware virtualization rootkits. In Proceedings of the Black Hat
                   USA Conference, 2006.

[DET]              Cyber-DEfense Technology Experimental Research laboratory Testbed (DETERlab)
                   (http://www.isi.edu/deter/).

[Fra2007]          J. Franklin, V. Paxson, A. Perrig, and S. Savage. An inquiry into the nature and
                   causes of the wealth of Internet miscreants. Proceedings of ACM Computer and
                   Communications Security Conference, pp. 375-388, October 2007.

[GAO2007]          CYBERCRIME: Public and Private Entities Face Challenges in Addressing Cyber Threats. Report
                   GAO-07705, U.S. Government Accountability Office, Washington, D.C., July 2007.

[Hal2006]          J.A. Halderman and E.W. Felten. Lessons from the Sony CD DRM episode. In
                   Proceedings of the 15th USENIX Security Symposium, August 2006.




48
       COMBATTING MALWARE AND BOTNETS
[Hol2008]    T. Holz, C. Gorecki, K. Rieck, and F. Freiling. In Proceedings of the 15th Annual
             Network & Distributed System Security (NDSS) Symposium, February 2008.

[Kim2004]    Hyang-Ah Kim and Brad Karp, Autograph: Toward automated, distributed worm signature
             detection, In Proceedings of the 13th USENIX Security Symposium, August 2004.

[IW2007]     L. Greenemeier. Estonian attacks raise concern over cyber ‘nuclear winter.’ Information Week, May 24,
             2007 (http://www.informationweek.com/news/internet/showArticle.jhtml?articleID=199701774).

[LAT2008]    J.E. Barnes. Cyber-attack on Defense Department computers raises concerns. Los
             Angeles Times, November 28, 2008 (http://www.latimes.com/news/nationworld/
             iraq/complete/la-na-cyberattack28-2008nov28,0,230046.story).

[Mes2003]    Ellen Messmer. Welchia Worm Nails Navy Marine Corps, Network World Fusion, August 19,
             2003. (http://pcworld.com/article/112090/welchia_worm_nails_navy_marine_corps.html).

[Pou2003]    Kevin Poulsen. Slammer worm crashed Ohio nuke plant network. SecurityFocus, August 19, 2003
             (http://www.securityfocus.com/news/6767).

[Sha2004]    H. Shacham, M. Page, B. Pfaff, E.-J. Goh, N. Modadugu, and D. Boneh. On the
             effectiveness of address-space randomization. In Proceedings of the 11th ACM Computer
             and Communications Security Conference, Washington, D.C., pp. 298-307, 2004.

[Sha2008]    M. Sharif, V. Yegneswaran, H. Saidi, P. Porras, and W. Lee. Eureka: A framework for
             enabling static malware analysis. In Proceedings of the 13th European Symposium on Research
             in Computer Security (ESORICS), Malaga, Spain, pp. 481-500, October 2008.

[Sch2005]    Bruce Schneier. Real story of the rogue rootkit. Wired, November 17, 2005 (http://
             www.wired.com/politics/security/commentary/securitymatters/2005/11/69601).

[SRI2009]    SRI Cyber-Threat Analytics (http://www.cyber-ta.org/) and Malware Threat
             Center (http://mtc.sri.com). For example, see analyses of Conficker.

[Thu2008]    R. Thurston. Coffee drinkers in peril after espresso overspill attack. SC Magazine, June 20, 2008 (http://
             www.scmagazineuk.com/coffee-drinkers-in-peril-after-espresso-overspill-attack/article/111458).

[Vir]        Virus Total (http://www.virus-total.com).

[Vra+2005]   M. Vrable, J. Ma, J. Chen, D. Moore, E. Vandekieft, A. Snoeren, G. Voelker, and S.
             Savage. Scalability, fidelity and containment in the Potemkin virtual honeyfarm. ACM
             SIGOPS Operating Systems Review, 39(5):148-162, December 2005 (SOSP ’05).




                                                               COMBATTING MALWARE AND BOTNETS                         49
Current Hard Problems in INFOSEC Research
              6. Global-Scale Identity Management
              BACKGROUND

              What is the problem being addressed?
              Global-scale identity management concerns identifying and authenticating entities
              such as people, hardware devices, distributed sensors and actuators, and software
              applications when accessing critical information technology (IT) systems from
              anywhere. The term global-scale is intended to emphasize the pervasive nature
              of identities and implies the existence of identities in federated systems that may
              be beyond the control of any single organization. This does not imply universal
              access or a single identity for all purposes, which would be inherently dangerous.
              In this context, global-scale identity management encompasses the establishment
              of identities, management of credentials, oversight and accountability, scalable
              revocation, establishment and enforcement of relevant policies, and resolution of
              potential conflicts. To whatever extent it can be automated, it must be administra-
              tively manageable and psychologically acceptable to users. It must, of course, also
              be embedded in trustworthy systems and be integrally related to authentication
              mechanisms and authorization systems, such as access controls. It also necessarily
              involves the trustworthy binding of identities and credentials. It is much broader
              than just identifying known individuals. It must scale to enormous numbers of
              users, computer systems, hardware platforms and components, computer programs
              and processes, and other entities.

              Global-scale identity management is aimed specifically at government and com-
              mercial organizations with diverse interorganizational relationships that today are
              hampered by the lack of trustworthy credentials for accessing shared resources. In
              such environments, credentials tend to proliferate in unmanageable ways. Identity
              management within single organizations can benefit from—and needs to be com-
              patible with—the global-scale problem.

              Our concern here is mainly the IT-oriented aspects of the broad problems of
              identity and credential management, including authentication, authorization, and
              accountability. However, we recognize that there will be many trade-offs and privacy
              implications that will affect identity management. In particular, global-scale identity
              management may require not only advances in technology, but also open standards,
              social norms, legal frameworks, and policies for the creation, use, maintenance,
              and audit of identities and privilege information (e.g., rights or authorizations).
              Clearly, managing and coordinating people and other entities on a global scale
              also raises many issues relating to international laws and regulations that must be
              considered. In addition, the question of when identifying information must be
              provided is fundamentally a policy question that can and should be considered. In
              all likelihood, any acceptable concept of global identity management will need to
              incorporate policies governing release of identifying information. Overall, countless
              critical systems and services require authenticated authorization for access and use,

50
and global-scale identity management          fronts by a wide range of potential          of integrity, confidentiality, and system
will be a critical enabler of future IT       attackers with diverse motivations,          survivability, as well as denial-of-service
capabilities. Furthermore, it is essential    within large-scale organizations and         attacks.
to be able to authorize on the basis of       across multiple organizations. Insider
attributes other than merely supposed         and outsider misuses are commonplace.        Threats described in other topic areas
identities. Identity management needs         Because of the lack of adequate iden-        can also affect global-scale identity
to be fully integrated with all the systems   tity management, it is often extremely       management, most notably defects in
into which it is embedded.                    difficult to identify the misusers. For      trustworthy scalable systems. In addi-
                                              example, phishing attacks have become        tion, defects in global-scale identity
Identity management systems must              a pervasive problem for which identify-      management can have negative impacts
enable a suite of capabilities. These         ing the sources and the legitimacy of the    on provenance and attack attribution.
include control and management of cre-        phishers and rendering them ineffective
dentials used to authenticate one entity      where possible are obvious needs.            Who are the potential
to another, and authorization of an
                                                                                           beneficiaries? What are their
entity to adopt a specific role and assert    Identity-related threats exist throughout
                                                                                           respective needs?
properties, characteristics, or attributes    the development cycle and the global
of entities performing in a role. Global-     supply chain, but the runtime threats        Governmental agencies, corporations,
scale identity management must also           are generally predominant. Misuse of         institutions, individuals, and particu-
support nonrepudiation mechanisms             identities by people and misuse of flawed    larly the financial communities [FSSCC
and policies; dynamic management of           authentication by remote sites and com-      2008] would benefit enormously from
identities, roles, and properties; and        promised computers (e.g., zombies) are       the existence of pervasive approaches
revocation of properties, roles, and iden-    common. The Internet itself is a source      to global identity management, with
tity credentials. Identity management         of numerous collateral threats, including    greater convenience, reduction of
systems must provide mechanisms for           coordinated, widespread denial-of-ser-       administrative costs, and possibili-
two-way assertions and authentica-            vice attacks, such as repeated failed        ties for better oversight. Users could
tion handshakes building mutual trust         logins that result in disabling access by    benefit from the decreased likelihood of
among mutually suspicious parties.            legitimate users. Various threats arise      impersonation, identity and credential
All the identities and associated asser-      when single-sign-on authentication           fraud, and untraceable misuse. Although
tions and credentials must be machine         of identities occurs across boundaries       the needs of different individuals and
and human understandable, so that all         of comparable trustworthiness. This          different organizations might differ
parties are aware of the identity interac-    is likely to be a significant concern in     somewhat, significant research in this
tions and relationships between them          highly distributed, widespread system        area would have widespread benefits for
(e.g., what these credentials are, who        environments. Additional threats arise       all of them.
issued them, who has used them, and           with respect to the misuse of identities
who has seen them). The lifetimes of          and authentication, especially in the        What is the current state of
credentials may exceed human lifetimes        presence of systems that are not ade-
                                                                                           the practice?
in some cases, which implies that pre-        quately trustworthy. Even where systems
vention of and recovery from losses are       have the potential for distinguishing        There are many current approaches to
particularly difficult problems.              among different roles associated with        identity management. Many of these
                                              different individuals and where fine-        are not yet fully interoperable with
What are the potential                        grained access controls can be used,         other required services, not scalable,
                                              operational considerations and inade-        only single-use, or limited in other
threats?
                                              quate user awareness can tend to subvert     ways. They do, however, collectively
Identification and authentication (I&A)       the intended controls. In particular,        exhibit pointwise examples that can lead
systems are being attacked on many            threats are frequently aimed at violations   toward enabling a global-scale identity

                                                                     GLOBAL-SCALE IDENTITY MANAGEMENT                              51
management framework. Examples               could potentially be useful as part of       management, including a government-
of existing approaches include the           the authentication process, but most         wide E-Authentication initiative, the
following:                                   biometric technologies currently have        Defense Department’s Common Access
                                             various potential implementation vul-        Card, and public key infrastructure for
     ƒƒ Personal ID and authentication.      nerabilities, such as fingerprint readers    the Global Information Grid. These
       Shibboleth is a standards-based,      being fooled by fake gelatin fingers.        are not research directions, but exhibit
       open-source software system for       Credit cards, debit cards, smart cards,      many problems that can motivate future
       single sign-on across multiple        user-card-system authentication, and         research. However, none of these can
       websites. (See http://shibboleth.     chip and PIN have all experienced some       scale to the levels required without sub-
       internet2.edu.) Also of interest      vulnerabilities and various misuses.         stantial problems regarding federation
       are Card Space, Liberty Alliance,     Per-message techniques such as DKIM          of certification authorities and delays in
       SAML, and InCommon (all of            (DomainKeys Identified Mail), authen-        handling revoked privileges. Moreover,
       which are federated approaches,       ticating e-mail messages, PGP, and S/        although it is perhaps a minor consid-
       in active use, undergoing further     MIME are also worth considering—             eration today, the existing standard and
                                             especially for their limitations and         implementations are based on public-
       development, and evolving in
                                             development histories.                       key cryptography that could eventually
       the face of various problems with
                                                                                          be susceptible to attack by quantum
       security, privacy, and usability).
                                             It is desirable to learn from the relative   computers.
     ƒƒ The Homeland Security                shortcomings of all these approaches
       Presidential Directive 12             and any experience that might be gained      Considerable research exists in policy
       (HSPD-12) calls for a common          from their deployment. However, for          languages, trust negotiation, and cer-
       identification standard for federal   the most part, these sundry existing         tificate infrastructures that have not
       employees and contractors. An         identity management concepts do not          yet been tried in practice. Research
       example of a solution in compliance   connect well with each other. Forming        strategies to achieve a strong I&A archi-
                                             appropriate and effective, semantically      tecture for the future include large-scale
       with HSPD-12 is the DoD
                                             meaningful connections between dis-          symmetric key infrastructures with key
       Common Access Card (CAC).
                                             parate identity management systems           distribution a priori, federated systems
                                             presents a significant challenge. Given      of brokers to enable such a system to
Various other approaches such as the         a future with many competing and             scale, strategies for scaling symmetric
following could play a role but are not      cooperating identity management              creation of one-time pads, schemes of
by themselves global-scale identity          systems, we must develop a system of         cryptography not reliant on a random
solutions. Nevertheless, they might          assurance for the exchange of identity       oracle, and other schemes of cryp-
be usefully considered. Open ID pro-         credentials across identity manage-          tography not susceptible to attack by
vides transitive authentication, but only    ment systems, and principled means           quantum computers (which seems pos-
minimal identification; however, trust is    to combine information from multiple         sible, for example, with lattice-based
inherently not transitive, and malicious     identity management systems as input         cryptography). The series of IDtrust
misuse is not addressed. Medical ID          to policy-driven authorization deci-         symposia at NIST summarize much
is intended to be HIPAA compliant.           sions. The threats noted above are poorly    work over the past 9 years [IDT2009],
Enterprise Physical Access is represen-      addressed today.                             including three papers from the 2009
tative of token-based or identity-based                                                   symposium from an ongoing collabora-
physical access control systems. Stateless   What is the status of current                tive I3P project on identity management.
identity and authentication approaches                                                    On the other hand, relatively little work
                                             research?
include LPWA, the Lucent Personal-                                                        has been done on avoiding monolithic
ized Web Assistant. OTP/VeriSign is          Currently, there are several major ini-      trusted roots, apart from systems such
a symmetric key scheme. Biometrics           tiatives involving large-scale identity      as Trusted Xenix. There is also not

52      GLOBAL-SCALE IDENTITY MANAGEMENT
                                            ƒƒ Federated bilateral user identity          accountability, credential
enough effort devoted to trustworthy
                                              and credential management on                renewals, problems that result
bindings between credentials and users.
Biometrics and radio frequency identi-        a very large scale, to facilitate           from system updates, and so on.
fication (RFID) tags both require such        interoperability among existing
                                                                                        ƒƒ Identity management for
binding. However, by no means should          systems.
                                                                                          nonhuman entities such as
research on potential future approaches     ƒƒ Efficient support for management           domain names, routers, routes,
be limited to these initial ideas.            of identities of objects, processes,        autonomous systems, networks,
                                              and transactions on a very large            and sensors.
FUTURE DIRECTIONS                             scale.
                                                                                     Note that merely making SSL client cer-
                                            ƒƒ Flexible management of identities     tificates work effectively in a usable way
On what categories can we                     (including granularity, aliases,       might be a useful initial step forward.
subdivide the topic?                          proxies, groups, and associated
Two categories seem appropriate for this      attributes).                           Policies for enhancing global identity
topic area, although some of the sug-       ƒƒ Support for multiple privacy and
                                                                                     management (some of which have
gested research areas may require aspects                                            mechanism implications) include the
                                              cross-organization information
of both categories:                                                                  following.
                                              exposure requirements,
  ƒƒ Mechanisms (e.g., for                    lightweight aliasing, and                 ƒƒ Risk management across a
     authentication, attribution,             unlinking.                                  spectrum of risks. This is tightly
     accountability, revocation,                                                          coupled with authorization.
                                            ƒƒ Effective presentation of specific
     federation, usable user interfaces,                                                  Game-theoretical analyses might
                                              attributes: multiple roles,
     user-accessible conceptual                                                           be useful.
                                              multiple properties, effective
     models, presentation, and
                                              access rights, transparency of            ƒƒ Trust or confidence in the
     evaluations thereof ).
                                              what has and has not been                   interactions (untrustworthy third
  ƒƒ Policy-related research (e.g.,           revealed.                                   parties; what happens when your
     privacy, administration,                                                             credentials get stolen or the third
     revocation policies, international     ƒƒ Enabling rapidly evolving and
                                                                                          party disappears).
     implications, economic, social           newly created attributes, such as
                                              value associated with identifiers.        ƒƒ User acceptance: usability,
     and cultural mores, and policies
                                                                                          interoperability, costs; fine-
     relating to the effective use of the
                                            ƒƒ Timely revocation of credentials           grained attribute release and
     above mechanisms)
                                              (altering or withdrawing                    presentation to users.
                                              attributes).
As is the case for the other topics, the                                                ƒƒ Explicating the structure,
term “research” is used here to encom-      ƒƒ Avoidance of having to carry too           meaning, and use of attributes:
pass the full spectrum of R&D, test,          many certificates versus the risks          semantics of identity and
evaluation, and technology transfer.          of single-sign-on authentication            attribute assertions.
Legal, law enforcement, political,            that must be trustworthy despite          ƒƒ Commercial success and
international, and cultural issues are        traversing untrustworthy systems.           acceptance: usability,
cross-cutting for both of these bins and                                                  interoperability, costs, sustainable
need to be addressed throughout.            ƒƒ Long-term implications
                                              of cryptographically based                  economic models; presentation
                                              approaches, with respect                    to users.
Mechanisms for enhancing global iden-
tity management (with some policy             to integrity, spoofability,               ƒƒ Accommodating international
implications) include the following:          revocation when compromised,                implications that require special


                                                                GLOBAL-SCALE IDENTITY MANAGEMENT                            53
       consideration, such as seemingly       must be able to present credentials for              hardware, individual packets,
       fundamental differences in             identities, roles, and attributes—inde-              messages, and so on.
       privacy policies among different       pendently but consistently interrelated,
       EU nations, the United States,                                                            ƒƒ Containment, detection, and
                                              relative to specific needs. For example,
       and the rest of the world.                                                                  remediation are poorly addressed,
                                              why should a liquor store clerk be able
                                                                                                   particularly following misuse of
     ƒƒ Compensating for possible             to view a person’s address and other
                                                                                                   identities, authentication, and
       implications of new approaches         personal details on a driver’s license               authorization.
       that enable new types of               when determining whether that person
       transactions and secondary uses        is at least 21, or, worse yet, to swipe            ƒƒ Maintaining consistency of
       that were not initially anticipated.   a card with unknown consequences?                    reputations over time across
                                              Services should be able to validate role             identities is extremely difficult.
     ƒƒ Understanding the implications                                                             However, carefully controlled
                                              or property credentials for some situa-
       of quantum computing and                                                                    mechanisms to revoke or
                                              tions without requiring explicit identity
       quantum cryptography, and                                                                   otherwise express doubts about
                                              as well. Entities and services must also
       exploring the possibilities of                                                              such reputations are also needed.
                                              be able to select appropriate levels of
       global identity management
                                              confidence and assurance to fit their              ƒƒ Past efforts to impose
       without public-key cryptography
                                              situation. In addition, secondary reuse              national standards for identity
       or with quantum-resistant public-
                                              of credentials by authorizing entities               management have met
       key cryptography.                      must be effectively prevented. Some                  considerable resistance (as
Table 6.1 provides an oversimplified          sort of mutual authentication should                 in Australia and the United
summary of the two categories.                be possible whenever desirable. That is,             Kingdom).
                                              a bidirectional trusted path between the
                                              authenticatee and the authenticator may            ƒƒ There is a serious lack of
What are the major research                                                                        economic models that would
                                              be needed in some cases.
gaps?                                                                                              underscore the importance of
                                                                                                   global-scale identity management
A key gap in identity management is           Major gaps include the following:
                                                                                                   and lead to coherent approaches.
the lack of transparent, fine-grained,
strongly typed control of identities,            ƒƒ Existing systems tend to
                                                                                                 ƒƒ There is also a serious lack of
roles, attributes, and credentials. Enti-          authenticate only would-                        understanding of cultural and
ties must be able to know and control              be identities of users, not                     social implications of identity,
what identity-related information has              transactions, applications,                     management authentication, and
been provided on their behalf. Entities            systems, communication paths,                   privacy among most citizens.


                                      TABLE 6.1: Some Illustrative Approaches

Category                                      Definition                                     PotentialƒApproaches
Mechanisms                                    Identity- and attribute-based systems          Globally trustworthy identities,
                                              implementing authentication,                   cryptographic and biometric authentication,
                                              authorization, accountability                  secure bindings to entities, distributed
                                                                                             integrity
Policies                                      Rules and procedures for enforcing identity-   Broadly based adversary detection systems
                                              based controls, using relevant mechanisms      that integrate misuse detection, network
                                                                                             monitoring, distributed management




54      GLOBAL-SCALE IDENTITY MANAGEMENT
Achieving the goal of open, globally         systems in which trustworthiness can         Some of the possibly relevant
accepted standards for identifying           not be assured.                              metrics might involve the following
individuals, system components, and                                                       considerations:
processes is difficult and will take con-    Measures of success                            ƒƒ Interoperability. How many
siderable coordination and cooperation
between industry and governments.            Ideally, any system for identification,           systems might be integrated?
Global-scale identity management is a        authentication, and access control                What efficiency can result as
hard problem for a number of reasons,        should be able to support hundreds of             scopes of scalability increase?
including standardization, scale, churn,     millions of users with identity-based or       ƒƒ Bilateral identity management.
time criticality, mitigation of insider      role-based authentication. IDs, authen-           How many identities might be
threats, and the prospect of threats         tication, and authorization of privileges         handled? What are the risks?
such as quantum computing to existing        may sometimes be considered separately,        ƒƒ Efficiency of identity transactions
cryptographic underpinnings. Main-           but in any case must be considered                at global scale. For example,
taining the anonymity of personal            compatibly within a common context.               what is the end-to-end minimum
information unless explicitly required       An identifier declares who a person is            time to process various types of
is another challenge. In addition, deter-    and may have various levels of granu-             transactions?
mining how system processes or threads       larity and specificity. Who that person
                                                                                            ƒƒ Revocation. What are the time
should be identified and privileged is       is (along with the applicable roles and
                                                                                               delays for expected propagation
an even more complex and daunting            other attributes, such as physical loca-
                                                                                               as the global scale increases?
undertaking. Part of the challenge is to     tion) will determine the privileges to be
distinguish between the user and the         granted with respect to any particular         ƒƒ Value metrics. What are the
subjects executing on his or her behalf.     system policy. The system should be               short-term and long-term values
Finally, although sensor networks and        able to handle millions of privileges             that might result from various
radio frequency identification (RFID)        and a heavy churn rate of changes in              approaches?
have tremendous utility, their current       users, devices, roles, and privileges. In      ƒƒ Privacy metrics. For example,
vulnerabilities and the desired scale of     addition, each user may have dozens of            how easily can behavior analysis
future deployment underscore the need        distinct credentials across multiple orga-        or pseudonymous profiling be
to address the hard challenges of identity   nizations, with each credential having its        used to link multiple identities?
management on a global scale.                own set of privileges. It should be pos-
                                                                                            ƒƒ Risk management metrics. What
                                             sible to measure or estimate the extent
                                                                                               are the risks associated with the
Resources                                    to which incremental deployment of
                                                                                               above items?
                                             new mechanisms and new policies could
Short-term gains can be made, par-           be implemented and enforced. Revoca-
ticularly in prototypes and in the policy    tion of privileges should be effective for   What needs to be in place for
research items noted in the Background       near-real-time use. Measurable metrics       test and evaluation?
section above. In particular, the intel-     need to encompass all these aspects of
ligent use of existing techniques and        global identity management. Overall,         Federated solutions will require realis-
implementations would help. However,         it should be extremely difficult for any     tic testbeds for test and evaluation of
serious effort needs to be devoted to        national-level adversary to spoof a criti-   global identity management approaches.
long-term approaches that address            cal infrastructure system into believing     Universities would provide natural envi-
inherent scalability, trustworthiness,       that anyone attempting access is any-        ronments for initial experimentation and
and resistance to cryptanalytic and sys-     thing other than the actual adversary or     might, under controlled circumstances,
temic attacks, particularly in federated     adversaries.                                 enable larger-scale collaborations.




                                                                    GLOBAL-SCALE IDENTITY MANAGEMENT                             55
Numerous opportunities will exist for      organizational and multi-organizational      Approaches to test markets require spe-
formal analysis of algorithms and pro-     requirements, and the number of orga-        cific attention to usefulness and usability
totypes, especially as they scale up to    nizations, not just the number of people.    and to cost-effectiveness. Possible test
federated solutions. These should com-     Testing is only part of what is neces-       markets include virtual environments
plement any testing.                       sary. Federated algorithms need some         such as World of Warcraft or Second
                                           formal analyses with respect to their        Life and real-world environments such
To what extent can we test                 consistency, security, and reliability.      as banking, financial services, eBay, the
                                           Experiences with failed or ineffective       Department of Energy, Department of
real systems?
                                           attempts in the past must be reflected       Veterans Affairs, federated hospitals,
Today’s test and evaluation are rather     in new directions. As is often the case,     and Las Vegas casinos. Realistic test-
ad hoc and leave beta testing to user      sharing of such experiences is difficult.    beds require realistic incentives such
communities. Test criteria, scalability,   So are multi-institutional testbeds and      as minimizing losses, ability to cope
robustness, and cost need to be con-       experiments. Incentives are needed to        with large-scale uses, ease of evaluation,
sidered. Some things can be tested;        facilitate sharing of experiences relating   and trustworthiness of the resulting
others require different kinds of analy-   to vulnerabilities and exploits. Algorith-   systems—including resilience to denials
sis, including large-scale simulations     mic transparency is needed, rather than      of service and other attacks, overall
and formal methods. Scalability is         closely held proprietary solutions.          system survivability, and so on.
needed with respect to the number of


References
[FSS2008]        Financial Services Sector Coordinating Council for Critical Infrastructure. Protection and Homeland
                 Security, Research and Development Committee. Research Agenda for the Banking and Finance
                 Sector. September 2008, (https://www.fsscc.org/fsscc/reports/2008/RD_Agenda-FINAL.pdf ).

[IDT2009]        8th Symposium on Identity and Trust on the Internet (IDtrust 2009), NIST, April 14-16, 2009
                 (http://middleware.internet2.edu/idtrust). The website contains proceedings of previous
                 years’ conferences. The 2009 proceedings include three papers representing team members
                 from the I3P Identity Management project (which includes MITRE, Cornell, Georgia
                 Tech, Purdue, SRI, and the University of Illinois at Urbana-Champaign).




56    GLOBAL-SCALE IDENTITY MANAGEMENT
Current Hard Problems in INFOSEC Research
             7. Survivability of Time-Critical Systems
             BACKGROUND

             What is the problem being addressed?
             Survivability is the capability of a system to fulfill its mission, in a timely manner,
             in the presence of attacks, failures, or accidents [Avi+1994, Ell+1999, Neu2000].
             It is one of the attributes that must be considered under trustworthiness, and is
             meaningful in practice only with respect to well-defined mission requirements
             against which the trustworthiness of survivability can be evaluated and measured.

             Time-critical systems, generally speaking, are systems that require response on non-
             human timescales to maintain survivability (i.e., continue to operate acceptably)
             under relevant adversities. In these systems, human response is generally infeasible
             because a combination of the complexity of the required analysis, the unavailabil-
             ity and infeasibility of system administrators in real time, and the associated time
             constraints. This section uses the following definition:

             With respect to survivability, a time-critical system is a system for which faster-
             than-human reaction is required to avoid adverse mission consequences and/or
             system instability in the presence of attacks, failures, or accidents.

             Of particular interest here are systems for which impaired survivability would have
             large-scale consequences, particularly in terms of the number of people affected.
             Examples of such systems include electric power grids and other critical infrastruc-
             ture systems, regional transportation systems, large enterprise transaction systems,
             and Internet infrastructure such as routing or DNS. Although impaired survivability
             for some other types of systems may have severe consequences for small numbers of
             users, they are not of primary relevance to this topic. Examples of such systems are
             medical devices, individual transportation systems, home desktop computers, and
             isolated embedded systems. Such systems are not always designed for an adequate
             level of survivability, but the problem is less challenging to address for them than for
             large and distributed systems. However, common-mode failures of large numbers
             of small systems (for example, a vulnerability in a common type of medical device)
             could have large-scale consequences. (Note that personal systems are not actually
             ignored here, in that certain major advances in survivability of large-scale time-
             critical systems may be applicable to smaller systems.)

             Time criticality is a central property to be considered. It connects directly to
             the “faster-than-human” aspect of the above definition of survivability. In some
             systems, failure to fulfill a mission for even fractions of a second could have severe
             consequences. In other types of systems, downtime for several minutes could be
             acceptable. In some other systems, system stability could be threatened if upsets
             are not handled on faster-than-human timescales. See Figure 7.1 for examples


                                                                                                  57
        Figure 7.1: Examples of Systems With Different Time-Criticality Requirements and
                                    Different User Populations
Time-criticality

              Secondary relevance            Primary relevance


             Pacemaker



                             Avionics
                                                                    Enterprise/sector
                                                                     critical transac-
                                                                       tion system

                                 Ad-hoc emergency
                                  response system




                                                  Corporate
                                                 office server
                   Home PC

                                                                                 Social networking website


                                                      Batch processing system




of systems categorized with respect to      failures, and accidents. Rather than enu-    educators and students, standards
relative time criticality and size of the   merate a long list, we refer throughout      bodies, and so on. These categories of
user population they serve. The systems     to “all relevant adversities” for which      beneficiaries have very different needs.
on the right side of the diagonal line      survivability is required.                   End users need to have a working system
are considered in primary scope for this                                                 whenever they need to use it (avail-
discussion, while systems to the left of    Who are the potential                        ability), and they need the system to
the line are of secondary interest, as                                                   continue working correctly once they
                                            beneficiaries? What are their
indirect beneficiaries.                                                                  have started using it (reliability). System
                                            respective needs?
                                                                                         owners have many additional needs;
What are the potential                      Beneficiaries include the ultimate end       for example, they need to have situ-
                                            users of critical infrastructure systems     ational awareness so that they can be
threats?
                                            (the public), system owners and opera-       warned about potential problems in
As noted in the definition of survivabil-   tors, system developers and vendors,         the system and manage system load,
ity, the threats include system attacks,    regulators and other government bodies,      and they need to be able to react to an




 58   SURVIVABILITY OF TIME-CRITICAL SYSTEMS
incident and to recover the system and       systems, we cannot afford to wait for        control systems is also under way in the
restore operations.                          such data to be gathered and analyzed.       I3P program (www.thei3p.org/research/
                                                                                          srpcs.html). However, considerable
                                             What is the status of current                effort is needed to extend fault toler-
What is the current state of                                                              ance concepts to survivability (including
                                             research?
practice?                                                                                 intrusion tolerance) and to pursue auto-
                                             The current state of research can be         mated and coordinated attack response
At present, IT systems attempt to maxi-      partitioned into three areas: understand-    and recovery.
mize survivability through replication       ing the mission and risks; survivability
of components, redundancy of infor-          architectures, methods, and tools; and       Test and Evaluation. We need to be able
mation (e.g., error-correcting coding),      test and evaluation.                         to test and evaluate the time-critical ele-
smart load sharing, journaling and trans-                                                 ments of systems. Some testbed efforts
action replay, automated recovery to a       Understanding the Mission and Risks.         have made general network testing
stable state, deferred committing for        We need to better understand the time-       infrastructures available to researchers
configuration changes, and manually          critical nature of our systems and           (for example PlanetLab, ORBIT, and
maintained filters to block repeated         their missions. We also need to better       DETER). Some other existing testbeds
bad requests. Toward the same goal,          understand the risks to our systems          are available only to restricted groups,
control systems today are supposedly         with respect to impaired survivability.      such as military or other government
disconnected from external networks          The concept of risk typically includes       research laboratories. However, testing
(especially when attacks are suspected),     threats, vulnerabilities, and conse-         of survivability is inherently unsatis-
although not consistently. Embedded          quences. (Experiences with the design        factory, because of the wide variety of
systems typically have no real protection    and operation of critical infrastructure     adversities and attacks, some of which
for survivability from malicious attacks     systems would be helpful toward these        may arise despite being highly improb-
(apart from some physical security),         goals.) Some methodologies and tools         able. In addition, testbeds tend to lack
even when external connections exist.        exist in this area, but many risk analysis   realism.
                                             methods are imprecise and suffer from
The current metrics for survivability,       limited data for one or several param-
                                                                                          FUTURE DIRECTIONS
availability, and reliability of time-       eters. However, the recent efforts by
critical systems are based on the            Haimes et al. and Kertzner et al. are        On what categories can we
probabilities of natural and random          worth noting [Hai+2007, Ker+2008].           subdivide the topics?
failures (e.g., MTBF). These metrics
typically ignore intentional attacks,        Survivability Architectures, Methods,        This topic is divided into three cat-
cascading failures, and other correlated     and Tools. Efforts in this area include      egories, as suggested in the preceding
causes or effects. For example, coordi-      the large body of work in fault toler-       section: understanding the mission
nated attacks and insider attacks are not    ance for systems and networks (e.g.,  see    and risks; survivability architectures,
addressed in most current approaches         [Neu2000] for many references). A            methods, and tools; and test and
to survivability. One often-cited reason     previous major R&D program in this           evaluation.
is that we do not have many real-world       area was DARPA’s OASIS (Organically
examples of intentional well-planned         Assured and Survivable Information           Survivability architectures, methods,
attacks against time-critical systems.       Systems), documented in the Third            and tools are further divided into
However, because of the criticality of the   DARPA Information Survivability Con-         protect, detect, and react subcatego-
systems considered here and because of       ference and Exhibition [DIS2003].            ries. Table 7.1 provides a summary of
many confirmed vulnerabilities in such       Some work in the area of survivable          the potential approaches.




                                                                SURVIVABILITY OF TIME-CRITICAL SYSTEMS                            59
What are the major research                          sets of requirements will apply                ƒƒ There is no one-size-fits-all
gaps?                                                to specific systems. We need                      architecture. Some systems will
                                                     processes and methods to identify                 be embedded and centralized;
As an attribute of trustworthiness,
survivability depends on trustworthy                 and locate time criticality in                    some will be networked
computer systems and communications                  systems and to express them in                    and distributed. However,
and trustworthy operations relating                  a rigorous manner. Similarly,                     composable, scalable trustworthy
to security, reliability, real-time per-             we need to be able to identify                    systems (Section 1) are likely to
formance where essential, and much                   and quantify consequences,                        play a major role.
more. Thus, it is in essence a meta-                 which could be life-critical,
requirement. Its dependence on other                 environmental, or financial. The            Survivability Architectures, Methods,
subrequirements must be made explicit.               interaction between physical                and Tools
(For example, see [Neu2000].) The                    and digital systems needs to be             Protect (protection that does not involve
absence of meaningful requirements                   understood with greater fidelity.
for survivability is a serious gap in prac-       ƒƒ Interdependencies among systems             human interaction)
tice and is reflected in various gaps in             and infrastructures need to be
research—for example, the inability to                                                              ƒƒ We need families of architectures
                                                     analyzed. We need to understand                   with scalable and composable
specify requirements in adequate detail
                                                     the extent to which a survivability               components that can satisfy
and completeness, and the inability
                                                     failure in one system can cause                   critical trustworthiness
to determine whether specifications
                                                     a failure in another system, and                  requirements for real-time system
and systems actually satisfy those
                                                     the ways in which survivability                   behavior. We need to understand
requirements.
                                                     properties can compose.                           how to balance confidentiality
Understanding the Mission and Risks               ƒƒ We need to be able to build                       and integrity against timely
  ƒƒ Rigorous definitions of properties              models of systems, threats,                       availability. Traditional security
     and requirements are needed                     vulnerabilities, and attack                       mechanisms tend to either
     that can apply in a wide range                  methods. These models should                      introduce human timescales
     of application environments.                    include evolution of attacks and                  or latency on a machine
     These include concepts such                     blended threats that combine                      timescale and could thereby
     as response time, outage time,                  independent and correlated                        impair availability. Techniques
     and recovery time. Specific                     attack methods.                                   for protecting integrity could


                                              TABLE 7.1: Potential Approaches

 Category                                      Definition                                       PotentialƒApproaches
 Protect                                       Protect systems from all relevant adversities    Inherently survivable system architectures
                                               in the system’s environment.                     with pervasive requirements.
 Detect                                        Detect potential failures and attacks as early   Broadly based adversity detection systems
                                               as possible.                                     that integrate misuse detection, network
                                                                                                monitoring, etc.
 React                                         Remediate detected adversities and recover       Use situational awareness and related
                                               as extensively as possible.                      diagnostics to assess damage; anticipate
                                                                                                potential recovery modes.




60       SURVIVABILITY OF TIME-CRITICAL SYSTEMS
  improve survivability, but not          ƒƒ We need to understand how             components into systems that are
  necessarily. Some integrity               core functions of systems can be       more survivable.
  protection mechanisms, such               isolated from functions that can     ƒƒ We need substantive methods
  as checksums, could introduce             be attacked, so that the time-         for composable survivability. See
  vulnerabilities if the checksums          critical properties of the core        Section 1 (Scalable Trustworthy
  could be manipulated or made              functions are preserved even           Systems) for a more detailed
  unavailable. Better techniques            when the systems are attacked.         discussion on composability.
  are needed to ensure self-                Research is needed on predictably      We need tools for reasoning
  monitoring and self-healing               trustworthy resource allocation        about composable survivability,
  system capabilities, as well              and scheduling applicable to           including assurances relating
  as autonomous operation.                  each of a wide range of different      to identity and provenance of
  Distributed systems must                  system architectures with              components (Sections 6 and 9,
  also be considered, not just              different types of distributed         respectively) and life cycle
  embedded systems. Trustworthy             control.                               evaluations (Section 3). For
  management (including control,          ƒƒ We need to explore how we can         example, survivability claims
  security, and integrity), timely          achieve useful redundancy, with        for a system composed of
  delivery of distributed data,             adequate assurance that single         components should be derivable
  and heterogeneous sensors                 points of failure are not present.     from survivability claims for
  will be particularly important.                                                  components. Developing and
                                          ƒƒ We must be able to identify
  Survivability also requires                                                      deploying generic building-
                                            and prevent the possibilities of
  protection against attacks, insider                                              block platforms for composable
                                            cascading failures. In particular,
  misuse, hardware faults, and                                                     survivability would be very
                                            we need mechanisms that detect
  other adversities. It may also                                                   useful.
                                            and stop cascading failures faster
  need to limit dependence on
                                            than they can propagate. This is     ƒƒ For networks, we need to explore
  untrustworthy components, such
                                            a complex problem that needs           the trade-offs between in-band
  as complex operating systems that
                                            large testbeds and new simulation      and out-of-band control with
  need frequent patches. Above all,                                                respect to survivability, time
                                            methodologies.
  operational interfaces to human                                                  criticality, and economics.
  controllers will be vital, especially   ƒƒ Common-mode failures are
  in emergency situations.                  a challenge in monocultures,         ƒƒ We must be able to ensure
                                            whereas system maintenance             survivability for services on
ƒƒ We need new communication
                                            is problematic in diversified          which our time-critical systems
  protocols that are designed for
                                            and heterogeneous systems.             depend. For example, all systems
  survivability. For example, a
                                            Techniques are needed                  depend on some form of power
  protocol could make sure that
                                            to determine appropriate               source, and the survivability
  an attacker needs to spend
                                            balances between diversity             of the system can never be
  more resources than the system
                                            and monoculture to achieve             better than the survivability
  needs to expend to defend itself
                                            survivability in time-critical         of its power sources. Other
  while preserving its time-critical
                                            systems.                               services to consider are cooling,
  properties. Frequency hopping
  and SYN cookies are examples of                                                  communications, DNS, and
                                          ƒƒ Considerable effort is being
  approaches using this principle.          devoted to developing                  GPS.
  Extending or replacing TCP/IP,            hypervisors and virtualization.      ƒƒ We need to investigate functional
  Modbus, and other protocols               Perhaps these approaches could         distribution as a strategy for
  might be considered.                      be applied to integrating COTS         time-critical survivability and

                                                         SURVIVABILITY OF TIME-CRITICAL SYSTEMS                      61
       consider challenges related to         is at risk, we need to react to make sure    important, but “depth” of hardening can
       that strategy. Issues to investigate   that survivability is preserved. The fol-    also be important, as can affordability—
       include the use of robust group        lowing approaches to reaction need to        an approach that is cost prohibitive will
       communication schemes—peer-            be investigated:                             not be very widely adopted.
       to-peer and multicast for time-
       critical systems.                         ƒƒ Self-healing systems that deploy       What R&D is evolutionary and
     ƒƒ Detection and recovery                     machine-time methods to restore         what is more basic, higher
       mechanisms themselves (see                  time-critical system properties.
                                                                                           risk, game changing?
       below) need to be protected,              ƒƒ Graceful degradation of service
       to make sure they cannot be                 (connection with mission                Near term
       disabled or tricked into reaction.                                                    ƒƒ Realistic, comprehensive
                                                   understanding requirements).
                                                                                                requirements
                                                 ƒƒ Predictable reactions with
Detect                                                                                       ƒƒ Existing protocols
                                                   appropriate timeliness.
                                                                                             ƒƒ Identification of time-critical
To detect when the survivability of a            ƒƒ Strategies for course of action             components
time-critical system is at risk, we need to        when intervention is required
have sophisticated and reliable detec-                                                     Medium term
                                                   (scenario planning before
tion methods. This capability requires                                                      ƒƒ Detection
                                                   reaction is needed, cyber
runtime methods to detect loss of                  playbook).                                ƒƒ Strategies for reaction
time-critical system properties, such                                                        ƒƒ Experimentation with
                                                 ƒƒ System change during operation
as degradation, and predict potential                                                           trustworthy protocols for
                                                   (to break adversarial planning, to
consequences. The following topics need                                                         networking and distributed
                                                   make planned attacks irrelevant).
investigation:                                                                                  control, out-of-band signaling,
                                                 ƒƒ Coordinating reaction with                  robustness, and emergency
     ƒƒ Self-diagnosis (heartbeats,                supporting services (e.g., tell ISP          recovery
       challenge-response, built-in                to reconfigure routing into user’s        ƒƒ Higher-speed
       monitoring of critical functions,           network, real-time black hole).              intercommunications and
       detection of process anomalies).                                                         coordination
                                                 ƒƒ Tarpitting, that is, slowing down
     ƒƒ Intrinsically auditable systems            an attacker without slowing               ƒƒ Development tools
       (systems that are by design                 down critical system functions.           ƒƒ System models
       instrumented for detection).              ƒƒ Bringing undamaged/repaired            Long term
     ƒƒ Network elements that                      components back online via                ƒƒ Evaluatable metrics
       participate and collaborate on              autonomous action (no human               ƒƒ Establishment of trustworthy
       detection.                                  intervention). This includes                 protocols for networking and
                                                   reevaluation of component                    distributed control
     ƒƒ Human-machine interfaces that
                                                   status and communication flows
       enable better detection and better                                                    ƒƒ Self-diagnosis and self-repair
                                                   (routing, ad-hoc networks).
       visualization.                                                                        ƒƒ Provisioning for automated
     ƒƒ Protocols that support closed-                                                          reaction and recovery
                                              What are the challenges that
       loop design (confirmation of
                                              must be addressed?
       actions).                                                                           Resources
                                              Significant advances in attacks on surviv-
React                                         ability may require research in new areas.   Making progress on the entire set
When we have detected that survivability      Breadth of service environments can be       of in-scope systems requires focused

62      SURVIVABILITY OF TIME-CRITICAL SYSTEMS
research efforts for each of the underly-           metrics). Resilience must be            ƒƒ Research infrastructures that are
ing technologies and each type of critical          possible in the face of unexpected        needed to support research in this
system, together with a research-coor-              inputs, when some partial degree          area include a “library” of devices:
dinating function that can discern and              of service must still be provided,        keep a copy of every reasonably
understand both the common and the                  with appropriate recovery time.           sized and priced manufactured
disparate types of solutions developed              Attack efforts in testing need to         device (compare this with seed
by those working on specific systems.               be appropriately high.                    banks). Also, keep templates
An important role for the coordinating
                                                  ƒƒ Measuring the relationships              or models of devices for use in
function is to expedite the flow of ideas
                                                    between complexity and time               design and evaluation.
and understanding among the focused
groups.                                             criticality is desired, especially      ƒƒ Access to real-world normal and
                                                    when a system requires faster-            attack data and system designs
For a subject this broad and all-encom-             than-human reactions.                     for evaluating research results is
passing (it depends on security, reliability,     ƒƒ High-fidelity simulations,               needed for all types of systems
situational awareness and attack attri-             including: how to simulate                covered in this section, not just
bution, metrics, usability, life cycle              physical aspects together with            for typical data but also for
evaluation, combating malware and                   control functions, integrate              extreme cases. Issues concerning
insider misuse, and many other aspects),            security in testing and                   proprietary data and data
it seems wise to be prepared to launch              simulation, and validate the              sanitization need to be addressed,
multiple efforts targeting this topic area.         simulation. Appropriate degrees           including post-incident data
                                                    of fidelity, and determining that a
                                                                                              and analysis such as flight data
Measures of success                                 simulation is sufficiently realistic.
                                                                                              records; and integration of
Success should be measured by the range           ƒƒ Private industry needs to be             testbeds (wireless, SCADA,
of environments over which the system               engaged.                                  general IT), enabling testbed
is capable of delivering adequate service         ƒƒ Analytical models should be              capabilities to be combined.
for top-priority tasks. These environ-              developed based on simulations.
ments will vary by topology and spatial           ƒƒ Red Teaming to assess structured
distribution: number, type, and location            survivability, with red teams
of compromised machines; and a broad                employing domain-specific skills.
range of disruption strategies.
                                                  ƒƒ Adversarial modeling that seeks
                                                    to understand the threat to time-
What needs to be in place for                       critical systems.
test and evaluation?
Many issues are relevant here:
                                                To what extent can we test
                                                real systems?
   ƒƒ Metrics for survivability:                  ƒƒ Testing of large systems:
     determining which existing                     survivability is not easy to test in
     metrics (MTBF, etc.) are                       a very large and complex system,
     applicable, which measures of                  such as an electric power grid.
     success are appropriate, what                  Relevant issues include: how to
     additional aspects of survivability            share access to existing testbeds
     and time criticality should be                 and how to compose results of
     measured (not covered by existing              subsystem tests.

                                                                  SURVIVABILITY OF TIME-CRITICAL SYSTEMS                       63
References
[Avi+2004]   A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr. Basic concepts
             and taxonomy of dependable and secure computing. IEEE Transactions on
             Dependable and Secure Computing, 1(1):11-33, January-March 2004.

[DIS2003]    3rd DARPA Information Survivability Conference and Exposition (DISCEX-III 2003), 22-24
             April 2003, Washington, DC, USA. IEEE Computer Society 2003, ISBN 0-7695-1897-4.

[Ell+1999]   R.J. Ellison, D.A. Fisher, R.C. Linger, H.F. Lipson, T. Longstaff, and N.R.
             Mead. Survivable Network Systems: An Emerging Discipline. Technical Report
             CMU/SEI-97-TR-013, Carnegie Mellon University, May 1999.

[Hai+2007]   Yacov Y. Haimes, Joost R. Santos, Kenneth G. Crowther, Matthew H. Henry, Chenyang Lian, and
             Zhenyu Yan. Analysis of Interdependencies and Risk in Oil & Gas Infrastructure Systems. I3P Research
             Report No. 11, June 2007 (http://www.thei3p.org/docs/publications/researchreport11.pdf ).

[Ker+2008]   Peter Kertzner, Jim Watters, Deborah Bodeau, and Adam Hahn. Process Control System Security
             Technical Risk Assessment Methodology & Technical Implementation. I3P Research Report No.
             13, March 2008 (http://www.thei3p.org/docs/publications/ResearchReport13.pdf ).

[Neu2000]    P.G. Neumann. Practical Architectures for Survivable Systems and Networks. SRI International,
             Menlo Park, California, June 2000 (http://www.csl.sri.com/neumann/survivability.html).




64   SURVIVABILITY OF TIME-CRITICAL SYSTEMS
Current Hard Problems in INFOSEC Research
             8. Situational Understanding and Attack Attribution
             BACKGROUND

             What is the problem being addressed?
             Situational understanding is information scaled to one’s level and areas of interest.
             It encompasses one’s role, environment, the adversary, mission, resource status, what
             is permissible to view, and which authorities are relevant. The challenges lie in the
             path from massive data to information to understanding, allowing for appropriate
             sharing at each point in the path.

             The questions to be answered, in rough order of ascending difficulty, are the
             following:

                ƒƒ Is there an attack or misuse to be addressed (detection, threat assessments)?
                ƒƒ What is the attack (identification, not just IDS signature)?
                ƒƒ Who is the attacker (accurate attribution)?
                ƒƒ What is the attacker’s intent (with respect to the present attack as well as
                  predicting behavior over time)?
                ƒƒ What is the likely impact?
                ƒƒ How do we defend (autonomous enterprises and the community as a
                  whole)?
                ƒƒ What (possibly rogue) infrastructure enables the attack?
                ƒƒ How can we prevent, deter, and/or mitigate future similar occurrences?

             Situational understanding includes the state of one’s own system from a defensive
             posture irrespective of whether an attack is taking place. It is critical to understand
             system performance and behavior during non-attack periods, in that some attack
             indicators may be observable only as deviations from “normal behavior.” This
             understanding also must include performance of systems under stress that are
             not caused by attacks, such as a dramatic increase in normal traffic due to sudden
             popularity of a particular resource.

             Situational understanding also encompasses both the defender and the adversary.
             The defender must have adversary models in order to predict adversary courses of
             action based on the current defensive posture. The defender’s system-level goals
             are to deter unwanted adversary actions (e.g., attacking our information systems)
             and induce preferred courses of action (e.g., working on socially useful projects as
             opposed to developing crimeware, or redirecting attacks to a honeynet).

             Attack attribution is defined as determining the identity or location of an attacker
             or an attacker’s intermediary. Attribution includes the identification of interme-
             diaries, although an intermediary may or may not be a willing participant in an

                                                                                                  65
attack. Accurate attribution supports        and how our decision makers interpret,        Adversaries may be able to exfiltrate
improved situational understanding and       react to, and mitigate those attacks. Of      sensitive data over periods of time,
is therefore a key element of research in    special concern are attacks on informa-       again without actually taking down
this area. Appropriate attribution may       tion systems with potentially significant     the targeted systems. Here, situational
often be possible only incrementally,        strategic impact, such as wide-scale          understanding should clearly include
as situational understanding becomes         power blackouts or loss of confidence         understanding of government threat
clearer through interpretation of avail-     in the banking system. Attacks may            models and concerns. Sharing such
able information.                            come from insiders, from adversaries          understanding is particularly impor-
                                             using false credentials, from botnets, or     tant—and sensitive in the sense that it is
Situational understanding is larger than     from other sources or a blend of sources.     likely to lead to recognition of additional
one user, or possibly even larger than one   Understanding the attack is essential for     weaknesses and vulnerabilities.
administrative domain, and addresses         defense, remediation, attribution to the
what is happening through consider-          true adversary or instigator, hardening       In addition, the more serious attacks
ation of a particular area of interest at    of systems against similar future attacks,    now occur at two vastly different time-
a granularity that is appropriate to the     and deterring future attacks. Attribution     scales. The classic fear is cyber attacks
administrator(s) or analyst(s). In partic-   should also encompass shell companies,        that occur faster than human response
ular, situational understanding of events    such as rogue domain resellers whose          times. Those attacks are still of concern.
within infrastructures spanning multiple     business model is to provide an enabling      However, another concern is “low and
domains may require significant coor-        infrastructure for malfeasance. There         slow” and possibly stealthy attacks
dination and collaboration on multiple       are numerous areas of open research           that break the attack sequences into
fronts, such as decisions about when/        when it comes to these larger questions       a series of small steps spread over a
whether to share data, how to depict the     of attribution. For example, we have          long time period. Achieving situational
situation as understanding changes over      not adequately addressed digital finger-      awareness for these two ends of the con-
time, and how to interpret or respond        printing of rogue providers of hosting        tinuum is likely to require very different
to the information. Attribution is a key     services. (See also Section 9.)               approaches.
element of this process, since it is con-
cerned with who is doing what and what       There have been numerous widely pub-          Who are the potential
should be done in response.                  licized large-scale attacks launched for
                                                                                           beneficiaries? What are their
                                             a variety of purposes, but recently there
                                                                                           respective needs?
What are the potential                       is a consensus that skilled nonstate
                                             actors are now primarily going after          Although all computer users and all
threats?
                                             financial gain [GAO2007, Fra2007].            consumers of information systems
Situational understanding addresses a        Click fraud, stock “pump and dump,”           products are potential victims of the
broad range of cyber attacks, specifically   and other manipulations of real-time          broad range of attacks we address, and
including large-scale and distributed        markets prove that it is possible to profit   would benefit from improved situational
attacks, where it is felt that adversary     from cybercrime without actually taking       awareness, we are primarily seeking tools
capabilities are outstripping our ability    down the systems that are attacked. In        and techniques to help the communities
to defend critical systems. Inability to     this context, situational understanding       whose challenges and needs are given
attribute sophisticated attacks to the       should clearly encompass law enforce-         in Table 8.1—although this is not an
original perpetrator leads to a growing      ment threat models and priorities, as         comprehensive set.
asymmetry in cyber conflict.                 well as how financial gains can accrue.
                                                                                           Because of time criticality for respond-
In this topic area, we are concerned         For state actors, the current concern         ing to certain cyber attacks, and hence
chiefly with the universe of cyber attacks   is targeting of our critical infrastruc-      the need to tie these to situational aware-
within the information systems domain        tures and key government systems.             ness, we consider developers and users

66    SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION
                                 TABLE 8.1: Beneficiaries, Challenges, and Needs

 Beneficiaries                                Challenges                                        Needs
 System Administrators                        Overwhelmed by attacks buried in massive          Timely detection, presentation, sharing with
                                              data volumes. Limited visibility beyond own       peers across administrative boundaries.
                                              domain.                                           Effective remediation.
 Service Providers                            Service continuity in spite of large-scale        Attack attribution. Identify and quarantine
                                              attacks. Understanding emerging attacks.          compromised systems. Reliable IP mapping
                                              Sharing with peers.                               to jurisdiction to support effective
                                                                                                cooperation with law enforcement.
 Law Enforcement                              Identify and prosecute perpetrators               Coordination with service providers
                                              (individuals and emerging cybercrime              and administrators. Data collection,
                                              syndicates).                                      presentation, and analysis of forensic
                                                                                                quality. Attribution to ultimate perpetrator.
 Civil Government                             Continuity in spite of large-scale attacks        Detection of attacks. Early identification
                                              on government and civilian systems.               of attacks on critical infrastructure sectors.
                                              Coordination of national-level response.          Sharing with private sector as well as state/
                                                                                                local agencies. Attribution.
 Military                                     Prevent attacks on defense systems.               Early detection and identification of attacks.
                                              Maintain system continuity in spite of            Attribution. All of the above.
                                              attacks. Prevent exfiltration of critical data.




of autonomic response systems as part         but is currently accomplished via ad               not know and trust each other. (For
of the customer base for advances in this     hoc and informal relationships. In a               example, how can an administrator in
topic area.                                   few instances, data is shared across               Domain A prove that a customer of
                                              organizations, but normally the kinds              Domain B is an attacker, and thereby
What is the current state of                  of information shared are limited                  persuade an administrator in that domain
                                              (e.g., only network packet headers).               to take corrective action?)
the practice?
Situational understanding currently           Intrusion detection/prevention tech-               Industry has made significant progress
is addressed within administrative            nology is widely deployed, but many                in the area of event/data correlation,
domains through intrusion detection/          question how much longer it will be                with several security information and
prevention systems and security event         effective as traffic volumes grow, attacks         event management (SIEM) commercial
correlation systems, with much of the         get more subtle, signature bases grow              products widely deployed in the market.
analysis still done through manual            correspondingly larger and unable to               These offer considerable value in timely
perusal of log files. There have been         cope with new attacks, and attackers               data reduction and alarm management.
efforts to provide visualizations and         use encryption, which makes packet                 However, with respect to visualization
other analytical tools to improve the         payload signature analysis difficult.              and presentation on a massive data scale,
ability to comprehend large amounts of        Response to large-scale attacks remains            these systems are inadequate and do not
data. These are largely special purpose       to a large degree informal, via personal           have scope well beyond organizational
and found within research laboratories        trust relationships and telephone com-             boundaries.
rather than being used widely within          munications. This situation makes it
the field. Sharing security-relevant infor-   difficult or impossible to achieve very            We need to consider the viewpoint of
mation across domains is essential for        rapid response or cooperation between              the defender (end host, infrastructure
large-scale situational understanding         domains where the administrators do                component, enterprise, Internet). An

                                              SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION                                               67
ISP wants an “inward” view of enterprise       There are several forums for security          techniques are applied to situational
customers since cooperative security           event information sharing, such as             understanding. There are significant
benefits from each domain’s filter-            SANS Internet Storm Center’s dshield           challenges and opportunities as link
ing outbound attack traffic from that          [ISC], which describes itself as a coop-       speeds become faster and data storage
domain (egress filtering). A defender at       erative network security community,            becomes cheaper.
an edge router is also looking outward         and PhishTank [Phi], which allows
at its peers to monitor the inbound            the defender community to contribute           In the area of attribution, there is
flows for attack traffic (ingress filter-      known or suspected instances of phish-         active research in traceback techniques.
ing). This ingress filtering is essential to   ing attacks. Phishing refers to a broad        However, most methods depend on
the cooperative awareness and response         class of fraudulent attempts to get a user     cooperative defense and do not function
mentioned above.                               to provide personal information that the       well with less than universal deploy-
                                               phisher can subsequently use for identity      ment. Skilled attackers easily evade most
Lack of trust between providers, issues of     theft, identity fraud, unlawful financial      currently deployed traceback systems.
scalability, and issues of partial deploy-     transactions, and other criminal activity.
ment of defenses make attribution                                                             There has been some research in attacker
difficult in many cases. Privacy regula-       For reasons ranging from customer              intent modeling, with an objective to
tions and the very real concern that data      privacy and concerns about revealing           predict the attacker’s next steps, but
sanitization techniques are ineffective        defensive posture to legal liability issues,   this has had only limited success. In
also present barriers. The differing legal     only limited meaningful progress has           addition, most academic research in
regimes in different countries, or within      been made in the area of interdomain           the cybersecurity field uses inadequate
different areas of governments within          security information sharing and in            adversary models that do not capture
the same country, inhibit attribution as       determining attacker location and              how high-level adversaries actually
well. There is a need for international        intent.                                        attack complex systems. As mentioned
dialogue in how to handle cybersecurity                                                       previously, the short-term goal is mod-
incidents, so that attackers can either be     What is the status of current                  eling adversary behavior to generate
identified and prosecuted or otherwise                                                        better attack indicators. The long-term
                                               research?
deterred from future wrongdoing.                                                              goal is to deter unwanted behaviors
                                               Research in attack detection has contin-       and to promote appropriate behaviors
Progress is being made in many areas           ued along the path of faster signature         (e.g., working for a legitimate organiza-
important to situational understanding,        development and propagation, seeking           tion as opposed to organized crime) via
including attribution. Protocols such          to reduce the time window in which             improved attribution. Most research in
as IPsec and IPv6’s extension headers          zero-day attacks have an impact.               this area is emphasizing the short-term
for authentication may improve the                                                            goal rather than the longer-term goal.
situation in the sense that spoofing the       Egress filtering is increasingly used to
attack source is more difficult than in        identify internal assets that may be cur-      Sharing actionable data while respect-
current IPv4 networks. However, these          rently compromised. This egress filtering      ing privacy, authenticating the shared
message authentication techniques do           (or, more generally, “unbiased introspec-      information in the absence of inter-
not solve the underlying problem of            tion”) also applies to ISPs, enterprises,      domain trust, the economy of sharing
compromised machines being used to             and home computers.                            (sharing marketplace), and sharing with
attack third parties. Thus, there is an                                                       privacy and anonymity are important
important linkage between this topic           Scalable information processing                research issues (see Section 10). Policy
and the topic addressing malware (see          (e.g., data reduction), data mining,           and legal barriers to sharing also need to
Section 5).                                    statistical analysis, and other similar        be addressed, in addition to the difficult




68     SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION
technical questions. Sharing lets one             reporting responsibilities, assure    en.wikipedia.org/wiki/OODA_Loop).
know if he or she is part of an attack            integrity, and how long to store      By analogy to physical security systems,
and needs to take action, and also lets           data and in what form.                “reaction” might be further broken out
one see the global picture. Some of the        ƒƒ Analysis. Analyze the collected       into delay, response, and mitigation
legal framework from the PREDICT                  data to abstract out meaning,         steps. Some courses of action by the
data repository may be applicable                 potentially seek additional           defender might delay the adversary from
(http://www.predict.org). There are               information for consolidation,        achieving the ultimate objective of the
also examples in international scientific         identify security incidents and       attack. This buys time for the defender
collaborations involving information              compute relevant metadata.            to mount an effective response that
systems that could be considered in ways       ƒƒ Presentation. Distill security        thwarts the adversary’s goal. Another
to collectively identify threats. Another         incidents and related contextual      response might be to seek out addi-
model for sharing is seen in the interna-         information to form enterprise-       tional information that will improve
tional honeynet community.                        level situational awareness; enable   situational awareness. If an effective
                                                  responses while maintaining           response is not possible, then mitigation
There are different variants of attribu-          forensic quality for attribution.     of the consequences of the adversary’s
tion. In closed user communities, users           Presentation may involve data         action is also a valuable course of action.
often consent to monitoring as a con-             sanitization or modification          Many responses may require coordina-
dition for system access, so it is easier         to comply with privacy or             tion across organizational boundaries,
to assert who did what. A consent-to-             classification-level requirements     and shared situational awareness will be
monitoring policy is not likely to be             on who is allowed to view what.       important in supporting such activities.
implemented globally, so attribution of        ƒƒ Sharing. Develop sharing
attacks that come in from the Internet            awareness across independent          What are the major gaps?
will remain difficult. This second type of        domains and mechanisms
attribution should be balanced against            to present relevant data to           Attack signature generation and propa-
the need for anonymity and free speech            appropriate communities, such         gation are falling short, as many “legacy
concerns arising from requiring that all          as network operators and law          attacks” are still active on the Internet
traffic to be subject to attribution.             enforcement, and preserve             years after they were launched. Legacy
                                                  privacy of users, sensitive           attacks persist for many reasons, such
                                                  corporate and national-security       as poor system administrative practices
FUTURE DIRECTIONS                                 data, and system defensive            or lack of support for system admin-
                                                  posture.                              istration, a proliferation of consumer
On what categories can we                      ƒƒ Reaction. Determine local and         systems not under professional system
subdivide this topic?                             cross-domain course of action         administration but with a high-band-
We frame this topic area on the follow-           to mitigate events. This includes     width connection, reemergence of older
ing categories:                                   measures to stop further damage,      machines after being in storage without
                                                  fix damage that has occurred,         appropriate attention (e.g.,  travel
   ƒƒ Collection. Identify what data              proactively change security           laptops put back into service), or use
     to collect; develop methods for              configurations, and collect           of legacy code or hardware in new
     data collection, preservation of             forensics to enable attribution       applications or devices. This persis-
     chain of custody (see Section 9),            and prosecution.                      tence indicates that research is needed
     validation, and organization.                                                      into better tools for system adminis-
   ƒƒ Storage. Decide how to protect         This framework may be considered an        tration, as well as for survivability of
     data in situ, efficiently access        adaptation of John Boyd’s OODA loop        well-administered systems in an envi-
     stored data, and establish              (Observe, Orient, Decide, Act (http://     ronment where many other systems are



                                             SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION                                   69
poorly maintained. Also, the ability to       apply within a single computer or local       In addition to the database hurdles
quickly scrutinize new applications and       network, but it could also be sufficient to   (such as scale and organization) that
devices to see whether legacy flaws have      provide attribution within a domain, or       must be overcome in the collection of
been reintroduced would be beneficial.        even a country. Moreover, adversaries are     these diverse sources, it is in the inter-
                                              getting better at hiding the true origin      est of the adversary to poison these data
There remain significant gaps in the          of their attacks behind networks of           sources. Research is needed so that data
intrusion detection field, and currently      compromised machines (e.g., botnets),         provenance can be maintained. (See
deployed intrusion detection systems          and throwaway computers may become            Section 9.)
(IDS) fall short of needs, especially         as common as throwaway cell phones as
with respect to enabling distributed          prices drop. Adversaries increasingly use     Analysis on Massive Data Scales. The
correlation. In particular, approaches        techniques such as fast flux, where the       analysis or evaluation process must
that include ever-growing signature sets      DNS is rapidly manipulated to make            consider the massive scale and hetero-
in attempting to identify the increasing      identification and takedown of adversary      geneity of data sources and the fact
variety of attacks may be approaching         networks difficult [Hol2008].                 that most of the data arriving from the
the end of their usefulness; alternative                                                    above sources is uninteresting chaff.
approaches are clearly needed.                                                              The data and analysis should support a
                                              What are some exemplary                       variety of granularities, such as Border
Detection of attacks within encrypted                                                       Gateway Protocol (BGP) routes, DNS
                                              problems for R&D on this
payloads will present an increasingly                                                       queries, domains in country-code top
                                              topic?
serious challenge. Many botnets now use                                                     level domains (TLDs), repeated pat-
encrypted command and control chan-           Collect and Store Relevant Data.              terns of interaction that arise over the
nels. There are researchers investigating     Understand how to identify, collect,          course of months or years, and unex-
techniques that take advantage of this,       and ultimately store data appropriate         pected connections between companies
such as using the presence of ciphertext      to the form of situational awareness          and individuals. These derived quan-
on certain communications channels as         desired. This might involve network-          tities should themselves be archived
an attack indicator. However, it is likely    centric data such as connectivity with        or, alternatively, be able to be easily
that the fraction of encrypted traffic will   peers over time, archives of name reso-       reconstructed. The availability of these
increase under legitimate applications,       lution, and route changes. In addition,       data sources plays an important role
and thus alternative approaches are once      data may need to be combined and/or           in enabling attack attribution and also
again needed.                                 sanitized to make it suitable for sharing     contributes to an incremental building
                                              or downstream retrieval, such as with         of situational awareness.
Attribution remains a hard problem. In        lower-layer alerts, local as well as exter-
most modern situations, it is useful to       nal view, system- and application-level       Novel Approaches to Presentation in
get as close as possible to the ultimate      alerts, packet contents supporting deep       Large-Scale Data. The massive scale
origin (node, process, or human actor).       packet inspection on demand without           of the data poses challenges to timely,
However, doing so touches on privacy,         violating privacy or organizational secu-     compact, and informative presentation.
legal, and forensic issues. For example,      rity, archives to support snapshots and       Scalable visualization, visualization with
public safety argues for full attribution     history, and externally deployed moni-        accurate geolocation, and zoomable
of all actions on the Internet, while         toring infrastructure such as honeynets.      visualization at varying levels of detail
free-speech rights in a civil society are     Finally, data outside networks and hosts      are just some of the difficult problems.
likely to require some forms of socially      is also relevant, such as “people layer”      Maintaining the ability to delve into the
approved anonymity. We also need to           knowledge, as in tracking the so-called       original data as well as broaden out to a
define the granularity of attack attribu-     Russian Business Network (RBN) over           high-level, people-aware view is an area
tion. In this respect, attribution could      time.                                         for future research.


70     SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION
Collaborative Collection, Vetting, and     peer-to-peer (P2P) systems. Multiple        What R&D is evolutionary and
Archiving. Collaborative collection        issues arise with modern approaches.        what is more basic, higher
of non-open data and the subse-            Sparse reports may be misleading,           risk, game changing?
quent vetting, archiving, correlation      because voting mechanisms may not
(for example inferring routes collab-      allow determining truth. Proving that       Along the collection dimension,
oratively), and generation of useful       only one organization is under attack       near- and medium-term areas include
metadata are important research issues.    may be difficult (likely to require sub-    identification of data types, sources,
Numerous database issues arise, includ-    mitting traffic samples that may reveal     collection methods, and categorization;
ing processing of huge repositories,       defensive posture, and subject to the       directed data selection; and instru-
definition and derivation of meaningful    possibility of spoofing). We require        mentation of software and hardware
metadata such as provenance, valida-       research in enabling technologies to        components and subsystems. Long-term
tion of inputs, and multilevel security    promote sharing across organizational       research may consider systems that are
(MLS). Such an archive would support       boundaries.                                 intrinsically enabled for monitoring and
both research and operations. There are                                                auditing. Challenges include the rapid
serious questions as to what to share      Situational Understanding at Multiple       growth of data and data rates, chang-
and to what degree, and these questions    Timescales. We must be aware that there     ing ideas about what can potentially
may occur at multiple levels. Examples     are multiple timescales at which situ-      be monitored, and privacy issues. (See
include what one controls, what one        ational understanding must be inferred      Section 10.)
shares in a trusted community, and what    and presented. For low and slow attacks,
we can observe about an uncooperative      such as those involved in insider-threat    With respect to analysis, there is con-
and possibly adversarial entity.           investigations, the attack traffic may      sensus that the current signature-based
                                           occur over long time spans (years or        approaches will not keep up with the
Cross-Boundary Sharing of Situ-            decades) and encompass multiple ingress     problem much longer, because of issues
ational Understanding. Crossing            points. In contrast, autonomic response     of scale as well as system and attack com-
organizational boundaries may require      requires millisecond situational under-     plexity. Attack triage methods should
reputation systems and other ways of       standing. For the human consumer, the       be examined in the short term. Traffic
quickly determining when it might be       timescale is intermediate.                  encryption and IPv6 may render many
safe to share information that cannot                                                  attack vectors harder but may also make
itself be gamed. It may be possible        Some exemplary approaches are sum-          analysis more difficult. In the long term,
to leverage research in reputation in      marized in Table 8.2.                       conceptual breakthroughs are required


                                          TABLE 8.2: Exemplary Approaches

Category                                   Definition                                 SampleƒSolutions
Collect and Analyze Data                   Understanding threats to overall           Broad-based threat and misuse detection
                                           trustworthiness and potential risks of     integrating misuses and survivability threats
                                           failures
Massive-Scale Analysis                     New approaches to distributed system and   Trustworthy systems with integrated
                                           enterprise attack analysis                 analysis tools
Situational Understanding across           Interpretation of multiple analyses over   Intelligent correlated interpretation of likely
boundaries and multiple timescales         space and time                             consequences and risks




                                          SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION                                         71
to stay even with or ahead of the threat.    reduction, alarm management, and             ability to change the situational aware-
For example, some botnet command             drill-down capability. In the near           ness information presented to agents or
and control (C2) traffic is already on       term, the emerging field of visual           other autonomous response vehicles is a
encrypted channels. Ideally, intrinsically   analytics may provide useful insights,       potential vulnerability.
monitorable systems would permit an          with new visualization devices pre-
adversary little or no space to operate      senting opportunities for new ways of        Sharing relevant information spans
without detection, or at least permit        viewing items. An emerging challenge         the gamut of levels from security alerts
observation that could be turned to          in displaying situational awareness is the   to sharing situational understanding
detection with additional analysis. Such     increase in reliance both on very large      obtained from analysis. Sharing can
systems detect attacks without a signa-      (wall-size) viewing screens and on very      enable global situational understand-
ture base that grows essentially without     small handheld screens (e.g., BlackBer-      ing and awareness, support reliable
bound. They also permit one to reliably      ries). A suggested long-term effort is to    attribution, and guide local response
assert that a system is operating in an      consider alternative metaphors suited to     appropriate to the global picture.
acceptably safe mode from a security         the various extremes available, including    Research is needed to determine how
standpoint. Additional approaches are        such options as the scrollable, zoom-        to achieve sharing with adequate privacy
needed that address monitoring and           able map. Inference and forecasting are      protections, and within regulatory
analysis in system design.                   also appropriate for long-term efforts.      boundaries, what to share across auton-
                                             We should build on the research in           omous systems, and possible market
The state of the art tends to rely on        information presentation for human           mechanisms for sharing. The issue of
detection. Some limited progress has         understanding and response. Another          liability for misuse or for fraudulent or
been made to date on predicting attack-      hard problem is visualization of low and     erroneous shared data will need to be
ers’ next steps or inferring attacker        slow attacks. Near- and medium-term          addressed.
intent. Advances in target analysis will     research is needed in how to assess the
better identify what is public and thus      way different situational awareness pre-     Research in appropriate reaction
presumed known to the adversary. This        sentation approaches affect an analyst’s     has both local (within an enterprise,
work may lead to solutions whereby           or administrator’s ability to perform.       within an autonomous systems) and
defenders manipulate the exposed                                                          global (across enterprise and autono-
“attack surface” to elicit or thwart         Presentation approaches need awareness       mous systems) components. Ideally,
attacker intent, or use cost-effective       as to whether the consumer is a human        the output of current and previous
defenses that increase protections when      or an autonomous agent; reliance on          research results should support an effec-
it is predicted they are needed. Cor-        intelligent agents or other forms of         tive course of action. When this is shared
related attack modeling advances are         automated response means that these          between entities, the shared information
appropriate to pursue as a medium-term       elements will also need “situational         should support effective local reaction,
area. Game theoretic and threat model        awareness” to provide context for their      while preserving privacy along with
approaches have made limited headway         programmed behaviors. We require             other information sanitization needs.
in this field but should be considered as    research to enable agent-based defenses      Research is required, for example in
long-term research. Threat and adversary     in instances where action is needed at       authenticating the authors of actionable
modeling may also support advances           faster than human response times. This       information and proof that a recom-
toward attribution and the ultimate goal     is a presentation issue that ought to be     mended course of action is appropriate.
of deterring future cyber attacks. This      addressed in the medium term, and            Research is also required in alternatives
is suitable for medium- to long-term         a sharing issue when agent-to-agent          to malfeasor blocking (it may be prefer-
research.                                    cooperation is required in the long          able to divert and observe), remediation
                                             term. It is important to keep in mind        of compromised assets (a need also
Information presentation will require        that autonomous response may be an           present in the malware research topic),
continued advancements in data               attack vector for the adversary, and the     and exoneration in the case of false

72    SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION
positives. Although response and reac-        involves human subjects. In many cases,     This section focuses on protection
tion are not directly a part of situational   the IRBs are inadequately equipped to       against cyber attack in the informa-
understanding, situational understand-        handle cybersecurity experiments, which     tion domain. However, adversaries may
ing is needed to enable response and          are crucial to understanding attackers’     choose to interleave their cyber-attack
reaction, and situational understanding       intent and next steps. Government           steps with attack steps in the other
may drive certain kinds of responses          could play a role in ensuring that IRBs     three domains of conflict—namely the
(e.g., changing information collection        are better equipped to expedite attack      physical, cognitive, and social domains.
to improve attribution). Thus, advances       attribution research. A set of best prac-   Research on situational understand-
in reaction and response techniques           tices would be beneficial in this area.     ing and attribution tools that integrate
directly affect the kind of situational                                                   attack indicators from all four domains
awareness that is required.                   Government roles also include               of conflict is also needed.
                                              developing policy, funding research
                                              (complementing industry), and exerting
Resources                                     market leverage through its acquisition     Measures of success
Situational understanding requires col-       processes. There is government-spon-        We will measure progress in numerous
lection or derivation of relevant data on     sored research in intrusion detection,      ways, such as decreased personnel hours
a diverse set of attributes. Some of the      software engineering for security,          required to obtain effective situational
attributes that support global situational    malware analysis, traceback, informa-       understanding; increased coverage of the
understanding and attack attribution          tion sharing, scalable visualization,       attack space; based on mission impact,
are discussed above relating to the kinds     and other areas that affect this topic.     improved ability to triage the serious
of data to collect. A legal and policy        Government has also implemented             attacks from the less important and
framework, including international            fusion centers, common databases for        the ones where immediate reaction is
coordination, is necessary to enable          experimentation, and testbeds, sup-         needed from those where an alterna-
the collection and permit the exchange        porting collaboration. Continuing these     tive approach is acceptable; improved
of much of this information, since it         investments is crucial, particularly in     response and remediation time; and
often requires crossing international         the long-term range for areas that are      timely attribution with sound forensics.
boundaries. In addition, coordination         not conducive to short-term industry        These all require reliable collection of
across sectors may be needed in terms         investment.                                 data on the diverse set of attributes listed
of what information can be shared and                                                     previously.
how to gather it in a timely way. Con-        This topic is particularly dependent
sider an attack that involves patient data    on public-private partnerships, and         On the basis of these attributes, we
information systems within a hospital         the definition of the nature of these       could define measures of success at
in the United States, a military base in      partnerships is essential. To a degree,     a high level within a given organiza-
Germany, and an educational institution       this depends on competing visions of        tion’s stated security goals. For example,
in France. All three institutions have        success. One may consider a central-        an organization aimed primarily at
different requirements for what can and       ized network operations center (NOC)        maintaining customer access to a par-
cannot be shared or recorded.                 staffed by government, industry, and        ticular service might measure success
                                              researchers with a policy and procedural    by observing and tracking over time
Modifications to U.S. law and policy          framework designed to allow seamless        such variables as the estimated number
may be needed to facilitate data sharing      cooperation. An alternative view is a       of hosts capable of serving information
and attack attribution research. As an        distributed capability in which differ-     over some service, and the estimated
example, institutional review boards          ent network operators share situational     near-steady-state number or growth
(IRBs) play an important role in protect-     understanding but different parts of the    trend of these machines.
ing individuals and organizations from        picture are relevant to different system
the side effects of experimentation that      missions.                                   Success depends on timely identification

                                              SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION                                     73
of adversaries, propagation of defenses,     We require a methodology to quantify        an open-source framework with defined
and remediation of affected systems.         mission impact. Many stakeholders have      standards and interfaces, and devel-
Another measure for success is tied to       a primary need to maintain continuity       oping relationships with entities that
a variation of the false-positive/true-      of operations in spite of a large-scale     could deploy it. Many results from this
positive discussion, in that effective       attack.                                     topic require distributed deployment
situational understanding should allow                                                   for meaningful test and evaluation. The
us to accurately categorize the potential                                                honeynet community may be a good
impact of a detected attack. For either      What needs to be in place for               deployment platform with less resis-
actual attacks or emulated attacks on a      test and evaluation?                        tance than commercial systems and less
realistic testbed, we would hope to be       Several research testbeds are online        concern about privacy issues. Significant
able to answer the following questions:      (e.g., the existing DETER lab testbed,      barriers exist in both the technical and
                                             http://www.deterlab.net) or planned;        organizational/policy domains, associ-
     ƒƒ Can we differentiate between         research in situational understanding       ated with the difficulty of protecting the
       nuisance and serious strategic        would be advanced via federation of         privacy and security of the real systems
       attacks, for example, by              these and other testbeds to emulate scale   being scrutinized.
       identifying a targeted attack         and cross-domain issues. Large-scale
       against a critical sector?            simulation may provide initial rough        Technologies resulting from research in
     ƒƒ Can we share information across      estimates of the efficacy of particular     this topic area range from individual-
       informational boundaries to           approaches. In terms of Internet-scale      host-level components (for example,
       enable cooperative response?          situational understanding, these testbeds   inherently monitorable systems) to
     ƒƒ Can we quickly quarantine            can support advances in the malware         global components (mechanisms for
       intermediate attack platforms?        and botnets topic area as well.             reliable geolocation). In the former
                                                                                         category, R&D should be conducted
     ƒƒ Can we maintain or quickly
                                                                                         from the start with system developers
       restore critical functions, perhaps   To what extent can we test                  to ensure adoptability of resulting solu-
       according to some contingency         real systems?                               tions. Success in the latter category may
       policy of acceptable degradation?     There are test environments that allow      require some new frameworks in law,
     ƒƒ Can we collect actionable data for   deployment of prototype cybersecurity       policy, and Internet governance.
       ultimate attribution?                 modules. We should consider developing


References
[Fra2007]          J. Franklin, V. Paxson, A. Perrig, and S. Savage. An inquiry into the nature and
                   causes of the wealth of Internet miscreants. Proceedings of ACM Computer and
                   Communications Security Conference, pp. 375-388, October 2007.

[GAO2007]          CYBERCRIME: Public and Private Entities Face Challenges in Addressing Cyber Threats. Report
                   GAO-07-705, U.S. Government Accountability Office, Washington, D.C., July 2007.

[Hol2008]          T. Holz, C. Gorecki, K. Rieck, and F. Freiling. Measuring and detecting fast-flux service networks. In
                   Proceedings of the 15th Annual Network & Distributed System Security (NDSS) Symposium, February 2008.




74      SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION
[ICA2008]   Draft Initial Report of the GNSO Fast Flux Hosting Working Group. ICANN. December 8, 2008
            (https://st.icann.org/pdp-wg-ff/index.cgi?fast_flux_pdp_wg).

[ISC]       Internet Storm Center: http://www.dshield.org/about.html.

[Phi]       PhishTank: http://www.phishtank.com.




                                  SITUATIONAL UNDERSTANDING AND ATTACK ATTRIBUTION                      75
Current Hard Problems in INFOSEC Research
              9. Provenance

              BACKGROUND

              What is the problem being addressed?
              Individuals and organizations routinely work with, and make decisions based on,
              data that may have originated from many different sources and also may have
              been processed, transformed, interpreted, and aggregated by numerous entities
              between the original sources and the consumers. Without good knowledge about
              the sources and intermediate processors of the data, it can be difficult to assess the
              data’s trustworthiness and reliability, and hence its real value to the decision-making
              processes in which it is used.

              Provenance refers to the chain of successive custody—including sources and
              operations—of computer-related resources such as hardware, software, documents,
              databases, data, and other entities. Provenance includes pedigree, which relates
              to the total directed graph of historical dependencies. It also includes tracking,
              which refers to the maintenance of distribution and usage information that enables
              determination of where resources went and how they may have been used.

              Provenance is also concerned with the original sources of any subsequent changes
              or other treatment of information and resources throughout the life cycle of data.
              That information may be in any form, including software, text, spreadsheets, images,
              audio, video, proprietary document formats, databases, and others, as well as meta-
              level information about information and information transformations, including
              editing, other forms of markup, summarization, analysis, transformations from one
              medium to another, formatting, and provenance markings. Provenance is generally
              concerned with the integrity and reliability of the information and meta-information
              rather than just the information content of the document.

              Provenance can also be used to follow modifications of information—for example,
              providing a record of how a document was derived from other sources or providing
              the pervasive history through successive versions (as in the Concurrent Versions
              System [CVS]), transformations of content (such as natural language translation
              and file compression), and changes of format (such as Word to PDF).

              The granularity of provenance ranges from whole systems through multi-level
              security, file, paragraph, and line, and even to bit. For certain applications (such as
              access control) the provenance of a single bit may be very important. Provenance
              itself may require meta-provenance, that is, provenance markings on the provenance
              information. The level of assurance provided by information provenance systems
              may be graded and lead to graded responses. Note that in some cases provenance
              information may be more sensitive, or more highly classified, than the underlying
              data. The policies for handling provenance information are complex and differ for
              different applications and granularities.

76
To determine provenance accurately, we      scientific fields are examples where prov-    What is the current state of
must have trustworthy systems that reli-    enance markings are beginning to be           practice?
ably track both usage and modification      used. Other fields that can benefit from
of information and other resources. As      provenance maintenance systems include        Physical provenance markings in jewelry
with all computer systems, security of      critical infrastructure providers (e.g., in   (e.g., claiming your diamond is from a
provenance tracking cannot be absolute,     SCADA and other control systems),             blood-free mining operation, your silver
and trustworthiness of provenance track-    emergency responders, military person-        or gold is pure, and the style is not a
ing systems will be relative to the value   nel, and other decision makers. Users in      knockoff copy of a designer’s), explo-
of the provenance to the users of the       all these areas need reliable information     sive components (e.g., nitrates), and
information and resources. For example,     obtained from many sources, commu-            clothing have historically added value
a simple change-tracking mechanism          nicated, aggregated, analyzed, stored,        and enabled tracing of origin. Docu-
in a document preparation system may        and presented by complex information          ment markings such as wax seals and
provide adequate provenance track-          processing systems. Information sources       signatures have been used to increase
ing from the point of view of a small       must be identified, maintained, and           assurance of authenticity of high-value
group of authors collaborating in the       tracked to help users make appropriate        documents for centuries. More recently
publication of an article, even though      decisions based on reliable understand-       the legal, auditing, and medical fields
the document change history might           ing of the provenance of the data used        have begun to employ first-level authen-
not be protected from unauthorized          as input to critical decisions.               ticated provenance markings.
modification. On the other hand, the
same mechanism may be inadequate in         In addition, new techniques are needed        The current practice is rather rudimen-
the context of legal discovery, precisely   that will allow management of prov-           tary compared with what is needed to
because the change-tracking mechanism       enance for voluminous data. Part of           be able to routinely depend on prov-
does not guarantee the authenticity of      what has made provenance easier to            enance collection and maintenance.
the change history.                         manage up to now is its small volume.         The financial sector (in part driven
                                            Now, geospatial information-gathering         by Sarbanes-Oxley requirements) has
What are the potential                      systems are being planned that will have      developed techniques to enable track-
                                            the capability of handling gigabytes of       ing of origins, aggregations, and edits of
threats?
                                            data per second, and the challenges of        data sets. Users of document production
Without trustworthy provenance track-       these data volumes will be exacerbated        software may be familiar with change-
ing systems, there are threats to the       by collection via countless other sensor      tracking features that provide a form of
data and to processes that rely on the      networks. Within 20 years, the govern-        provenance, although one that cannot
data, including, for example, unattrib-     ment will hold an exabyte of potentially      necessarily be considered trustworthy.
uted sources of software and hardware;      sensitive data. The systems for handling
unauthorized modification of data           and establishing provenance of such           As an example of provenance in which
provenance; unauthorized exposure of        volumes of information must function          security of the provenance has not been
provenance, where presumably pro-           autonomously and efficiently with infor-      a direct concern, software development
tected; and misattribution of provenance    mation sources at these scales.               teams have relied for decades on version
(intentional or otherwise).                                                               control systems to track the history of
                                            Note that situations are likely to arise      changes to code and allow for historical
Who are the potential                       where absence of provenance is impor-         versions of code to be examined and
beneficiaries? What are their               tant—for example, where information           used. Similar kinds of systems are used
respective needs?                           that needs to be made public must not         in the scientific computing community.
The legal, accounting, medical, and         be attributable.




                                                                                                          PROVENANCE             77
What is the status of current                   ƒƒ Provenance-aware storage               ƒƒ Pedigree management. The
research?                                         systems. A provenance-                     Pedigree Management and
Current research appears to be driven             aware storage system supports              Assessment Framework (PMAF)
largely by application- and domain-spe-           automatic collection and                   [SPI2007] enables a publisher
cific needs. Undoubtedly, these research          maintenance of provenance                  of information in a network-
efforts are seen as vital in their respective     metadata. The system creates               centric intelligence gathering and
communities of interest.                          provenance metadata as new                 assessment environment to record
                                                  objects are created in the system          standard provenance metadata
Examples of active, ongoing research              and maintains the provenance               about the source, the manner
areas related to information and resource         just as it maintains ordinary file-        of collection, and the chain of
provenance include the following areas:           system metadata. See [PAS]. The            modification of information as it
                                                  Lineage File System [LFS] records          is passed through processing and
     ƒƒ Data provenance and
                                                  the input files, command-line              assessment.
       annotation in scientific
                                                  options, and output files when a
       computing. Chimera [Fos2002]                                                     For further background, see the proceed-
                                                  program is executed; the records
       allows a user to define a                  are stored in an SQL database         ings of the first USENIX workshop on
       workflow, consisting of data sets          that can be queried to reconstruct    the theory and practice of provenance
       and transformation scripts. The            the lineage of a file.                [TAP2009].
       system then tracks invocations,
                                                ƒƒ Chain of custody in computer
       annotating the output with
                                                  forensics and evidence and            FUTURE DIRECTIONS
       information about the runtime
                                                  change control in software
       environment. The myGrid                                                          On what categories can we
                                                  development. The Vesta
       system [Zha2004], designed                                                       subdivide the topic?
                                                  [Hey2001] approach uses
       to aid biologists in performing
                                                  provenance to make software           Provenance may be usefully subdivided
       computer-based experiments,                builds incremental and
       allows users to model their                                                      along three main categories, each of
                                                  repeatable.                           which may be further subdivided, as
       workflows in a Grid environment.
                                                ƒƒ Open Provenance Model. The           follows:
       CMCS [Pan2003] is a toolkit for
                                                  Open Provenance Model is a
       chemists to manage experimental                                                    ƒƒ Representation: data models
                                                  recently proposed abstract data
       data derived from fields such                                                         and representation structures
                                                  model for capturing provenance.
       as combustion research. ESSW                                                          for provenance (granularity and
                                                  The model aims to make it easier
       [Fre2005] is a data storage system                                                    access control).
                                                  for provenance to be exchanged
       for earth scientists; the system                                                   ƒƒ Management (creation; access;
                                                  between systems, to support
       can track data lineage so that             development of provenance tools,           annotation [mark original
       errors can be traced, helping              to define a core set of inference          documents/resources with
       maintain the quality of large              rules that support queries on              provenance metadata]; editing
       data sets. Trio [Wid2005] is a             provenance, and to support                 [provenance-mark specific
       data warehouse system that uses            a technology-neutral digital               fine-grained changes through
       data lineage to automatically              representation of provenance               the life cycle]; pruning [delete
       compute the accuracy of the                for any object, regardless of              provenance metadata for
       data. Additional examples can be           whether or not it is produced              performance, security, and
       found in the survey by Bose and            by a computer system. See                  privacy reasons]; assurance; and
       Frew [Bos2005].                            [OPM2007].                                 revocation)


78      PROVENANCE
   ƒƒ Presentation (query [request            In the following itemization of gaps, the       ƒƒ Pruning provenance, deleting
     provenance information]; present         letters R, M, P annotating each point              and sanitizing extraneous item
     [display provenance markings];           refer to the main categories—represen-             for privacy and purpose of
     alert [notify when provenance            tation, management, and presentation,              performance. (RMP)
     absence, compromise, or fraud is         respectively—where uppercase denotes            ƒƒ Efficiently representing
     detected])                               high relevance (R, M, P), and lowercase            provenance. An extreme
                                              denotes some relevance (r, m, and p).              goal would be to efficiently
Other useful dimensions to consider                                                              represent provenance for every
that are cross-cutting with respect to the       ƒƒ Appropriate definitions and
                                                    means for manipulating                       bit, enabling bit-grained data
following dimensions:                                                                            transformations, while requiring
                                                    meaningful granularity of
   ƒƒ System engineering (human-                                                                 a minimum of overhead in time
                                                    information provenance
     computer interfaces; workflow                                                               and space. (RMp)
                                                    markings. Taxonomy of
     implications; and semantic webs)               provenance. (R)                           ƒƒ Scale: the need for solutions that
   ƒƒ Legal, policy, and economic                                                                scale up and down efficiently. (R)
                                                 ƒƒ Given trends in markup
     issues (regulation; standards;                                                           ƒƒ Dealing with heterogeneous data
                                                    languages, the metadata and
     enforcement; market incentives)                                                             types and data sensors, domain
                                                    the underlying data are often
These are summarized in Table 9.1.                  intermixed (as in XML),                      specificity, and dependency
                                                    thus presenting challenges                   tracking. (Rm)
What are the major research                         in appropriate separation of              ƒƒ Partial or probabilistic
gaps?                                               concerns with data integrity and             provenance (when the chain of
                                                    integrity of the provenance. (R)             custody cannot be stated with
Numerous gaps in provenance and                                                                  absolute certainty). (RMp)
                                                 ƒƒ Confidential provenance
tracking research remain to be filled,
                                                    and anonymous or partially                ƒƒ Coping with legacy systems.
requiring a much broader view of the
                                                    anonymous provenance,                        (RM)
problem space and cross-disciplinary
                                                    to protect sources of                     ƒƒ Intrinsic vs. extrinsic provenance
efforts to capture unifying themes and
                                                    information. (R)                             and the consistency between
advance the state of the art for the
benefit of all communities interested in         ƒƒ Representing the trustworthiness             them when both are available.
provenance.                                         of provenance. (R)                           (RMp)


                                             TABLE 9.1: Potential Approaches

 Category                                     Definition                                  PotentialƒApproaches
 Representation                               Data models and structures for provenance   Varied granularities, integration with access
                                                                                          controls
 Management                                   Creation and revocation of indelible        Trustworthy distributed embedding with
                                              distributed provenance                      integrated analysis tools
 Presentation                                 Queries, displays, alerts                   Usable human interfaces
 System engineering                           Secure implementation                       Integration into trustworthy systems
 Legal, policy, economic issues               Social implications                         Regulation, standards, enforcement,
                                                                                          incentives




                                                                                                            PROVENANCE                79
     ƒƒ Developing and adopting tools            Teams (CERTs) need to be              What R&D is evolutionary,
       based on existing research results.       able to prove from where              and what is more basic,
       (RMP)                                     they got information about            higher risk, game changing?
     ƒƒ Centralized versus distributed           vulnerabilities and fixes; when       Information provenance presents a
       provenance. (M)                           they publish alerts, they should      large set of challenges, but significant
                                                 be able to reliably show that         impact may be made with relatively
     ƒƒ Ensuring the trustworthiness of
                                                 the information came from an          modest technical progress. For example,
       provenance (integrity through the
                                                 appropriate, credible source—for      it may be possible to develop a coarse-
       chain of custody). (M)
                                                 example, to avoid publishing          grain information provenance appliance
     ƒƒ Tracking: where did the                  an alert based on incorrect           that marks documents traversing an
       information/resources go; how             information submitted by a
                                                                                       intranet or resting in a data center and
       were they used? (M)                       competitor. They also need
                                                                                       makes those markings available to deci-
     ƒƒ Usable provenance respecting             their customers to believe that
                                                                                       sion makers. Although this imagined
       security and privacy concerns.            the information being sent is
                                                                                       appliance may not have visibility into
       (Mp)                                      not from an imposter (although
                                                                                       all the inputs used to create a docu-
                                                 certificates are supposed to take
     ƒƒ Information provenance systems                                                 ment, it could provide relatively strong
                                                 care of this problem).
       should be connected to chain of                                                 assurances about certain aspects of the
                                               ƒƒ Law enforcement forensics            provenance of the information in ques-
       custody, audit, and data forensic
                                                 for computer-based evidence,          tion. It is important to find methods
       approaches. Provenance should
                                                 surveillance data, and other          to enable incremental rollout of prove-
       connect and support, not repeat
                                                 computer artifacts, of sufficient     nance tools and tags in order to maintain
       functionality of these related
                                                 integrity and oversight to            compliance with existing practices and
       services. (MP)
                                                 withstand expert counter-             standards. Another incremental view is
     ƒƒ User interfaces. When dealing            testimony.                            to consider provenance as a static type
       with massive amounts of data            ƒƒ Crime statistics and analyses from   system for data. Static type systems exist
       from many sources with massive            which patterns of misuse can be       for many programming languages and
       communication processes, how is           deduced.                              frameworks that help prevent runtime
       the end user informed and about
                                               ƒƒ Medical and health care              errors. By analogy, we could create an
       what aspects of the information
                                                 information, particularly with        information provenance system that is
       integrity? (P)
                                                 respect to data access and data       able to prevent certain types of misuse of
     ƒƒ Users of aggregated information          modification.                         data by comparing the provenance infor-
       need to be able to determine                                                    mation with policies or requirements.
                                               ƒƒ Identity-theft and identity-fraud
       when less reliable information
                                                 detection and prevention.
       is interspersed with accurate                                                   Resources
       information. It is of critical          ƒƒ Financial sector—for example,        With respect to the extensive list of
       importance to identify and                with respect to insider               research gaps noted above, resources will
       propagate the source and                  information, funds transfers, and     be needed for research efforts, experi-
       derivation (or aggregation) of            partially anonymous transactions.     mental testbeds, test and evaluation, and
       the chain of custody with the           ƒƒ Provenance embedded within           technology transition.
       information itself. (P)                   digital rights management.
                                                                                       Measures of success
What are some exemplary                      In many of the above examples, some       One indicator of success will be the
problem domains for R&D in                   of the provenance may have to be          ability to track the provenance of infor-
this area?                                   encrypted or anonymized—to protect        mation in large systems that process and
     ƒƒ Computer Emergency Response          the identity of sources.                  transform many different, heterogeneous

80      PROVENANCE
types of data. The sheer number of dif-     also provide measures of success. Effi-      ƒƒ In medical systems, personally
ferent kinds of sensors and information     ciency of representations might also           identifiable information
systems involved and, in particular, the    be a worthwhile indicator, as would be         connected with embarrassing or
number of legacy systems developed          measures of overhead attributable to           insurance-relevant information
without any attention to maintenance        maintaining and processing provenance.         may be used to make life-critical
of provenance present major challenges      Metrics that consider human usability          health care decisions.
in this domain.                             of provenance would be very appropri-
                                                                                         ƒƒ An emergency responder system
                                            ate—especially if they can discern how
                                                                                           might be considered that could
Red Teaming can give added analysis—        well people actually are able to distin-
                                                                                           provide more reliable provenance
for example, assessing the difficulty of    guish authentic and bogus information
                                                                                           information to decision makers
planting false content and subverting       based on provenance.
provenance mechanisms.                                                                     (e.g., who must be evacuated,
                                                                                           who has been successfully
                                            What needs to be in place for                  evacuated from a building).
Also, confidence-level indicators are
                                            test and evaluation?
desirable—for example, assessing the                                                     ƒƒ A provenance system for the legal
estimated accuracy of the information       Testing and evaluating the effectiveness       profession.
or the probability that information         of new provenance systems is challeng-       ƒƒ Credit history and scoring—for
achieves a certain accuracy level.          ing because some of the earliest adopters      example, provenance on credit
                                            of the technology are likely to be in
                                                                                           history data might help reduce
More generally, analytic tools can evalu-   domains where critical decisions depend
                                                                                           delays involved in getting a
ate (measure) metrics for provenance.       on provenance data. Thus, the impact
                                                                                           mortgage despite errors in credit
                                            of mistaken provenance could be large.
                                                                                           reports.
Cross-checking provenance with
archived file modifications in environ-     Potential testbed applications should be     ƒƒ Depository services; title history;
ments that log changes in detail could      considered, such as the following:             personnel clearance systems.


References
[Bos2005]        R. Bose and J. Frew. Lineage retrieval for scientific data processing: a survey. ACM Computing Surveys,
                 37(1):1-28, 2005.

[Fos2002]        I.T. Foster, J.-S. Voeckler, M. Wilde, and Y. Zhao. Chimera: A virtual data system for representing, querying,
                 and automating data derivation. In Proceedings of the 14th Conference on Scientific and Statistical Database
                 Management, pp. 37-46, 2002.

[Fre2005]        J. Frew and R. Bose. Earth System Science Workbench: A data management infrastructure for earth science
                 products. In Proceedings of the 13th Conference on Scientific and Statistical Database Management, p. 180,
                 2001.

[Hey2001]        A. Heydon, R. Levin, T. Mann, and Y. Yu. The Vesta Approach to Software Configuration Management.
                 Technical Report 168, Compaq Systems Research Center, Palo Alto, California, March 2001.

[LFS]            Lineage File System (http://theory.stanford.edu/~cao/lineage).




                                                                                                      PROVENANCE             81
[OPM2007]   L. Moreau, J. Freire, J. Futrelle, R.E. McGrath, J. Myers, and P. Paulson. The Open Provenance Model.
            Technical report, ECS, University of Southampton, 2007 (http://eprints.ecs.soton.ac.uk/14979/).

[Pan03]     C. Pancerella et al. Metadata in the collaboratory for multi-scale chemical science. In Proceedings of the
            2003 International Conference on Dublin Core and Metadata Applications, 2003.

[PAS]       PASS: Provenance-Aware Storage Systems (http://www.eecs.harvard.edu/~syrah/pass/).

[SPI2007]   M.M. Gioioso, S.D. McCullough, J.P. Cormier, C. Marceau, and R.A. Joyce. Pedigree management and
            assessment in a net-centric environment. In Defense Transformation and Net-Centric Systems 2007. Proceedings
            of the SPIE, 6578:65780H1-H10, 2007.

[TAP2009]   First Workshop on the Theory and Practice of Provenance, San Francisco, February 23, 2009 (http://www.
            usenix.org/events/tapp09/).

[Wid2005]   J. Widom. Trio: A system for integrated management of data, accuracy, and lineage. In Proceedings of the
            Second Biennial Conference on Innovative Data Systems Research, Pacific Grove, California, January 2005.

[Zha2004]   J. Zhao, C.A. Goble, R. Stevens, and S. Bechhofer. Semantically linking and browsing provenance logs for
            e-science. In Proceedings of the 1st International Conference on Semantics of a Networked World, Paris, 2004.




82   PROVENANCE
Current Hard Problems in INFOSEC Research
             10. Privacy-Aware Security

             BACKGROUND

             What is the problem being addressed?
             The goal of privacy-aware security is to enable users and organizations to better
             express, protect, and control the confidentiality of their private information, even
             when they choose to—or are required to—share it with others. Privacy-aware
             security encompasses several distinct but closely related topics, including anonymity,
             pseudo-anonymity, confidentiality, protection of queries, monitoring, and appropri-
             ate accessibility. It is also concerned with protecting the privacy of entities (such as
             individuals, corporations, government agencies) that need to access private informa-
             tion. This document does not attempt to address the question of what information
             should be protected or revealed under various circumstances, but it does highlight
             challenges and approaches to providing technological means for safely controlling
             access to, and use of, private information. The following are examples of situations
             that may require limited sharing of private information:

                ƒƒ The need to prove things about oneself (for example, proof of residence)
                ƒƒ Various degrees of anonymity (protection of children online, victims of
                  crime and disease, cash transactions, elections)
                ƒƒ Enabling limited information disclosure sufficient to guarantee security,
                  without divulging more information than necessary
                ƒƒ Identity escrow and management
                ƒƒ Multiparty access controls
                ƒƒ Privacy-protected sharing of security and threat information, as well as
                  audit logs
                ƒƒ Control of secondary reuse
                ƒƒ Remediation of incorrect information that is disclosed, especially if done
                  without any required user approval
                ƒƒ Effective, appropriate access to information for law enforcement and
                  national security
                ƒƒ Medical emergencies (for example, requiring information about allergic
                  reactions to certain medications)


             What are the potential threats?
             Threats to private information may be intrinsic or extrinsic to computer systems.
             Intrinsic computer security threats attributable to insiders include mistakes, acciden-
             tal breach, misconfiguration, and misuse of authorized privileges, as well as insider
             exploitations of internal security flaws. Intrinsic threats attributable to outsiders
             (e.g., intruders) include potential exploitations of a wide variety of intrusion tech-
             niques. Extrinsic threats arise once information has been viewed by users or made

                                                                                                  83
available to external media (via printers,     information bureaus) is also              bidding on a job, engaging in a
e-mail, wireless emanations, and so on),       needed.                                   collaborative venture, pursuing
and has come primarily outside the           ƒƒ Organizations do not want                mergers, and the like.
purview of authentication, computer            proprietary information disclosed      ƒƒ Social networks need means
access controls, audit trails, and other       for other than specific agreed            to share personal information
monitoring on the originating systems.         purposes.                                 within a community while
                                             ƒƒ Research communities (e.g.,              protecting that information from
The central problem in privacy-aware                                                     abuse (such as spear-phishing).
security is the tension between compet-        in medical research and social
ing goals in the disclosure and use of         sciences) need access to accurate,     ƒƒ Governments need to collect
private information. This document             specific, and complete data for           and selectively share information
takes no position on what goals should         such purposes as analysis, testing        for such purposes as census,
be considered legitimate or how the            hypotheses, developing potential          disease control, taxation, import/
tension should be resolved. Rather,            treatments/solutions.                     export control, and regulation of
the goal of research in privacy-aware        ƒƒ Law enforcement requires                 commerce.
security is to provide the tools necessary     access to personal information to
to express and implement trade-offs            conduct thorough investigations.     What is the current state of
between competing legitimate goals           ƒƒ National security/intelligence      practice?
in the protection and use of private           needs to detect and prevent
information.                                   terrorism and hostile activity by    Privacy-aware security involves a complex
                                               nation-states and nonstate actors    mix of legal, policy, and technological
Who are the potential                          while maintaining the privacy        considerations. Work along all these
                                               of U.S. persons and coalition        dimensions has struggled to keep up
beneficiaries? What are their
                                               partners.                            with the pervasive information sharing
respective needs?
                                                                                    that cyberspace has enabled. Although
                                             ƒƒ Financial sector organizations
The beneficiaries for this topic are many                                           the challenges have long been recog-
                                               need access to data to analyze for
and widely varied. They often have                                                  nized, progress on solutions has been
                                               indicators of potential fraud.
directly competing interests. An exhaus-                                            slow, especially on the technology side.
tive list would be nearly impossible to      ƒƒ Health care industries need         At present, there are no widely adopted,
produce, but some illustrative examples        access to private patient            uniform frameworks for expressing and
include the following:                         information for treatment            enforcing protection requirements for
                                               purposes, billing, insurance, and    private information while still enabling
     ƒƒ Individuals do not generally           reporting requirements.              sharing for legitimate purposes. On the
       want to reveal any more private                                              technology side, progress has been made
                                             ƒƒ Product development and
       information than absolutely                                                  in certain application areas related to
                                               marketing uses data mining
       necessary to accomplish a                                                    privacy. Examples of privacy-enhancing
                                               to determine trends, identify
       specific goal (transaction,                                                  technologies in use today include the
                                               potential customers, and tune
       medical treatment, etc.) and                                                 following:
                                               product offerings to customer
       want guarantees that the
                                               needs.
       information disclosed will be                                                  ƒƒ Access controls (e.g., discretion-
       used only for required and            ƒƒ Business development,                    ary and mandatory, role-
       authorized purposes. The ability        partnerships, and                         based, capability-based, and
       to detect and correct erroneous         collaborations need to selectively        database management system
       data maintained by other                reveal proprietary data to a              authorizations) attempt to limit
       organizations (such as credit           limited audience for purposes of          who can access what information,

84      PRIVACY-AWARE SECURITY
     but they are difficult to configure       ƒƒ Minimizing data retention time           parties, or providing the ability later
     to achieve desired effects, are             appropriately                             to identify the misusers. A significant
     often too coarse-grained, and                                                         challenge to the DRM approach is the
                                               ƒƒ Protecting data in transmission
     may not map well to actual                                                            development of an indisputable defini-
                                                 and storage (e.g., with
     privacy and data use policies.                                                        tion of who controls the distribution.
                                                 encryption)
  ƒƒ Encrypted storage and                                                                 For example, should medical informa-
                                               ƒƒ Conducting sensible risk analyses        tion be controlled by the patient, by
     communications can prevent
     wholesale loss or exposure of             ƒƒ Auditing of access audit logs            doctors, by nurses, by hospitals, or by
     sensitive data but do very little to        (actually examining them, not             insurance companies, or by some com-
     prevent misuse of data accessed                                                       bination thereof? Each of them may be
                                                 just keeping them)
     within allowed privileges or                                                          the originator of different portions of
                                               ƒƒ Privacy policy negotiation and           the medical information. Information
     within flawed system security.
                                                 management                                provenance (Section 9) interacts with
  ƒƒ Anonymous credential systems
                                                                                           privacy in defining the trail of who did
     may enable authorization without                                                      what with the medical information, and
     necessarily revealing identity (for    What is the status of current
                                            research?                                      both interact with system and informa-
     example, Shibboleth [Shib]).                                                          tion integrity.
  ƒƒ Anonymization techniques,              Security with privacy appears to require
     such as mix networks, onion            establishment of fundamental trust             Many examples of ongoing or planned
     routing, anonymizing proxy             structures to reflect demands of privacy.      privacy-related research are of interest
     servers, and censorship-resistant      It also requires means for reducing the        here. For example, the following are
     access technology, attempt             risks of privacy breaches that can occur       worth considering. NSF Trustworthy
     to mask associations between           (accidentally or intentionally) through        Computing programs have explicitly
     identities and information             the use of technologies such as data           included privacy in recent solicitations
     content.                               mining. Ideas for reconciling such             (http://www.nsf.gov/funding/). Some
  ƒƒ One-time-use technologies,             technologies in this context include           research projects funded by the National
     such as one-time authenticators        privacy-aware, distributed association-        Research Council Canada are also
     and smart cards, can also              rule mining algorithms that preserve           relevant (http://iit-iti.nrc-cnrc.gc.ca/
     contribute.                            privacy of the individual sites, queries on    r-d/security-securite_e.html), as are
                                            encrypted data without decrypting, and         British studies of privacy and surveil-
At the same time, there are known best      a new formulation to address the impact        lance, including a technology roadmap
practices that, if consistently adopted,    of privacy breaches that makes it possible     (http://www.raeng.org.uk/policy/
would also advance the state of the prac-   to limit breaches without knowledge of         reports/pdf/dilemmas_of_privacy_and_
tice in privacy-preserving information      original data distribution.                    surveillance_report.pdf ).
sharing. These include
                                            Digital rights management (DRM)                Other privacy related research includes
  ƒƒ Use of trustworthy systems and         techniques, while not currently applied        the following:
     sound system administration,           for privacy protection, could be used
     with strong authentication,                                                             ƒƒ Microsoft Research database
                                            to protect information in such diverse
                                            settings as health care records and cor-            privacy: (http://www.research.
     differential access controls, and
                                            porate proprietary data, allowing the               microsoft.com/jump/50709
     extensive monitoring
                                            originator of the information to retain             and http://www.microsoft.com/
  ƒƒ Adherence to the principle of                                                              mscorp/twc/iappandrsa/research.
                                            some degree of access control even after
     least privilege                        the information has been given to third             mspx)



                                                                                          PRIVACY-AWARE SECURITY                85
     ƒƒ Project Presidio: collaborative       ƒƒ See also (http://www.itaa.org/             Oxley; HIPAA), and economics
       policies and assured information          infosec/faith.pdf ) and (http://           and security (e.g., http://www.
       sharing                                   www.schneier.com/blog/                     cl.cam.ac.uk/~rja14/econsec.
       (http://www.projectpresidio.com)          archives/2007/03/security_                 html).
                                                 plus_p.html)
     ƒƒ Stanford University Web Security
       Research: private information                                                   What are the major research
       retrieval (http://crypto.stanford.   FUTURE DIRECTIONS                          gaps?
       edu/websec/)
                                                                                       Following are some of the gaps in pri-
     ƒƒ Security with Privacy ISAT          On what categories can we                  vacy-aware security that need to be
       briefing (http://www.cs.berkeley.    subdivide the topic?                       addressed.
       edu/~tygar/papers/ISAT-final-        For purposes of a research and devel-
       briefing.pdf )                       opment roadmap, privacy-aware              Selective disclosure and
                                            information sharing can be usefully        privacy-aware access
     ƒƒ Naval Research Lab: Reputation
                                                                                          ƒƒ Sound bases are needed for
       in Privacy Enhancing                 divided along the following categories,
                                            directly mirroring the gaps noted above.         selective disclosure through
       Technologies (http://chacs.
                                            See Table 10.1.                                  techniques such as attribute-
       nrl.navy.mil/publications/
                                                                                             based encryption, identity-based
       chacs/2002/2002dingledine-
                                              ƒƒ Selective disclosure and                    encryption, collusion-resistant
       cfp02.pdf )
                                                 privacy-aware access to data:               broadcast encryption, private
     ƒƒ ITU efforts related to security,                                                     information retrieval (PIR), and
                                                 theoretical underpinnings and
       privacy, and legislation: (http://                                                    oblivious transfer.
                                                 system engineering.
       www.itu.int/ITU-D/cyb/
                                                                                         ƒƒ How do we share data sets
       publications/2006/research-            ƒƒ Specification frameworks for
                                                                                            while reducing the likelihood
       legislation.pdf )                         providing privacy guarantees:
                                                                                            that arbitrary users can infer
     ƒƒ DHS report on the ADVISE                 languages for specifying privacy
                                                                                            individual identification? (The
       program (http://www.dhs.gov/              policies, particularly if directly
                                                                                            U.S. Census Bureau has long
       xlibrary/assets/privacy/privacy_          implementable; specifications              been concerned about this
       rpt_advise.pdf )                          for violations of privacy; and             problem.)
     ƒƒ UMBC Assured privacy                     detecting violations of privacy.
                                                                                         ƒƒ Data sanitization techniques are
       preserving data mining, recipient      ƒƒ Policy issues: establishing                needed that are nonsubvertible
       of DoD’s MURI award (http://              privacy policies, data correction,         and that at the same time do not
       ebiquity.umbc.edu/blogger/tag/            propagation of updates, privacy            render analysis useless.
       muri/)                                    implications of data integrity.         ƒƒ More generally, data quality
     ƒƒ Anonymous communication                  This also includes legal (aspects          must be maintained for research
       (http://freehaven.net/anonbib)            of current law that constrain              purposes while protecting
     ƒƒ Statistics research community, as        technology development;                    privacy, avoiding profiling or
       in the Knowledge Discovery and            aspects of future law that could           temporal analysis to deanonymize
       Data Mining (KDD) conferences             enable technology development;             source data.
       (http://sigkdd.org)                       questions of jurisdiction),             ƒƒ Irreversible transformations
     ƒƒ Framework for privacy metrics            standards (best practices; privacy         of content are needed that
       [Pfi+2001])                               standards analogous to Sarbanes-           exhibit statistical characteristics




86       PRIVACY-AWARE SECURITY
     consistent with the original data         Policy issues                                    ƒƒ Policies are needed for dealing
     without revealing the original              ƒƒ Distinctions between individual                with privacy violations, detection
     content.                                       and group privacy are unclear.                 of violations, consequences of
  ƒƒ Privacy and security for very large          ƒƒ Release of bogus information                  violations, and remediation of
     data sets does not scale easily—               about individuals is poorly                    damage.
     for example, maintaining privacy               handled today. However, with
     of individual data elements is                 stronger protection it becomes
                                                                                             What are some exemplary
     difficult.                                     more difficult to check validity of
                                                    information.                             problems for R&D on this
  ƒƒ Associations of location with                                                           topic?
     users and information may                    ƒƒ Information gathered from some
                                                                                             Several problem domains seem par-
     require privacy protection,                    persons can allow probabilistic
                                                                                             ticularly relevant, namely, data mining
     particularly in mobile devices.                inference of information about
                                                                                             for medical research, health care
                                                    others.
  ƒƒ Low-latency mix networks can                                                            records, data mining of search queries,
     provide anonymization, but need              ƒƒ Policies for data collection and        census records, and student records at
     further research.                              sharing with regard to privacy           universities.
                                                    are needed, especially relating
  ƒƒ Mechanisms to enforce retention
                                                    to what can be done with the
     limits are lacking.                                                                     What R&D is evolutionary and
                                                    private data. For example, who
                                                                                             what is more basic, higher
  ƒƒ Sharing of security information                are the stakeholders in genetic          risk, game changing?
     such as network trace data needs               information? What policies are
     privacy controls.                              needed for retention limits?             Near term
                                                                                               ƒƒ Deriving requirements for
Specification frameworks                          ƒƒ Communications create further                automating privacy policies:
  ƒƒ Specification frameworks for                   privacy problems relating to                  learning from P3P
     expressing privacy guarantees are              identification of communication
                                                                                                ƒƒ Policy language development
     weak or missing. In particular,                sources, destinations, and
     specification and enforcement of               patterns that can reveal                    ƒƒ Implement best practices
     context-dependent policies for                 information, even when other                ƒƒ Research into legal issues in
     data sharing and use are needed.               data protections are in place.                 communications privacy


                                              TABLE 10.1: Potential Approaches

Categories                                     Definition                                   PotentialƒApproaches
Selective disclosure and privacy-preserving    Technology to support privacy policies       Varied granularities, integration with access
access to data                                                                              controls and encryption
Specification frameworks                       Creation and revocation in distributed       Implementable policy languages, analysis
                                               provenance                                   tools
Other privacy issues                           Policies and procedures to support privacy   Canonical policies, laws, standards,
                                                                                            economic models underlying privacy




                                                                                            PRIVACY-AWARE SECURITY                      87
Medium term                                    considerable commitment from govern-                have worked on this, as in
 ƒƒ Anonymous credentials                      ment funding agencies, corporations,                determining statistical similarity
     ƒƒ Role-based Access Control              and application communities such as                 of purposely fuzzed data sets.)
       (RBAC)                                  health care to ensure that the research is          How many queries are needed
                                               relevant and that it has adequate testbeds          to get to specific data items
     ƒƒ Attribute-based encryption
                                               for practical applications. It will also            for individuals in databases
     ƒƒ Distributed RBAC: no central                                                               that purport to hide such
                                               engender considerable scrutiny from
       enforcement mechanism required                                                              information?
                                               the privacy community to ensure that
     ƒƒ Protection against excess              the approaches are adequately privacy            ƒƒ Adversary work factors to violate
       disclosure during inference and         preserving.                                         privacy.
       accumulation
                                                                                                ƒƒ Risk analysis: This has been
     ƒƒ Application of DRM techniques
                                               Measures of success                                 applied to security (albeit
       for privacy
                                                                                                   somewhat haphazardly). Can risk
     ƒƒ Searching encrypted data               A goal for addressing concerns regarding
                                                                                                   analysis be effectively applied to
       without revealing the query;            both data mining and identity theft is to
                                                                                                   privacy?
       more generally, computation on          quantify users’ ability to retain control
       encrypted data                          of sensitive information and its dissemi-        ƒƒ Costs for identity-fraud
                                               nation even after it has left their hands.          insurance.
Long term
                                               For data mining, quantitative measures           ƒƒ Black market price of stolen
  ƒƒ Private information retrieval
                                               of privacy have been proposed only                  identity.
     (PIR)
                                               recently and are still fairly primitive. For
     ƒƒ Multiparty communication               example, it is difficult to quantify the
                                               effect of a release of personal informa-
                                                                                              What needs to be in place for
     ƒƒ Use of scale for privacy
                                               tion without knowing the full context          test and evaluation?
     ƒƒ Resistance to active attacks for
       deanonymizing data                      with which it may be fused and within          Access to usable data sets is important,
                                               which inferences may be drawn. Evalu-          for example,
     ƒƒ Developing measures of privacy         ation and refinement of such metrics are
                                                                                                ƒƒ Census data (see http://www.
Game changing                                  certainly in order.
  ƒƒ Limited data retention
                                                                                                   fedstats.gov)
                                               Useful realistic measures are needed for         ƒƒ Google Trends
     ƒƒ Any two databases should be
       capable of being federated              evaluating privacy and for assessing the         ƒƒ PREDICT (e.g., network traffic
       without loss of privacy (privacy        relative values of information.                     data; http://www.predict.org)
       composability)                                                                           ƒƒ Medical research data
                                               Possible measures of progress/success
     ƒƒ Low-latency private                    include the following:                           ƒƒ E-mail data (e.g., for developing
       communications resistant to                                                                 spam filters)
       timing attack                              ƒƒ Rate of publication of privacy-
                                                    breach stories in the media.              Possible experimental testbeds include
Resources                                         ƒƒ Database measures: Can we                the following:
                                                    simulate a database without
                                                    real data? How effective would              ƒƒ Isolated networks and their users
This topic is research-intensive, with
considerable needs for testbeds demon-              approaches be that cleanse data             ƒƒ Virtual societies
strating effectiveness and for subsequent           by randomization? Can we
technology transfer to demonstrate the              use such approaches to derive             In addition, privacy Red Teams could
feasibility of the research. It will require        metrics? (Statistical communities         be helpful.

88      PRIVACY-AWARE SECURITY
References
[Bri+1997]      J. Brickell, D.E. Porter, V. Shmatikov, and E. Witchell. Privacy-preserving remote diagnostics, CCS ’07,
                October 29 – November 2, 2007.

[Pfi+2001]      A. Pfitzmann and M. Köhntopp. Anonymity, unobservability, and pseudonymity: A proposal for terminology.
                In Designing Privacy Enhancing Technologies, pp. 1-9, Springer, Berlin/Heidelberg, 2001.

[Rab1981]       M. Rabin. How to exchange secrets by oblivious transfer. Technical Report TR-81, Aiken Computation
                Laboratory, Harvard University, 1981.

[Shib]          The Shibboleth System (http://shibboleth.internet2.edu/).

Many additional references can be found by browsing the URLs noted above in the text of this section.




                                                                                PRIVACY-AWARE SECURITY               89
Current Hard Problems in INFOSEC Research
              11. Usable Security

              BACKGROUND

              What is the problem being addressed?
              Security policy making tends to be reactive in nature, developed in response to an
              immediate problem rather than planned in advance based on clearly elucidated
              goals and requirements, as well as thoughtful understanding and analysis of the
              risks. This reactive approach gives rise to security practices that compromise system
              usability, which in turn can compromise security—even to the point where intended
              improvements in a system’s security posture are negated. Typically, as the security
              of systems increases, the usability of those systems tends to decrease, because secu-
              rity enhancements are commonly introduced in ways that are difficult for users to
              comprehend and that increase the complexity of users’ interactions with systems.
              Any regular and frequent user of the Internet will readily appreciate the challenge
              of keeping track of dozens of different passwords for dozens of different sites, or
              keeping up with frequent patches for security vulnerabilities in myriad applications.
              Many users also are confused by security pop-up dialogs that offer no intuitive
              explanation of the apparent problem and, moreover, appear completely unable
              to distinguish normal, legitimate activity, such as reading e-mail from a friend,
              or from a phishing attempt. Such pop-ups are typically ignored, or else blindly
              accepted [Sun+09].

              People use systems to perform various tasks toward achieving some goal. Unless the
              tasks at hand are themselves security related, having to think about security inter-
              feres with accomplishing the user’s main goal. Security as it is typically practiced in
              today’s systems increases complexity of system use, which often causes confusion and
              frustration for users. When the relationship between security controls and security
              risks is not clear, users may simply not understand how best to interact with the
              system to accomplish their main goals while minimizing risk. Even when there is
              some appreciation of the risks, frustration can lead users to disregard, evade, and
              disable security controls, thus negating the potential gains of security enhancements.

              Security must be usable by persons ranging from nontechnical users to experts and
              system administrators. Furthermore, systems must be usable while maintaining
              security. In the absence of usable security, there is ultimately no effective security.
              The need for usable security and the difficulties inherent in realizing adequate
              solutions are increasingly being recognized. In attempting to address the chal-
              lenges of usability and security, several guiding principles are worth considering.
              Furthermore, when we refer here to usable security, we are really concerned with
              trustworthy systems whose usability has been designed into them through proactive
              requirements, constructive architectures, sound system and software development
              practices, and sensible operation. As observed in previous sections, almost every
              system component and every step in the development process has the potential to
              compromise trustworthiness. Poor usability is a huge potential offender.

90
Security issues must be made as trans-       it or switch to an alternative system that   productivity. Security is poorly
parent as possible. For example, security    is more user friendly but less secure.       understood by nonexperts, and the
mechanisms, policies, and controls must                                                   consequences of disabled or weakened
be intuitively clear and perspicuous to      What are the potential                       security controls are often indirect and
all users and appropriate for each user.                                                  not immediately felt; and the worst
                                             threats?
In particular, the relationships among                                                    effects may be felt by those not directly
security controls and security risks must    The threats from the absence of usable       involved (e.g., credit card fraud), leading
be presented to users in ways that can be    security are pervasive and mostly noted      users to question the value of having
understood in the context of system use.     in the above discussion. However, these      security technology at all.
                                             threats are somewhat different from
Users must be considered as fundamen-        those in most of the other 10 topics—in      At the same time, consciousness of secu-
tal components of systems during all         that the threats are typically more likely   rity issues is becoming more widespread,
phases of the system life cycle. Different   to arise from inactions, inadvertence,       and technology developers are paying
assumptions and requirements pertain-        and mistakes by legitimate users. On         increasing attention to security in their
ing to users’ interactions with systems      the other hand, threats of misuse by         products and systems. However, usabil-
must be made explicit to each type           outsiders and insiders similar to those      ity in general appears not to be much
of user—novices, intermittent users,         in the other topics can certainly arise as   better understood by software practi-
experts, and system administrators, to       a result of the lack of usability.           tioners than security is. This situation
name a few. In general, one-size-fits-all                                                 makes the problem of usable security
approaches are unlikely to succeed.          Who are the potential                        even more challenging, since it com-
                                                                                          bines two problems that are difficult to
                                             beneficiaries? What are their
Relevant education about security prin-                                                   solve individually.
                                             respective needs?
ciples and operational constraints must
be pervasive. Security issues can never      Although the problem of achieving            Usability of systems tends to decrease as
be completely hidden or transparent.         usable security is universal—it affects      attempts are made to increase security
There will always be the possibility of      everyone, and everyone stands to benefit     and, more broadly, trustworthiness.
conflict between what users might want       enormously if we successfully address        Many current security systems rely on
to accomplish most easily and the secu-      usability as a core aspect of security—it    humans performing actions (such as
rity risks involved in doing so. Helping     affects different users in different ways,   typing passwords) or making decisions
users to understand these trade-offs must    depending on applications, settings,         (such as whether or not to accept an
be a key component of usable security.       policies, and user roles. The guiding        SSL certificate). For example, one e-mail
                                             principles may indeed be universal, but      system requires that users reauthenticate
Security metrics must take usability into    as suggested above there is certainly        every 8 hours to assure that they are
account. Although one might argue that       no general one-size-fits-all solution.       actually the authorized person. This
a system with a certain security control     Examples of different categories of users    requirement is a direct counter to system
is in principle more secure than an oth-     and ways in which they are affected by       usability. For example, some web brows-
erwise equivalent system without that        problems in usable security are shown        ers warn users before any script is run.
control—for example, a web browser           in Table 11.1.                               But users may still browse onto a web
that supports client/server authentica-                                                   server that has scripts on every page,
tion vs. one that does not—the real          What is the current state of                 causing pop-up alerts to appear on each
security may in fact be no greater (and                                                   page.
                                             practice?
possibly even less) in a system that
implements that security control, if its     Although the importance of secu-             Many of the potential impacts of security
introduction compromises usability to        rity technology is widely recognized,        that is not usable involve increased sus-
the point that users are driven to disable   it is often viewed as a hindrance to         ceptibility to social-engineering attacks.

                                                                                                    USABLE SECURITY               91
                                   TABLE 11.1: Beneficiaries, Challenges, and Needs

 Beneficiaries                               Challenges                                     Needs
 Nontechnical users                          Unfamiliar technology and terminology;         Safe default settings; automated assistance
                                             security risks unclear                         with simple, intuitive explanations when
                                                                                            user involvement is required
 Occasional users                            Changing security landscape; deferred          Automated, offline system maintenance;
                                             security maintenance (e.g., antivirus          automated adaptation of evolving security
                                             updates, software patches) inhibits on-        controls to learned usage patterns
                                             demand system use
 Frequent and expert users                   Hidden or inflexible security controls         Security controls that adapt to usage
                                             intended for nontechnical users; obtrusive     patterns; security control interfaces that
                                             security pop-up dialogs                        remain inconspicuous and unobtrusive, yet
                                                                                            readily accessible when needed
 Users with special needs (e.g., visual,     From a security standpoint, similar to other   Adaptations of security controls (such as
 auditory, motor control challenges)         users, but with added challenges arising       biometrics) that accommodate special
                                             from special interface needs                   needs; for example, fingerprint readers may
                                                                                            be unsuitable for users with motor control
                                                                                            challenges
 System administrators                       Configuration and maintenance of systems       Better tools that help automatically
                                             across different user categories; evolving     configure systems according to
                                             security threats and policies                  organizational policies and user
                                                                                            requirements; better tools for monitoring
                                                                                            security posture and responding to security
                                                                                            incidents
 System designers                            Lack of security and/or usability emphasis in Design standards and documented best
                                             education and training                        practices for usable security
 System developers                           Complexity of adding security and usability    Integrated development environments
                                             requirements into development processes        (IDEs) that incorporate security and usability
                                                                                            dimensions
 Policy makers                               Difficulty in capturing and expressing         Tools for expressing and evaluating security
                                             security requirements and relating them to     policies, especially with respect to trade-
                                             organizational workflows                       offs between usability (productivity) and
                                                                                            security



This might be an adversary sending an        A few illustrative examples from the            was cumbersome to configure, even
e-mail “this configuration change makes      current state of the practice may help          for experts, and imposed significant
your system more usable” to “this patch      illuminate challenges in usable security        system overhead. Key management
must be manually installed”. But it also     and identify some promising directions          was typically either cumbersome, or
involves attackers who gain the trust of     from which broader lessons may be               reduced to one key or perhaps just a
users by helping those users cope with       drawn.                                          few. Many newer operating systems
difficult-to-use systems. Thus, resistance                                                   now offer ready-to-use full-disk
to social engineering must be built into     Somewhat positive examples of usable            encryption out of the box, requiring
systems, and suitable requirements and       security might include transparent              little more than a password from the
metrics included from the outset of any      file-system encryption. When first              user, while imposing no noticeable
system development.                          introduced, file encryption technology          performance penalty.

92     USABLE SECURITY
Other, more mixed examples illustrate         understanding, leading to the           Net Trust). Although this seem
how security technology still falls short     frustration effects noted earlier.      to enhance usability, many users
in terms of usability:                      ƒƒ Mail authentication. There             may not adequately understand
                                              are mechanisms to authenticate          the implications of accepting
  ƒƒ Passwords. Security pitfalls of                                                  trust information from systems
     poorly implemented password              senders of valid e-mails, such
                                              as SPF (sender permitted                that may be unknown to those
     schemes have been extensively                                                    users. They are also unlikely to
     documented over the years.               from). DomainKeys Identified
                                              Mail (DKIM) is an e-mail                understand fully what factors
     When users must resort to                                                        might be helpful, harmful, or
     writing them on slips of paper           authentication technology that
                                              allows e-mail recipients to verify      some of each.
     or storing them unencrypted
     on handheld devices, the risk            whether messages that claim to        ƒƒ CAPTCHA systems. A
     of password exposure may                 have been sent from a particular        CAPTCHA (Completely
     outweigh the increased security of       domain actually originated there.       Automated Public Turing test
     strong passwords. Nevertheless,          It operates transparently for end       to tell Computers and Humans
     passwords are often simplistically       users and makes it easier to detect     Apart) is a challenge-response
     believed to be a usable security         possible spam and phishing              mechanism intended to ensure
     mechanism, and elaborate                 attacks, both of which often rely       that the respondent is a human
     procedures are promulgated               on domain spoofing. Some large          and not a computer. CAPTCHAs
     purporting to define sensible            e-mail service providers now            are familiar to most web users
     password practices (with respect         support DKIM.                           as distorted images of words or
     to frequency of changing, not          ƒƒ Client-side certificates. Most         other character sequences that
     using dictionary words, including                                                must be input correctly to gain
                                              web browsers and e-mail
     nonalphabetic characters, etc.).                                                 access to some service (such
                                              applications in widespread use
     Tools that help users select good                                                as a free e-mail account). To
                                              today support user authentication
     passwords and manage their                                                       make a CAPTCHA effective
                                              via certificates based on public-
     passwords have been touted                                                       for distinguishing humans from
                                              key cryptography. However, the
     to enhance both usability and                                                    computers, solving it must be
                                              technology is not well understood
     security. However, to make                                                       difficult for computers but
                                              by nonexpert users, and typically
     passwords more effective for                                                     relatively easy for humans. This
                                              the integration of client-side          balance has proven difficult to
     stronger security, they must be so       certificate authentication into
     long and so complex that users                                                   achieve, resulting in CAPTCHAs
                                              applications makes the use and          that are either breakable by
     cannot remember them, which
                                              management of these certificates        computers or too difficult for
     seriously compromises usability.
                                              opaque and cumbersome for               humans. Another challenge is
  ƒƒ Security pop-up dialogs. No              users.                                  to produce CAPTCHAs that
     matter how much effort is put                                                    accommodate users with special
                                            ƒƒ The SSL lock icon. This
     into making security controls                                                    needs.
                                              approach gives the appearance
     automated and transparent,
                                              of security, but its limitations      ƒƒ Not accounting for cultural
     there are inevitably situations
                                              are not generally understood.           differences and personal
     that require users to make
                                              For example, it may be totally          disabilities. For example,
     security-related decisions. Today,
     unfortunately, user involvement          spoofed. Its presence or absence        people of one ethnic group tend
     appears to be required too               may also be ignored.                    to have difficulty recognizing
     often and usually in terms that        ƒƒ “Web of trust”-like approaches         different faces of people in
     nontechnical users have difficulty       to certificate trust (e.g., Google,     other ethnic groups, which

                                                                                          USABLE SECURITY            93
       could cause usability differences      ƒƒ Overloading of security                     this area. An example of a new
       in authentication. Similarly,             attributions in the context                 direction might be making Tor
       CAPTCHAs could be culture                 of domain-validation                        more usable for administration.
       dependent. In addition, people            certificates. People tend to trust        ƒƒ Highlighting important changes
       with a prosopagnosia disorder             certificates too much or else are
                                                                                             to systems (e.g., operating
       have difficulty distinguishing            overwhelmed by their presence.
                                                                                             systems, middleware, and
       between different people by sight.     ƒƒ Revocation. Dealing with
       This would seriously impair their                                                     applications) that could improve
                                                 change is typically difficult, but          security and usability (rather than
       ability to distinguish among              usability may be impaired when
       different pictorial authenticators                                                    just one).
                                                 revocation is required. If not
       and CAPTCHAs.                                                                       ƒƒ Reevaluating decisions/trade-offs
                                                 carefully designed into systems
     ƒƒ Policies and centralized                 in advance with usability and               made in past systems. A sense of
       administration. Lack of user              understandability in mind,                  history in cybersecurity is vital
       flexibility is common. On the             mechanisms for revocation                   but is too often weak.
       other hand, it is generally unwise        are likely to have unintended
       to expect users to make security/                                                   ƒƒ One Laptop Per Child Bitfrost
                                                 consequences.
       usability trade-off evaluations.                                                      security model.
     ƒƒ Federated identity                  What is the status of current                  ƒƒ Integration of biometrics with
       management. Cross-domain             research?                                        laptops (e.g., fingerprint, facial
       access is complex. Simplistic                                                         recognition); this is in practice
                                            Following is a brief summary of some
       approaches such as single sign-
                                            current research, along with gaps. For           today, for better or worse. It
       on can lead to trust violations.
                                            background, see [SOU2008].                       may be good for administration,
       Conversely, managing too
       many passwords is unworkable.                                                         but perhaps not so good from
                                              ƒƒ Usable authentication. For
       More work is needed on access                                                         the point of view of user
                                                 example, visual passwords and
       cards such as the CAC system,                                                         understanding.
                                                 various other authentication
       DoD’s Common Access Card,
                                                 approaches exist but need much
       (which combines authentication,
       encryption of files and e-mail,           further work to determine
                                                 whether they can be used               FUTURE DIRECTIONS
       and key escrow) and other such
       systems to identify security              effectively. At present, they are
       vulnerabilities. In all such              often very difficult to use and        On what categories can we
       systems, usability is critical.           seem unlikely to scale well to         subdivide the topic?
     ƒƒ PGP, S/MIME, and other                   large numbers of passwords.            We consider the following three cat-
       approaches to secure e-mail.           ƒƒ User security. Currently funded        egories as a useful subdivision for
       Many past attempts to                     security-related usability research    formulating a research roadmap for
       encapsulate encryption into mail          includes the CMU CyLab Usable          usability and security:
       environments have been hindered           Privacy and Security Laboratory
       by the lack of seamless usability.                                                  ƒƒ Interface design (I)
                                                 (CUPS), and Stanford University
     ƒƒ Links. Phishing, cross-site                                                        ƒƒ Science of evaluation for usable
                                                 work on Web integrity. A list of
       scripting, and related problems                                                       security (E)
                                                 CUPS projects with descriptions
       with bogus URLs are laden with            and papers can be found at                ƒƒ Tool development (T)
       risks. URLs may seem to increase
                                                 http://cups.cs.cmu.edu.
       usability, but malicious misuse                                                  The following are second-level bins, with
       of them can seriously diminish         ƒƒ Ease of administration.                descriptors defining their relevance to I,
       security.                                 Relatively little research exists in   E, and T:

94      USABLE SECURITY
  ƒƒ Principles of usable security; a     designing for and evaluating usability        security of novel approaches and out-
    taxonomy of usable security (E)       of computer systems. However, only            of-the-box thinking in usable security.
  ƒƒ Understanding users and their        a small fraction of this research has
    interactions with security controls   focused on usability specifically as it       There is a need to increase knowledge of
    (IET)                                 relates to security. At the same time,        usability among security practitioners.
                                          security research tends to focus on spe-      A common lament in industry is that
  ƒƒ Usable authentication and
                                          cific solutions to specific problems, with    programmers are too rarely taught how
    authorization technology (IT)         little or no regard for how those solu-       to create secure programs, but even
  ƒƒ Design of usable interfaces for      tions can be made practical and, most         those who do receive such training are
    security, with resistance to social   importantly, transparent to users and         unlikely to be taught how to provide
    engineering (I)                       system administrators. To the extent that     both security and usability simultane-
  ƒƒ Development tools that assist in     security practitioners do consider the        ously. Just as with security, usability is
    the production of systems that        practical implications of their proposed      not a property that can easily be added
    are both more secure and more         solutions, the result is often a new or       to existing systems, and it is not a prop-
    usable (T)                            modified user interface component for         erty that one member of a large team can
                                          configuring and controlling the security      provide for everyone else. The implica-
  ƒƒ Adapting legacy systems
                                          technology, which does little to address      tion is that a large body of designers,
  ƒƒ Building new systems                 the fundamental problem that most             programmers, and testers needs to have a
  ƒƒ Usable security for embedded         users cannot and do not want to be            much deeper understanding of usability.
    and mobile devices (IET)              responsible for understanding and man-        Adding usability to existing curricula
                                          aging security technology; they simply        would be a good start but could not
  ƒƒ Evaluation approaches and
                                          want it to do the right thing and stay        be expected to pay dividends for years
    metrics for usability and
                                          out of the way.                               to come. Methods to increase under-
    security (E)
                                                                                        standing of usability among software
  ƒƒ User education and                   In short, usable security is not funda-       developers already working in industry
    familiarization with security         mentally about better user interfaces to      are equally necessary.
    issues and technology (IE)            manage security technology; rather, it is
  ƒƒ User feedback, experience (e.g.,     about evaluating security in the context      We need to identify a useful framework
    usability bug reports) (E)            of tasks and features and of the user, and    for discussing usability as it relates to
  ƒƒ Security policies (especially,       rearchitecting it to fit into that context.   security, such as the following:
    implementation of them) that
                                          It is important to note the inherently           ƒƒ Research on usable security
    increase both usability and
                                          interdisciplinary nature of usability              “out of the box” (security
    security (ET)
                                          and security. Security researchers and             transparency).
  ƒƒ Tools for evaluating security
                                          practitioners cannot simply expect that          ƒƒ Identification of the most useful
    policies
                                          the HCI experts will fix the usabil-               points in the R&D pipeline at
  ƒƒ Market creation for usable           ity problem for trustworthy systems.
    security technology                                                                      which to involve users in the
                                          Addressing the problem adequately
                                                                                             development of trustworthy
                                          will require close collaboration between
                                                                                             systems.
                                          members of the security and usabil-
What are the major research
                                          ity research communities. One goal               ƒƒ Research into the question of
gaps?
                                          is to develop the science of usability             how to evaluate usability as it
Human-computer interaction (HCI)          as applied to security. For example,               relates to security. Here we would
research has made strides in both         we need to have ways to evaluate the               expect significant contributions


                                                                                                  USABLE SECURITY              95
       from HCI research that has               ƒƒ Lessons from the automotive              effects they want to achieve but are not
       already developed methodologies            industry                                  experts in system administration. In
       for evaluating usability.                                                            addition, if a user decides to modify the
                                             What are some exemplary                        access configuration, how could that be
     ƒƒ System architectures that starkly
                                             problems for R&D on this                       done in a usable way, while achieving
       reduce the size and complexity                                                       only the desired modifications (e.g., not
                                             topic?
       of user interfaces, perhaps by                                                       making access to sensitive data either
       simplifying the interface, hiding     One exemplary problem is protecting            more or less restrictive than intended)?
       the complexity within the             users against those who pose as someone
       interface, providing compatible       else on the Internet. Techniques like          What R&D is evolutionary and
       interfaces for different types of     certificates have not worked. Alerts from
                                                                                            what is more basic, higher
       users (such as administrators), or    browsers and toolbars and other add-ins
                                                                                            risk, game changing?
       various other strategies, without     about suspicious identities of websites or
       losing the ability to do what must    e-mail addresses do not work, because          In the short term, the situation can be
       be done especially in times of        users either do not understand the alerts      significantly improved by R&D that
       system or component failures.         or do not bother using the tools. Note         focuses on making security technology
                                             that, if used properly, these techniques       work sensibly “out of the box”—ideally
     ƒƒ The ability to reflect physical-     could be effective. The failure is in their    with no direct user intervention. More
       world security cues in computer       lack of easy usability. The goal here          basic, higher-risk, game-changing
       systems.                              should be not just to find any alternative     research would be to identify funda-
     ƒƒ Consideration of usability           approach, but rather to find approaches        mental system design principles for
       from a data perspective; for          that can work well for ordinary users.         trustworthy systems that minimize
       example, usability needs can                                                         direct user responsibility for trustwor-
       drive collection of data that can     Another exemplary problem is the secure        thy operation.
       lead to security problems (PII as     handling of e-mail between an arbitrary
       authenticators, for example)          sender and an arbitrary receiver in a          Near term
                                             usable way. Judging from the limited             ƒƒ Informing the security research
Hard problems                                use of encrypted e-mail today, existing             community on the results
  ƒƒ Usable security on mobile devices       approaches are not sufficiently usable.             obtained in the usable security
                                             Yet, users are regularly fooled into believ-
     ƒƒ Usable mutual authentication                                                             community on the design and
                                             ing that forged e-mail is actually from
                                                                                                 execution of usability studies
     ƒƒ Reusable “clean” abstractions for    the claimed sender. It is only a matter
                                                                                                 [Cra+2005]
       usable security                       of time before serious problems are
                                             encountered because of e-mail traveling          ƒƒ Developing a bibliography of
     ƒƒ Usable management of access
                                             across its entire path unencrypted and              best practices and developing
       controls                                                                                  a community expectation that
                                             unauthenticated. For a general discus-
     ƒƒ Usable secure certificate services   sion on why cryptography is typically               security researchers will use them
     ƒƒ Resistance to social engineering     not very easily used, see [Whi+1999].               in their work
                                                                                              ƒƒ Identifying the common
Other areas we might draw on                 Another possibility is configuring an               characteristics of “good” usable
  ƒƒ Usability in avionics: reducing         office environment so that only the
                                                                                                 security (and also common
                                             people who should have access to sensi-
     the cognitive load on pilots                                                                characteristics of usability done
                                             tive data can actually access it—so that
     ƒƒ Lessons from safety in general,                                                          badly)
                                             such a configuration can be accom-
       especially warnings science           plished by users who understand the              ƒƒ Developing a useful framework

96      USABLE SECURITY
     for discussing usability (in the        Measures of success                               as part of all applicable research
     context of security)                    Meaningful metrics for usable security            in other areas.
   ƒƒ Developing interdisciplinary           must be established, along with                 ƒƒ Guidelines/How-Tos for
     connections between the security        generic principles of metrics. These              usability studies. (See Garfinkel
     and HCI communities (relates to         must then be instantiated for specific            & Cranor [Cra+2005].)
     the first bullet above)                 systems and interfaces. We need to              ƒƒ A “Usable Security 101” course,
   ƒƒ Identifying ways of involving          measure whether and to what extent                including how to develop and
     users in the security technology        increased usability leads to increased
                                                                                               evaluate usable systems.
     R&D process                             security, and to be able to find “sweet
                                             spots” on the usability and security            ƒƒ Standardized testbed for
Medium term                                  curves. Usable security is not a black-           conducting usability studies
 ƒƒ Usable access control mechanisms         and-white issue. It must also consider            (perhaps learning from DETER
    (such as a usable form of RBAC)          returns on investment.                            and PlanetLab).
   ƒƒ Usable authentication                                                                  ƒƒ Anonymous reporting system
                                             We do not have metrics that allow
   ƒƒ Developing a common                                                                      within a repository for usability
                                             direct comparison of the usability of
     framework for evaluating                                                                  problems (perhaps learning from
                                             two systems (e.g., we cannot say defini-
     usability and security                                                                    the avionics field).
                                             tively that system A is twice as usable as
   ƒƒ Long term                              system B), but we do perhaps have some
   ƒƒ Composability of usable
                                                                                          To what extent can we test
                                             well-established criteria for what consti-
     components: can we put together         tutes a good usability evaluation. One       real systems?
     good usable components for              possible approach would be to develop        Usability studies need to be based on real
     particular functions and get            a usable solution for one of the exemplar    systems. They need not be live systems
     something usable in the total           problems and demonstrate both that           used to conduct actual business, but
     system?
                                             users understand it and that its adoption    they need to be real in the sense that they
   ƒƒ Tools, frameworks, and standards       reduces the incidence or severity of the     offer the same interfaces and operate in
     for usable security                     associated attack. For example, demon-       the same environments as such systems.
                                             strate that a better anti-phishing scheme
Resources                                    reduces the frequency with which users       Usability competitions might be con-
                                             follow bogus links. Admittedly, this         sidered (e.g., who can come up with
Designing and implementing systems           would demonstrate success on only a          the most usable system for application/
with usable security is an enormously        single problem, but it could be used to      function X that satisfies security require-
challenging problem. It will necessitate     show that progress is both possible and      ments Y). A possible analogy would
embedding requirements for usability         demonstrable, something that many            be to the challenge of creating a more
in considerable detail throughout the        people might not otherwise believe is        usable shopping cart. Building test and
development cycle, reinforced by exten-      true about usable security.                  evaluation into the entire research and
sive evaluation of whether it was done                                                    development process is essential.
adequately. If those requirements are        What needs to be in place for
incomplete, it could seriously impair        test and evaluation?
the resulting usability. Thus, significant
resources—people, processes, and soft-       Several approaches could help:
ware development—need to be devoted
                                                ƒƒ Test and evaluation for usability
to this challenge.




                                                                                                    USABLE SECURITY               97
References
[Cra+2005]       L.F. Cranor and S. Garfinkel, editors. Security and Usability: Designing Secure Systems That People Can Use.
                 O’Reilly Media, Inc., Sebastopol, California, 2005
                 (http://www.oreilly.com/catalog/securityusability/toc.html).

[Joh2009]        Linda Johansson. Trade-offs between Usability and Security. Master’s thesis in computer
                 science, Linkoping Institute of Technology Department of Electrical Engineering,
                 LiTH-ISY-EX-3165, 2001 (http://www.accenture.com/xdoc/sv/locations/sweden/
                 pdf/Trade-offs%20Between%20Usiability%20and%20Security.pdf ).

[SOU2008]        Symposium on Usable Privacy and Security. The fourth conference
                 was July 2008 (http://cups.cs.cmu.edu/soups/2008/).

[Sun+09]         J. Sunshine, S. Edelman, H. Almuhimedi, N. Atri, and L.F. Cranor. Crying Wolf:
                 An empirical study of SSL warning effectiveness. USENIX Security 2009.

[Whi+1999]       Alma Whitten and J.D. Tygar. Why Johnny can’t encrypt: A usability evaluation of PGP 5.0.
                 In Proceedings of the 8th USENIX Security Symposium, Washington, D.C., August 23–26, 1999,
                 pp. 169–184 (http://www.usenix.org/publications/library/proceedings/sec99/whitten.html).

In addition, several other websites might be worth considering.
         http://people.ischool.berkeley.edu/~rachna/security_usability.html
         http://www.jnd.org/recommended_readings.html
         http://gaudior.net/alma/biblio.html
         http://www.laptop.org/
         http://mpt.net.nz/archive/2008/08/01/free-software-usability




98    USABLE SECURITY
Appendix A
                 Appendix A. Interdependencies Among Topics

                 This appendix considers the interdependencies among the 11 topic areas—namely,
                 which topics can benefit from advances in the other topic areas and which topics
                 are most vital to other topics. Although it is in general highly desirable to separate
                 different topic areas in a modular sense with regard to R&D efforts, it is also desir-
                 able to explicitly recognize their interdependencies and take advantage of them
                 synergistically wherever possible.

                 These interdependencies are summarized in Table A.1.


                                     Table A.1: Table of Interdependencies

                                                                   Y:
             X: Topic           1      2    3   4    5    6    7        8   9   10   11   H      M     L
             1: Scalable
                                 -     H    H   H    H    H    H        H   H   H    H    10      0    0
             Trustworthiness
             2: Enterprise
                                M       -   H   H    H    H    H        H   H   H    H     9      1    0
             Metrics
             3: Evaluation
                                H      M    -   H    H    H    H        H   H   M    H     8      2    0
             Life Cycle
             4: Combatting
                                H      M    M   -    H    M    M        H   M   M    H     4      6    0
             Insider
             5: Combatting
                                H      M    M   M    -    M    H        H   M   M    H     4      6    0
             Malware
             6: Global ID
                                H      M    M   H    H    -    M        H   H   H    H     7      3    0
             Management
             7: System
                                H      M    M   H   M     M    -        M   M   L    H     3      6    1
             Survivability
             8: Situational
                                M      M    M   H    H    M    H        -   M   M    H     4      6    0
             Awareness
             9: Provenance      M      M    M   M    H    M    M        H   -   H    H     4      6    0
             10: Privacy-
                                M      M    L   H    L    H    M        H   M   -    H     4      4    2
             Aware Security
             11: Usable
                                M      M    M   M   M     M    M        M   M   M    -     0     10    0
             Security
             H                   5     1    2   7    7    4    5        8   4   4    10   *57

             M                   5     9    7   3    2    6    5        2   6   5    0           *50

             L                   0     0    1   0    1    0    0        0   0   1    0                 *3

             * Totals for H, M, and L, for both X and Y.
             Note: H = high, M = medium, L = low. These are suggestive of the extent to which:
                      X can contribute to the success of Y.
                      Y can benefit from progress in X.
                      Y may in some way depend in the trustworthiness of X.


                                                                                                       A1
Almost every topic area has some poten-          other topic areas, most obviously       evolution must also be driven by
tial influence and/or dependence on the          including enterprise-level metrics      feedback from those other topics.
success of the other topics, as summa-           and the system evaluation life        ƒ	Combatting Insider Threats
rized in the table. The extent to which          cycle (which together could drive      (topic 4) will share some
topic X can contribute to topic Y is rep-        the definitions and assessments         common benefits with
resented by the letter H, M, or L, which         of trustworthiness), global-scale       Combatting Malware and
indicate that Topic X can make high,             identity management, system             Botnets (topic 5), particularly
medium, or low contribution to the               survivability, and usable security,
                                                                                         with respect to the development
success of Y. These ratings, of course are       but also including work on
                                                                                         and systematic use of fine-
very coarse and purely qualitative. On           combatting insider misuse and
the other hand, any finer-grained ratings                                                grained access controls and
                                                 combatting malware.
are not likely to be useful in this context.                                             audit trails. However, note
                                               ƒ	Enterprise-Level Metrics                that combatting insider threats
The purpose of the table is merely to
                                                (ELMS) (topic 2) is particularly         can contribute highly (H) to
illustrate the pervasive nature of some
                                                 interesting. It is one topic to         combatting malware, although
relatively strong interdependencies.
                                                 which all other topic areas             the reverse contributions may
A preponderance of H in a row indicates          must contribute to some extent,         be somewhat less (M). Both
that the corresponding row topic is of           because each other topic area           of these topics have significant
fundamental importance to other topics.          must explicitly include metrics         benefits for the other topics.
That is, it can contribute strongly to the       specific to that area. In the other    Also, Situational Understanding
success of most other topics.                    direction of dependence, the           (topic 8) is fundamental to both,
                                                 mere existence of thorough and          and clearly is relevant to both
Examples: rows 1 (SCAL: all H),                  well-conceived enterprise-level         insider threats and malware.
2 (METR: 9 H), 3 (EVAL: 8 H).                    metrics would drive R&D in             Thus, the potential synergies here
                                                 the individual topic areas to           will be very important.
The preponderance of H in a column               help them contribute to the
                                                                                       ƒ	Global-Scale Identity
indicates that the corresponding column          satisfaction of the enterprise-
                                                                                         Management (topic 6) and
topic is a primary beneficiary of the            level metrics. This can also
                                                                                         Provenance (topic 9) can be
other topics.                                    inspire the composability of the
                                                                                         mutually beneficial: the former
                                                 evaluation of topic metrics into
                                                                                         can significantly enhance the
Examples: columns 11 (USAB: 10 H),               the evaluation of the enterprise-
                                                                                         latter (H), whereas the latter can
8 (SITU: 8 H), 4 (INSI: 7 H),                    level metrics, which is a major
                                                                                         enhance the former somewhat
5 (MALW: 7 H).                                   research need. The enterprise-
                                                                                         less (M), although it can increase
                                                 level metrics topic area thus
                                                                                         the assurance of the former.
Not surprisingly, the table is not sym-          interacts bidirectionally with all
metric. However, there are numerous              the other topics, as exhibited by     ƒ	Survivability of Time
potential synergies here, such as the            the H entries in that row and the       Critical Systems (topic 7) is
following:                                       M entries in that column.               strongly linked with Scalable
                                                                                        Trustworthy Systems (topic 1),
   ƒƒ Scalable Trustworthy Systems              The
                                               ƒ	 System Evaluation Life                 because survivability is one
     (topic 1) is the one topic that             Cycle (topic 3) is similar to           of the fundamental aspects of
     can highly enhance all the other            Enterprise-Level Metrics (topic 2)      trustworthiness. In addition,
     topics. However, its success could          in this context. It is fundamental      it is particularly relevant to
     also derive significant benefits            to trustworthiness in almost all        combatting insider threats and
     from advances in some of the                the other topic areas, but its          malware.

A2
       INTERDEPENDENCIES AMONG TOPICS
   ƒ	Situational Understanding and           development, and operation. Failure to following topics can also contribute to
    Attack Attribution (topic 8) is          satisfy any of these requirements can advances in this topic area.
     important throughout.                   potentially undermine the trustworthi-
                                                                                       ƒƒ Enterprise-level metrics (that
   ƒ	Privacy-Aware Security                  ness of entire systems and indeed entire
                                             enterprises.                                 is, measures of trustworthiness
    (topic 10) is somewhat of an                                                          that apply to systems and
     outlier with respect to strong                                                       systems of systems as a whole):
     dependence in both directions.          To illustrate the pervasiveness of the
                                             interdependencies summarized in              Evaluation methodologies must
     It is only moderately dependent                                                      allow composability of lower-
     on other topics, and most other         Table A.1, we consider the 11 topics,
                                             in greater detail. For each topic, we        layer metrics and the resulting
     topics are only moderately                                                           evaluations. Formalization of
     dependent on it. Nevertheless,          consider first how success in the other
                                             topic areas might contribute to that         the ways in which metrics and
     it is a very important and often                                                     evaluations can compose should
     neglected broad topic area—one          particular topic (that is, represented by
                                             the corresponding column of the table),      contribute to the composability
     that is becoming increasingly                                                        of scalable systems and their
     important as more applications          and then consider how success in that
                                             particular topic might benefit the other     ensuing trustworthiness.
     become heavily dependent on the
     need for trustworthy computer           10 topics (represented by the corre-              ƒƒ System evaluation life cycle:
     systems.                                sponding rows of the table). These more             Methodologies for evaluating
                                             detailed descriptions are intended to be            security should be readily
   ƒ	Usable Security (topic 11) is           beneficial for readers who are interested           applicable to trustworthy system
     fundamental throughout. It can          in a particular column or row. They also            developments; evaluations must
     strongly influence the success of       amplify some of the concepts raised in              themselves be composable and
     almost all the other topics but is      the 11 sections of this report.                     scalable. Similar to the enterprise-
     also a critical requirement of each
                                                                                                 level metrics topic, advances in
     of those topics. Generic gains
                                                                                                 evaluation methodologies can
     in achieving usability will have        Topic 1: Scalable Trustworthy                       contribute to the composability
     enormous impact throughout, in          Systems                                             of trustworthy systems of systems.
     both directions. This is one of         We consider first how success in the
     many examples of an iterative                                                             ƒƒ Combatting insider threats:
                                             other topic areas could contribute to
     symbiotic feedback loop, where                                                              Various advances here could
                                             scalable trustworthy systems, and then
     advances in usability will help                                                             benefit scalable trustworthy
                                             how success in scalable trustworthy
     other topics, and advances in                                                               systems, including policy
                                             systems might benefit the other topic
     other topics will help usability.                                                           development, access control
                                             areas.
                                                                                                 mechanisms and policies,
The low incidence of low-order inter-
                                                                                                 containment and other forms of
dependencies in Table A.1 may at first       What capabilities from other topic areas
                                                                                                 isolation, compromise-resistant
seem odd. However, it may actually           are required or would be particularly desir-
                                                                                                 and compromise-resilient
be a testament to the relative impor-        able for effective progress in this topic area?
                                                                                                 operation, and composable
tance of each of the 11 topic areas
                                                                                                 metrics and evaluations
and the mutual synergies among the           Research on the theory and practice
topics, as well as the inherently holistic   of scalable trustworthiness is essen-               applicable to insider threats.
nature of trustworthiness [Neu2006],         tial. Although some of that research              ƒƒ Combatting malware: Advances
which ultimately requires serious atten-     must result from the pursuit of scalable            such as those in the previous
tion to all the critical requirements        trustworthy systems per se, research                topic relating to malware
throughout system architecture, system       and development experience from the                 detection and prevention can



                                                                            INTERDEPENDENCIES AMONG TOPICS                        A3
     also contribute, including the          ƒƒ Privacy-aware security: Of              directly or indirectly to almost all areas,
     existence of contained and                 considerable interest are               particularly global identity manage-
     confined execution environments            cryptographic techniques (for           ment, time-critical system survivability,
     (e.g., sandboxing), along with             example, functional encryption          provenance, privacy-aware security, and
     vulnerability analysis tools and           such as attribute-based                 usability. Usability is an example of
     composable metrics.                        encryption that is strongly             two-way interdependence: a system
 ƒƒ Identity management: Tools                  linked with access controls),           that is not scalable and not trustwor-
     for large-scale trust management           authentication, and authorization       thy is likely to be difficult to use; a
     would enhance scalability and              mechanisms that can scale               system that is not readily usable by users
     trustworthiness of systems and of          easily into distributed systems,        and administrators is not likely to be
     systems of systems.                        networks, and enterprises,              operationally trustworthy. In addition,
                                                especially if they transcend            usability would be mutually reinforc-
 ƒƒ System survivability:                                                               ing with evaluation methodologies and
                                                centralized controls and
     Availability models and                    management.                             global metrics. Other topic areas can
     techniques, self-healing trusted                                                   benefit with respect to composability
     computing bases (TCBs)                  ƒƒ Usable security: Techniques
                                                are needed for building                 and scalability. Metrics must themselves
     and subsystems, robustness                                                         be composable and scalable in order to
     analysis, composable metrics               trustworthy systems that are also
                                                usable. Thus, any advances in           be extended into enterprise-level metrics.
     and evaluations would all be                                                       Time-critical systems must compose
     beneficial to scalable trustworthy         usability can contribute to the
                                                development and maintenance             predictably with other systems. Global-
     systems.                                                                           scale identity management, of course,
                                                of trustworthiness operationally,
 ƒƒ Situational understanding                   especially if they can help with        must scale. Usability must compose
     and attack attribution: Of                 scalability.                            smoothly.
     considerable interest would be
     scalable analysis tools. Such tools   With respect to prototype systems,           More detailed technological issues relat-
     must scale in several dimensions,     systems of systems and enterprises,          ing to scalable trustworthy systems
     including number of system            testbeds and test environments are           might address questions such as the
     components, types of system           needed that can be cost-effective and        following. What fundamental building
     components, and attack time           enable timely evaluations, integrated        blocks might be useful for other topic
     scales.                               attention to interface design for internal   areas, such as insider threats, identity
                                           (developer) and external (user) inter-       management, and provenance? Can any
 ƒƒ Provenance: The ability of
                                           faces, and composability with respect        of these areas, such as usability, use these
     provenance mechanisms and
                                           to usability metrics. Methods for accu-      building blocks composably? Clearly,
     policies to scale cumulatively and    rately evaluating large-scale systems in     detailed metrics are needed for trustwor-
     iteratively to entire enterprises     testbeds of limited size would be useful,    thiness, composability, and scalability.
     and federated applications and        especially if the methods themselves can     Thoroughly documented examples are
     be maintained under large-scale       scale to larger systems.                     needed that cut across different topic
     compositions of components                                                         areas. For example, trustworthy separa-
     that would enhance scalable           How does progress in this area support       tion kernels, virtual machine monitors,
     trustworthiness overall. Such         advances in others?                          and secure routing represent areas of
     mechanisms must be tamper             Overall, this topic area has significant     considerable interest for the future.
     resistant, providing abilities for    impact on each of the other areas. Scal-
     both protection and detection.        able composability would contribute




A4
      INTERDEPENDENCIES AMONG TOPICS
                                                ƒƒ Scalable trustworthy systems                 over insider misuse can also
Topic 2: Enterprise-Level
                                                   would help address remote                    help prevent or at least limit the
Metrics (ELMs)
                                                   access by logical insiders as                deleterious effects of malware.
What capabilities from other topic areas
are required for effective progress in this        well as local access by physical             The prevention aspects are closely
topic area?                                        insiders, by virtue of distributed           related.
                                                   authentication, authorization,
Each of the other topic areas is expected                                                    ƒƒ Life cycle protection must
                                                   and accountability.
to define local metrics relevant to its                                                         account for the insider threat.
own area. Those local metrics are likely        ƒƒ Situational understanding and
                                                                                             ƒƒ Survivability of systems can
to influence the enterprise-level metrics.         attack attribution must apply to
                                                                                                be aided by knowledge of the
                                                   insiders as well as other attackers.
                                                                                                presence of potential malware or
How does progress in this area support             This dependency implies that
                                                                                                of insiders who may have been
advances in others?                                synergy is required between
                                                                                                detected in potential misuse.
Proactive establishment of sensible                misuse detection systems and the
enterprise-level metrics would natu-               access controls used to minimize
                                                                                          Topic 5: Combatting Malware
rally tend to drive refinements of the             insider misuse.
                                                                                          and Botnets
local metrics.
                                                ƒƒ Identity management relates            What capabilities from other topic areas
                                                   to the accountability aspects of       are required for effective progress in this
Topic 3: System Evaluation                         the insider threat, as well as to      topic area?
Life Cycle                                         remote access by insiders.
What capabilities from other topic areas                                                  Malware is a principal mechanism
are required for effective progress in this     ƒƒ Malware can be used by insiders        whereby machines are taken over for
topic area?                                        or could act as an insider on          botnets. Significant progress in the
Advances in scalability, composability,            behalf of an outside actor. Thus,      malware area will go far toward enabling
and overall system trustworthiness are             malware prevention can help            effective botnet mitigation. Economic
likely to contribute to the development            combat insider threats.                analysis of adversary markets supports
of scalable, composable evaluation meth-        ƒƒ Provenance can also help combat        this area, as well as botnet defense, and
odologies, and suggest some synergistic            insider threats. For example,          may provide background intelligence
evolution. Metrics that facilitate evalu-          strong information provenance          in support of situational understanding.
ation will also contribute significantly.          can help detect instances where
                                                                                          How does progress in this area support
                                                   insiders improperly altered
How does progress in this area support                                                    advances in others?
                                                   critical data.
advances in others?                                                                       Progress in the area of inherently secure
Effective evaluation methodologies can          ƒƒ Privacy-aware security requires        systems that can be thoroughly moni-
provide major benefits to all the other            knowledge of insiders who were         tored and audited will benefit other
topics. Otherwise, the absence of such             detected in misuse, as well as         topics, especially situational understand-
methodologies leaves significant doubts.           mechanisms for privacy.                ing. Attribution also links this topic to
                                                                                          situational understanding. Advances in
Topic 4: Combatting Insider                   How does progress in this area support      detection enable malware repositories,
Threats                                       advances in others?
                                                                                          which can be mined to identify families
What capabilities from other topic areas        ƒƒ Progress in combatting insider         and histories of malware, which in turn
are required for effective progress in this        threats will support advances          may make attribution possible.
topic area?                                        in privacy and survivability for
Several dependencies on other topic                time-critical systems, as well as      Collaborative detection may depend
areas are particularly relevant:                   conventional systems. Controls         on progress in global-scale identity



                                                                         INTERDEPENDENCIES AMONG TOPICS                            A5
management, to prevent adversaries from   of identity-laden information. It could       tion would make it significantly harder
thwarting such an approach through        simplify security evaluations. It could       for an attacker to avoid attribution. This
spoofed information.                      also reduce the proliferation of malware      depends on progress in global-scale
                                          if identities, credentials, authentication,   identity management.
Progress in security metrics is likely to authorization, and accountability were
make it easier to evaluate the effective- systematically enforced on objects and        Subsystems for detecting and combat-
ness of proposed solutions to malware other computational entities.                     ting malware must be designed to
problems.                                                                               enhance situational understanding and
                                            Topic 7: Survivability of Time              attack attribution. Local malware, of
Topic 6: Global-Scale Identity                                                          course, is a serious problem. However,
                                            Critical Systems
                                                                                        botnets and the malware that can com-
Management                                  What capabilities from other topic areas    promise unsuspecting systems to make
What capabilities from other topic areas    are required for effective progress in this them part of botnets are adversarial
are required for effective progress in this topic area?
                                                                                        enablers supporting important classes
topic area?
                                            Advances in the development of scal- of attacks for which situational under-
Scalable trustworthy systems are essen- able trustworthy systems would have standing is critical. Attribution in the
tial to provide a sound basis for global immediate benefits for system surviv- case of botnets is difficult because the
identity management. Privacy-aware ability. Basic advances in usability launch points for attacks are them-
security could be highly beneficial. For could help enormously in reducing the selves victimized machines, and the
example, assurance that remote creden- burdens on system operators and system adversaries are becoming more adept
tials are in fact what they purport to be administrators of survivable systems. at concealing their control channels
would help. In addition, analyses, simu- Advances in situational understanding and “motherships” (e.g., via encryption,
lations, and data aggregation using real would also be beneficial in remediating environmental sensing, and fast-flux
data require strong privacy preservation survivability failures and compromises. techniques [ICANN 2008, Holz 2008]).
and some anonymization or sanitiza-
tion. Provenance will be important How does progress in this topic area sup-            Advances in privacy-aware security
for increasing the trustworthiness and port advances in others?                         (particularly with respect to privacy-
reputations of remote identities. Usabil- Concise and complete requirements aware sharing of security-relevant
ity is fundamental, of course for users for survivability would greatly enhance information) would address many of
as well as administrators. Survivability enterprise-level metrics and contribute the hurdles to sharing as considered in
of identity management systems will be to the effectiveness of evaluation meth- this topic area.
critical especially, in real-time control odologies. They would also improve the
and transactional systems.                  development of scalable trustworthy The measures of success enumerated
                                            systems overall, because of the many below require fundamental advances
How does progress in this topic area sup-   commonalities between survivability, in metrics definition, collection, and
port advances in others?                    security, and reliability.                  evaluation.
Identity management would contrib-
                                                                                           ƒƒ Synthetic attacks (emulating the
ute to the trustworthiness of large-scale Topic 8: Situational
networked systems and certainly help                                                          best current understanding of
                                            Understanding and Attack                          adversary tactics) provide some
in reducing insider misuse, particu-
                                            Attribution                                       metrics for attribution. Possible
larly by privileged insiders who are
accessing systems remotely. It would What capabilities from other topic areas                 metrics include time to detect,
also enhance privacy-preserving secu- are required for effective progress in this             how close to the true origin
rity—for example, because assurances topic area?                                              of the attack (adversary and
are required whenever there is sharing Effective authentication and authoriza-                location), and the rate of fast flux

A6
       INTERDEPENDENCIES AMONG TOPICS
     that can be tolerated while still   mitigation draw on advances in surviv- can completely undermine would-be
     being able to follow the adversary  ability, for example.                     solutions. Global-scale identity man-
     assets.                                                                       agement is essential for enterprise-wide
   ƒƒ We should examine metrics          Topic 9: Provenance                       privacy. Usability is essential, because
      related to human factors to assess What capabilities from other topic areas  otherwise mechanisms tend to be
      effectiveness of presentation                                                misused or bypassed and policies tend
                                         would facilitate progress in this topic
      approaches.                                                                  to be flouted. Situational understand-
                                         area?
                                                                                   ing and attack attribution, as well as
   ƒƒ We should explore metrics
                                         Provenance is dependent on most of the the ability to combat malware, may be
      for information sharing—for        other topics and most of the other topics somewhat less important but still can
      example, the tradeoff between      are dependent on provenance, but a few contribute to the detection of privacy
      how much the sharer reveals        topics have more direct connections. violations.
      versus how actionable the          Global-scale identity management
      community perceives the shared                                               How does progress in this area support
                                         is required to track authorship as well
      data to be. This issue may touch                                             advances in others?
                                         as chain-of-custody through informa-
      on sharing marketplaces and        tion processing systems. Privacy-aware Global-scale identity management
      reputation systems.                security is highly relevant to the dis- can benefit—for example, by being
   ƒƒ The current state of metrics with  semination of provenance information. shown how to build identity manage-
      respect to adversary nets and fast Scalable trustworthiness is essential ment systems that protect privacy. The
      flux are not adequately known.     to trustworthy provenance. Usability system evaluation life cycle can benefit
      We should examine how SANS         would be important as well.               from provenance. To some extent, this
      and similar organizations collect                                            topic can influence requirements for
      measurement data.                  How does progress in this area support    how scalable trustworthy systems are
                                         advances in others?                       designed and developed.
How does progress in this area support   Trustworthy provenance would con-
advances in others?                      tribute significantly to combatting Topic 11: Usable Security
                                         malware and to situational under- What capabilities from other topic areas
For many attack situations of inter- standing. It could also contribute are required for effective progress in this
est, advances in analysis and attack to privacy-aware security. It would topic area?
taxonomy would also support malware provide considerable improvements in              ƒƒ Identity management: Large-
defense and therefore mitigate botnets. system usability overall.                        scale identity management
Systems that are intrinsically monitor-                                                  systems could solve one of the
able and auditable would presumably Topic 10: Privacy-Aware                              most vexing security problems
be easier to defend and less prone to                                                    users face today—namely, how
                                         Security
malware.                                 What capabilities from other topic areas        to establish trust between
                                             are required for effective progress in this     and among users and systems,
Advances in attribution to the ultimate      topic area?                                     particularly within systems
attack source would support advances                                                         and networks that are easy to
in defense against botnets and other         Information provenance is needed
                                                                                             use by ordinary users and by
attacks where the immediate launch           for many different privacy mechanisms
                                                                                             administrators.
point of the attack is itself a victimized   applied to data. Scalable trustworthy
machine.                                     systems are needed to ensure the integ-       ƒƒ Survivability of time-
                                             rity of the privacy mechanisms and              critical systems: Advances in
This topic and the survivability area        policies. Combatting insider threats            availability directly enhance
are mutually reinforcing. Reaction and       is essential, because otherwise insiders        usability, especially whenever


                                                                           INTERDEPENDENCIES AMONG TOPICS                     A7
     manageability of configurations       ƒƒ Privacy-aware security: As            How does progress in this area support
     and remediation of potentially          with the other topics, this topic      advances in others?
     dangerous system configurations         must address usability as a core       Usability goes hand in hand with the
     are included in the design and          requirement.                           other topic areas; without success in
     operation of those systems.                                                    usability, the benefits of progress in the
                                           ƒƒ Malware: Technology that
 ƒƒ Scalable trustworthy systems:                                                   other areas may be diminished. This
                                             neutralizes the threat posed
     Large-scale systems that are                                                   applies directly to each of the other topic
                                             by malware would be of great
     trustworthy must, by the                                                       areas, more or less bilaterally. Usabil-
                                             benefit to usability, since it could
     definition of the usability                                                    ity considerations must be addressed
                                             eliminate any need for users to
     problem, be usable, or they                                                    pervasively.
                                             think about malware at all.
     will not be trustworthy, either
     architecturally or operationally.     ƒƒ Metrics and evaluation: The
 ƒƒ Provenance: Automated tools              ability to know how well we are
     for tracking provenance could           doing in making secure systems
     enhance usability by reducing           usable (and usable systems that
     the need for users to consider          maintain security) would be
     explicitly the source of the            useful; a usable system lets you
     information they are dealing            know whether you got things
     with.                                   right or wrong.


Reference
[Neu2006]        Peter G. Neumann. Holistic systems. ACM SIGSOFT Software Engineering Notes 31(6):4-5, November 2006.




A8
      INTERDEPENDENCIES AMONG TOPICS
Appendix B
             Appendix B. Technology Transfer

             This appendix considers approaches for transitioning the results of R&D on the
             11 topic areas into deployable systems and into the mainstream of readily available
             trustworthy systems.

             B.1 Introduction
             R&D programs, including cyber security R&D, consistently have difficulty in
             taking the research through a path of development, testing, evaluation, and tran-
             sition into operational environments. Past experience shows that transition plans
             developed and applied early in the life cycle of the research program, with prob-
             able transition paths for the research products, are effective in achieving successful
             transfer from research to application and use. It is equally important, however, to
             acknowledge that these plans are subject to change and must be reviewed often.
             It is also important to note that different technologies are better suited for differ-
             ent technology transition paths; in some instances, the choice of the transition
             path will mean success or failure for the ultimate product. Guiding principles for
             transitioning research products involve lessons learned about the effects of time/
             schedule, budgets, customer or end-user participation, demonstrations, testing and
             evaluation, product partnerships, and other factors.

             A July 2007 Department of Defense Report to Congress on Technology Transition
             noted evidence that a chasm exists between the DoD S&T communities focused
             on demonstration of a component and/or breadboard validation in a relevant
             environment and acquisition of a system prototype demonstration in an opera-
             tional environment. DoD is not the only government agency that struggles with
             technology transition. That chasm, commonly referred to as the valley of death,
             can be bridged only through cooperative efforts and investments by research and
             development communities as well as acquisition communities.

             In order to achieve the full potential of R&D, technology transfer needs to be a
             key consideration for all R&D investments. This requires the federal government
             to move past working models in which most R&D programs support only limited
             operational evaluations/experiments, most R&D program managers consider their
             job done with final reports, and most research performers consider their job done
             with publications. Government-funded R&D activities need to focus on the real end
             goal, namely technology transfer, which follows transition. Current R&D Principal
             Investigators (PIs) and Program Managers (PMs) are not rewarded for technology
             transfer. Academic PIs are rewarded for publications, not technology transfer. The
             government R&D community needs to reward government program managers
             and PIs for transition progress.

             There are at least five canonical transition paths for research funded by the
             Federal Government. These transition paths are affected by the nature of the
             technology, the intended end-user, participants in the research program, and

                                                                                               B1
other external circumstances. Success B.2 Fundamental Issues for                       concepts discussed in this topic area into
in research product transition is often Technology Transition                          the mainstream of education, training,
accomplished by the dedication of the                                                  experience, and practice will be essential.
program manager through opportunistic What are likely effective ways to trans-
channels of demonstration, partnering, fer the technology?                             B.3 Topic-Specific
and occasional good fortune. However,                                                  Considerations
no single approach is more effective than There is no one-size-fits-all approach       In this section, certain issues that are
a proactive technology champion who to technology transfer. Each of the 11             specific to each of the 11 topics are
is allowed the freedom to seek potential topic areas will have its own special         considered briefly.
utilization of the research product. The considerations for effective transitioning.
five canonical transition paths can be For example, effective transitioning will       Topic 1: Scalable Trustworthy Systems
identified simply, as follows:            depend to some extent on the relevant
                                          customer bases and the specific applica-     Easy scalability, pervasive trustworthi-
    ƒƒ Department/Agency direct to        tions. However, this section considers       ness, and predictable composability all
       Acquisition (Direct)               what might be common to most of the          require significant and fundamental
    ƒƒ Department/Agency to               11 topics. A few issues that are specific    changes in how systems are developed,
       Government Lab (Lab)               to each topic are discussed subsequently.    maintained, and operated. Therefore,
     ƒƒ Department/Agency to Industry                                                  this topic clearly will require consider-
       (Industry)                          It will be particularly important that able public-private collaboration among
                                           the results (such as new systems, mecha- government, industry, and academia,
     ƒƒ Department/Agency to Academia
                                           nisms, policies, and other approaches) with some extraordinary economic,
       to Industry (Start-up)              be deployable incrementally, wherever social, and technological forcing func-
     ƒƒ Department/Agency to Open          appropriate.                                tions (see Section B.4). The marketplace
       Source Community (Open                                                          has generally failed to adapt to needs for
       Source)                             Technologies that are to be deployed trustworthiness in critical applications.
                                           on a global scale will require some
Many government agencies and com-          innovative approaches to licensing and Topic 2: Enterprise-Level Metrics
mercial companies use a measure known      sharing of intellectual property, and (ELMs)
as a Technology Readiness Level (TRL).     serious planning for test, evaluation,
The TRL is a term for discussing the       and incremental deployment. They will This is perhaps a better-mousetrap
maturity of a technology, to assess the    also require extensive commitments to analogy: if enterprise-level metrics
maturity of evolving technologies (mate-   sound system architectures, software were well developed and able to be
rials, components, devices, etc.) prior    engineering disciplines, and commit- readily evaluated (topic 3), we might
to incorporating that technology into      ment to adequate assurance.                 presume the world would make a beaten
a system or subsystem. Whereas this                                                    path to their door. Such metrics need
mechanism is primarily used within         Carefully documented worked examples to be experimentally evaluated and
the DoD, it can be considered a rea-       would be enormously helpful, especially their practical benefits clearly demon-
sonable guideline for new technologies     if they are scalable. Clearly, the concepts strated, initially in prototype system
for almost any department or agency.       addressed in this document need to environments and ultimately in realistic
Table B.1 lists the various technology     become a pervasive part of education large-scale applications.
readiness levels and descriptions from     and training. To this end, relevant R&D
a systems approach for both hardware       must be explicitly oriented toward real
and software.                              applicability. Furthermore, bringing the




B2
        TECHNOLOGY TRANSFER
                                   Table B1: Typical Technology Readiness Levels

TechnologyƒReadinessƒLevel                                    Description
1. Basic principles observed and reported.                    Lowest level of technology readiness. Scientific research begins to be
                                                              translated into applied research and development. Examples might
                                                              include paper studies of a technology’s basic properties.
2. Technology concept and/or application formulated.          Invention begins. Once basic principles are observed, practical
                                                              applications can be invented. Applications are speculative and there
                                                              may be no proof or detailed analysis to support the assumptions.
                                                              Examples are limited to analytic studies.
3. Analytical and experimental critical function and/or       Active research and development is initiated. This includes analytical
characteristic proof of concept.                              studies and laboratory studies to physically validate analytical
                                                              predictions of separate elements of the technology. Examples include
                                                              components that are not yet integrated or representative.

4. Component and/or breadboard validation in laboratory       Basic technological components are integrated to establish that they
environment.                                                  will work together. This is relatively “low fidelity” compared to the
                                                              eventual system. Examples include integration of “ad hoc” hardware in
                                                              the laboratory.
5. Component and/or breadboard validation in relevant         Fidelity of breadboard technology increases significantly. The basic
environment.                                                  technological components are integrated with reasonably realistic
                                                              supporting elements so it can be tested in a simulated environment.
                                                              Examples include “high fidelity” laboratory integration of components.

6. System/subsystem model or prototype demonstration in a     Representative model or prototype system, which is well beyond that
relevant environment.                                         of TRL 5, is tested in a relevant environment. Represents a major step
                                                              up in a technology’s demonstrated readiness. Examples include testing
                                                              a prototype in a high-fidelity laboratory environment or in simulated
                                                              operational environment.
7. System prototype demonstration in an operational           Prototype near, or at, planned operational system. Represents a major
environment.                                                  step up from TRL 6, requiring demonstration of an actual system
                                                              prototype in an operational environment such as an aircraft, vehicle, or
                                                              space. Examples include testing the prototype in a test bed aircraft.

8. Actual system completed and qualified through test and     Technology has been proven to work in its final form and under
demonstration.                                                expected conditions. In almost all cases, this TRL represents the end of
                                                              true system development. Examples include developmental test and
                                                              evaluation of the system in its intended weapon system to determine if
                                                              it meets design specifications.
9. Actual system proven through successful mission operations. Actual application of the technology in its final form and under
                                                               mission conditions, such as those encountered in operational test
                                                               and evaluation. Examples include using the system under operational
                                                               mission conditions.




                                                                                            TECHNOLOGY TRANSFER                        B3
Topic 3: System Evaluation Life              system survivability requires an           encourage and fund research and
Cycle                                        overarching commitment to system           development relating to all of the
Similarly, if effective evaluation meth-     trustworthiness that must transcend        topics considered here, with particular
odologies could be developed, their          what has been done in the past.            emphasis on trustworthy systems, com-
usefulness would need to be clearly dem-                                                posability, scalability, and evolutionary
onstrated on real systems, as in topic 2.    Topic 8: Situational Understanding         system architectures. It also needs to
Thoroughly specified and relatively          and Attack Attribution                     encourage the incorporation of source-
complete requirements would also be          R&D in this area has been slow to          available and nonproprietary systems
required. Given a few well-documented        find its way into commercial products.     that can demonstrably contribute to
demonstrations of effectiveness, the         Recognition of the pervasive needs for     trustworthiness.
incentives for technology transfer would     monitoring and accountability would
be greatly increased.                        be of great value.                         Academic research needs to pursue theo-
                                                                                        ries and supporting tools that enable
Topic 4: Combatting Insider Threats          Topic 9: Provenance                        systematic development of composable
Once again, the proof is in the pudding.     Provenance would be very useful in and scalable trustworthy systems and all
Demonstrations of the effectiveness          finance, government, health care, and the other topics discussed here.
of approaches that combat insider            many other application areas, and
misuse would encourage adoption of           would facilitate forensics.                Commercial developers need to instill a
the techniques.                                                                         more proactive discipline of principled
                                             Topic 10: Privacy-Aware Security           system developments that allow interop-
Topic 5: Combatting Malware and              Advances in this topic could be particu- erability among different systems and
Botnets                                      larly useful in many application areas, subsystems, that employ much better
As noted in Appendix A, the com-             such as health care, financial records, software engineering practices, that
monalities among insider threats and         communication logs, and so on.             result in trustworthy systems that are
malware suggest that demonstrations                                                     more composable and scalable, and that
of the effectiveness of approaches that      Topic 11: Usable Security                  provide cost-effective approaches for all
combat malware are likely to be rapidly      Almost anything that significantly the topics discussed here.
and widely adopted in practice.              increased the usability of security and
                                             helped manage its inherent complex- Topic 4: Combatting Insider Threats
Topic 6: Global-Scale Identity Man-          ity would be likely to find its way into Governments need to establish base-
agement                                      practice fairly readily.                   lines and standards. Legal issues
It will be important to design mech-                                                    relating to trap-based defensive strate-
anisms and policies that can be                                                         gies and entrapment law should be
incrementally deployed. Technologies         B.4 Forcing Functions (Some                addressed. Applying these to the many
that are to be deployed on a global scale    Illustrative Examples)                     real situations in government activity
will require some innovative approaches      For several of the 11 topics, this section where insider behavior is a genuine
to licensing and sharing intellectual        addresses the question What are the threat would be beneficial. Current
properties, and serious planning for test,   appropriate roles for government, aca- government efforts to standardize on
evaluation, and incremental deployment.      demia, industry, and markets? Many authentication and authorization (e.g.,
                                             of the suggested forcing functions are the Common Access Card) are worth-
Topic 7: Survivability of Time Criti-        applicable in other topics as well.        while despite their potential limitations,
cal Systems                                                                             particularly in helping combat insider
R&D communities have long under-             Topic 1: Scalable Trustworthy Sys-         misuse. Academia needs to pursue
stood how to take advantage of               tems                                       R&D that is realistically relevant to the
fault-tolerance mechanisms. However,         The federal government needs to insider threat. Industry research needs

B4
      TECHNOLOGY TRANSFER
to be more closely allied with the needs eat its own dog food, establishing sound    ƒƒ Provide suitable funding for basic
of practical systems with fine-grained identity management mechanisms and               research in usable security.
access controls and monitoring facilities. policies, and adhering to them.           ƒƒ Encourage interdisciplinary
Industry is also the most likely source of                                              research in usable security.
data sets that contain instances of insider Academia needs to recognize more
                                                                                     ƒƒ Adopt usability reviews for
misbehavior, or at least more detailed widely the realistic problems of global
                                                                                        security research.
knowledge of some kind on how real identity management and to embed
insider misbehavior tends to manifest more holistic and realistic approaches         ƒƒ Establish appropriate standards,
itself. The marketplace needs to be into research.                                      criteria, and best practices.
responsive to customers demanding                                                    ƒƒ Pervasively embed usability
better system solutions. Note also the Industry needs to recognize the enor-            requirements into the
possible relevance of HSPD-12 PIV-I mous need for interoperability within               procurement process.
and PIV-II.                                 multivendor and multinational feder-
                                                                                     ƒƒ Reconsider security policies from
                                            ated systems.
                                                                                        a usability perspective.
Various incentive structures might be
considered:                                 The marketplace needs to anticipate      ƒƒ Ensure that usable security is
                                            long-term needs and somehow inspire         a criteria for evaluating NSA
   ƒƒ Business cases as incentive           governments, academia, and industry         centers of academic excellence.
      (investment vs. potential cost)       to realize the importance of realistic      (This will provide an incentive
   ƒƒ Insurance as financial protection     approaches.                                 to get usability into the
      against insiders                                                                  curriculum.)

   ƒƒ Major players in the bonding          Topic 11: Usable Security              Academia
      markets, who might possibly                                                    ƒƒ Incorporate usability pervasively
                                            Government
      provide data for research                                                         into computer system curricula.
                                            Remove impediments to usability
      in exchange for better loss-                                                   ƒƒ Lead by example by making their
                                            research. For example, federal law
      reduction approaches                                                              own systems more usably secure.
                                            currently requires review before data
  ƒƒ Nonfinancial incentives, as in        can be used in an experiment/study;      ƒƒ Incorporate usability into the
     the FAA near-miss reporting,          simply having the data in your posses-      research culture by demanding
     granting some sort of immunity        sion does not give you the right to use     that system security research
     (but being careful not to shoot       it (e.g., e-mail you have received and      papers and proposals always
     the whistle-blowers)                  wish to use to test a new spam filtering    address issues of usability.
                                           algorithm); Minimize administrative
  ƒƒ International efforts might
                                           burdens; making sure Institutional Industry
     include bilateral and multilateral                                             ƒƒ Develop standards for usable
                                           Review Boards (IRBs) are familiar with
     quid-pro-quo cooperations.                                                        security.
                                           the unique aspects of usable security
                                           research (especially as contrasted, for  ƒƒ Develop consistent terminology.
                                           example medical research); and create
Topic 6: Global-Scale Identity Man-                                                 ƒƒ Identify best practices.
                                           mechanisms for expediting usability
agement
                                           research approval.                       ƒƒ Contribute deployment
Governments need to unify some of                                                      experience. (Provide feedback to
the conflicting requirements relating to      ƒƒ Avoid inappropriate restrictions      the research community: what
identity management, credentials, and            that prevent government entities      works and what does not.)
privacy. The U.S. government needs to            from participating in research.



                                                                                    TECHNOLOGY TRANSFER                B5
Appendix C
Appendix C. List of Participants in the Roadmap Development
We are very grateful to many people who contributed to the development of this roadmap for cybersecurity research,
development, test, and evaluation. Everyone who participated in at least one of the five workshops is listed here.

Deb Agarwal                          Bob Hutchinson                         William H. Sanders
Tom Anderson                         Cynthia Irvine                          Mark Schertler
Paul Barford                         Markus Jakobsson                        Fred Schneider
Steven M. Bellovin                   David Jevans                            Kent Seamons
Terry Benzel                         Richard Kemmerer                       John Sebes
Gary Bridges                         Carl Landwehr                           Frederick T. Sheldon
KC Claffy                            Karl Levitt                             Ben Shneiderman
Ben Cook                             Jun Li                                  Pete Sholander
Lorrie Cranor                        Pat Lincoln                             Robert Simson
Rob Cunningham                       Ulf Lindqvist                           Dawn Song
David Dagon                          Teresa Lunt                            Joe St Sauver
Claudiu Danilov                      Doug Maughan                            Sal Stolfo
Steve Dawson                         Jenny McNeill                           Paul Syverson
Drew Dean                            Miles McQueen                           Kevin Thompson
Jeremy Epstein                       Wayne Meitzler                          Gene Tsudik
Sonia Fahmy                          Jennifer Mekis                          Zach Tudor
Rich Feiertag                        Jelena Mirkovic                        Al Valdes
Stefano Foresti                      Ilya Mironov                           Jamie Van Randwyk
Deb Frincke                          John Mitchell                          Jim Waldo
Simson Garfinkel                     John Muir                               Nick Weaver
Mark Graff                           Deirdre Mulligan                        Rick Wesson
Josh Grosh                           Clifford Neuman                         Greg Wigton
Minaxi Gupta                         Peter Neumann                           Bill Woodcock
Tom Haigh                            David Nicol                             Bill Worley
Carl Hauser                          Chris Papadopoulos                      Stephen Yau
Jeri Hessman                         Vern Paxson                             Mary Ellen Zurko
James Horning                        Peter Reiher
James Hughes                         Robin Roy




                                                                                                               C1
Appendix D
Appendix D. Acronyms

A/V          antivirus
AMI          Advanced Metering Infrastructure
BGP          Border Gateway Protocol
C2           command and control
CAC          Common Access Card
CAPTCHA      Completely Automated Public Turing test to tell Computers and Humans Apart
CASSEE       computer automated secure software engineering environment
CERTs        Computer Emergency Response Teams
CMCS         Collaboratory for Multi-scale Chemical Science
COTS         commercial off-the-shelf
CUI          Controlled Unclassified Information
CVS          Concurrent Versions System
DAC          discretionary access controls
DARPA        Defense Advanced Research Projects Agency
DDoS         distributed denial of service
DETER        cyber-DEfense Technology Experimental Research
DHS          Department of Homeland Security
DKIM         DomainKeys Identified Mail
DNS          Domain Name System
DNSSEC       DNS Security Extensions
DoS          denial of service
DRM          digital rights management
ESSW         Earth System Science Workbench
EU           European Union
FIPS         Federal Information Processing Standards
FISMA        Federal Information Security Management Act
GPS          Global Positioning System
HDM          Hierarchical Development Methodology
HIPAA        Health Insurance Portability and Accountability Act
HSI          human-system interaction
HVM          hardware virtual machine
I&A          identification and authentication
I3P          Institute for Information Infrastructure Protection
IDA          Institute for Defense Analyses
IDE          integrated development environment
IDS          intrusion detection system
INL          Idaho National Laboratory
IPS          intrusion prevention system


                                                                                          D1
IPsec     Internet Protocol Security
IPv4      Internet Protocol Version 4
IPv6      Internet Protocol Version 6
IRB       institutional review board
ISP       Internet service provider
IT        information technology
LPWA      Lucent Personalized Web Assistant
MAC       mandatory access controls
MIT       Massachusetts Institute of Technology
MLS       multilevel security
MTBF      mean time between failures
NIST      National Institute of Standards and Technology
NOC       network operations center
OODA      Observe, Orient, Decide, Act
OS        operating system
OTP       one-time password
P2P       peer-to-peer
P3P       Platform for Privacy Preferences
PDA       personal digital assistant
PGP       Pretty Good Privacy
PII       personally identifiable information
PIR       private information retrieval
PKI       public key infrastructure
PL        programming language
PMAF      Pedigree Management and Assessment Framework
PSOS      Provably Secure Operating System
PREDICT   Protected Repository for the Defense of Infrastructure against Cyber Threats
QoP       Quality of Protection
RBAC      role-based access control
RBN       Russian Business Network
RFID      radio frequency identification
ROM       read-only memory
SBU       Sensitive But Unclassified
SCADA     Supervisory Control and Data Acquisition
SCAP      Security Content Automation Protocol
SIEM      security information and event management
SOHO      small office/home office
SPF       sender permitted from
SQL       Structured Query Language
SRS       Self-Regenerative Systems
SSL       Secure Sockets Layer
T&E       test and evaluation
TCB       trusted computing base
TCP/IP    Transmission Control Protocol/Internet Protocol
TLD       top-level domain
TPM       Trusted Platform Module
TSoS      trustworthy systems of systems
UI        user interface
UIUC      University of Chicago at Urbana-Champaign
USB       universal serial bus
US-CERT   United States Computer Emergency Readiness Team
VM        virtual machine
VMM       Virtual Machine Monitor

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:81
posted:8/4/2011
language:English
pages:126