Virtually Pwned by bestt571


More Info
									“You Are Doing It WRONG”

 Failures in Virtualization
      BlackHat Europe 2011

          Claudio Criscione, CTO

                March 2011

                Version 1.0
PART 1                                                  For instance, that we don't need bleeding edge
                                                        attacks to own Virtualization infrastructures. In
                                                        some previous works I've shown how critical
IN       WHICH         I     RANT         ABOUT         systems can be taken over by rather low-profile
VIRTUALIZATION SECURITY                                 attacks such as path traversal in application
                                                        servers, when combined with unsecure
                                                        practices, like session IDs stored in log files. We
                                                        keep seeing old attacks compromising new
"I Learn in This Letter That Don Peter of
                                                        technologies, and we can’t simply expect them to
Arragon, Comes This Night to Messina."
                                                        “go away” by themselves.
     -   Much Ado About Nothing
                                                        Furthermore, we find some "interesting behavior"
When looking at the state of virtualization             in the way vendors in this market are handling
security from an hacker perspective, there are          security. Virtualization infrastructures are highly
many different possible considerations.                 critical systems, often very hard to update from a
                                                        version to another. As such, one would expect
First and foremost, virtualization infrastructures      vendors to provide backporting for anything even
are good targets. Own the virtualization server,        remotely security related, to guarantee security
own an average of 8 VMs. Own a central node             to an as large as possible user base. However, if
and you get even more, in the same way in               we look at CTX127541, an XSS in Citrix Web
which owning a Domain Controller will get you           Interface (the one that is used to manage Xen
the whole Active Directory in a Microsoft world.        based infrastructures) we discover that while all
Yet, while Domain Controllers are today slimmer         versions from 5.0 to 5.3 are vulnerable, a patch
and slimmer, with less and less services to             has only been provided to 5.4, with no
attack, virtualization systems are nowadays the         backporting, effectively leaving older versions
opposite: fat, feature-full targets which are slowly    unprotected. One might argue that this is a
moving.                                                 commercial move, pushing for upgrades, or
                                                        maybe that these older versions are very
We thus have an interesting target with a large         outdated and should be upgraded anyway. True,
attack surface: this is where we start wondering        and yet a question should arise.
what the actual level of security of such critical
and wide platforms is, and make interesting             What is going to happen when the security team
findings.                                               asks for a version upgrade to patch a medium
                                                        level bug such as an XSS?
For instance, we find that similar issues can pop
multiple times on the same “target”, or                 The answer is trivial, but this question is actually
infrastructure. For instance, if a trivial XSS attack   the tip of the iceberg in the virtualization security
is identified in a critical web interface it gets       sphere.
patched, and we're fine with that. However,             On one side, we have an IT task force in charge
Secure Development best practices would                 of virtualization or cloud computing who has just
suggest that, since a given vulnerability was           managed to free itself from the burden of many
identified in the codebase, it should be checked        policies and restrictions that the security team (or
for more instances of the same.                         simply the enterprise environment) has usually
                                                        placed upon them, thanks to the possibility of
CVE-2009-2277, CVE-2009-3731, CVE-2010-                 managing the whole stack, including network
1137 and CVE-2010-1193 are a random sample              and applications, which comes with virtualization.
of XSSes affecting the same web interface (or
slight variations of it) in VMware products. Said       On the other, the team in charge of security is
interface is, without any doubt, a very large           losing more and more control and (to a degree)
software, but the repetition of the same class of       visibility as virtual machines are stored,
bug in just a few pages is telling us something.        templates are created and solutions to run on-

                                                            See “Virtually Pwned” from Black Hat US 2010

You are doing it wrong – Failures in virtualization systems                                           1
machine firewalls and security systems are                    innovations from a security perspective, at
deployed through the virtualization platform.                 least at the moment.
Portions of said platform are even truncated from            Bugs and errors are being repeated, and
the organization itself as they are moved to                  the general huge push for the lowest
virtual private cloud or similar solutions. It’s not          possible time to market is leading to sub-
only a matter of skills and mindset, but also an              optimal      products     from     a    security
issue of control and influence spheres.                       perspective. No specific vendor is to blame,
                                                              but the whole industry.
Segregation of duties is left behind, and it's               Security logics in virtualization infrastructure
harder and harder to draw a clear line on                     are still those of the traditional "old times",
security rules and requirements for business                  and       the     struggle     between       the
services powered by virtual infrastructures. This             IT/virtualization team and the security team
is not the current scenario, but some of the most             is going to be lost every time by the latter,
advanced infrastructures are heading exactly                  since the former will produce actual benefits
there.                                                        for the company.
What is the IT team going to state, as a security            If you add cloud to the mix, you have more
requirement? "Thou shall patch and upgrade"?                  of the same (and thus an even grimmer
"Thou shall not run virtual machines on thy                   landscape).
hosts"? Even stating the new requirements is a
                                                       This is where we are, or at least a vision of
difficult and unsolved task, and there is a definite
                                                       where we are. Where do we go from here?
lack of tools in the segment. Just one example:
there is today no tool available to ensure that the
time flow is always as expected, that is
preventing administrators from using old
versions of a given virtual machine. Is that an
issue? Think about restarting a system with a
worm inside a network which has just been
cleaned, and you can easily answer that

Even from a design perspective, the lack of
security awareness in the field is showing in flat
designs with little more than VLANs in place to
ensure segregation – when that happens – and
lot of trust put in an encryption which is not yet
delivering the level of security it should.
What’s basically happening is that virtualization
servers are being treated just like any other
system: access to management interfaces might
be restricted and backend communication
networks created, but that’s more or like it. This,
as it seems, is simply not enough to keep them
secure in the wake of any random exploit.

Let's sum up what we discussed until now in an
orderly fashion:

    Virtualization infrastructures are core
     systems, perceived as highly complex (often
     for a good reason) but linked to rather
     simple technologies and with only lesser

You are doing it wrong – Failures in virtualization systems                                            2
PART 2                                                    in-the-cloud, mind you, but something which can
                                                          be used with secure design to help guarantee
                                                          the security of virtualized infrastructures.
COULD TRY TO FIX IT                                       Let's try to sort out what are the problems we are
                                                          trying to fix – pardon, improve:
"Before We Proceed Any Further, Hear Me
Speak."                                                    1.   Provide a semantic to enforce rules on
                                                                "services" rather than simple access
    -    Coriolanus                                             control rules on virtual machines or groups
                                                                as it is happening today
"Fixing" a situation is a daunting task: most of           2.   Apply these rules on services or logical
the times, it will require at least as much time as             items rather than single physical systems
getting it "broken" in the first place. The fact is        3.   Return the control of security related
that we are not looking at a "Broken" landscape:                issues back to the security guys
virtualization    is   working      and   delivering
consistently, even from a security point of view,         In order to solve these problems, we need to
just like enterprise technologies are supposed to         introduce a new concept. Indeed, every rule or
do. Only, as "security people", we look at it and         constriction has to apply to an "object" of some
think "there is something wrong going on". For            sort. In the virtualization space we do have some
the sake of discussion, thus, we will speak about         "objects" ready to use: Hosts, Folders, Virtual
"improving" security from now on.                         Machines, Users (or Security Principals),
                                                          Datacenters. However, all these concepts are, in
This said, how do we "improve" security?                  one way or another, restricted to a given
                                                          implementation or to a given specific item. For
Let's look at history. We can easily identify
                                                          our purposes, we need something different, a
multiple times in IT where we faced similar
                                                          logical item which can - in some cases - be
challenges: a technology which is not really
                                                          directly mapped to an actual implementation.
ready from a security perspective, but which we
have to use, and which is often outside the               We will call these logical items "virtual Cells", or
control of the security guys. Think about the way         vcells: a logical object which we can define and
networks have grown, and network daemons                  enforce rules on.
have mated and spread. When the Morris Worm
hit, the net was not ready for that, we had no            What we want, now, is an item which is able to
rules in place to avoid or block the attack. Then         enforce a rather heterogeneous set of business
we invented the firewall: a way to enforce rules.         rules on these yet-to-be-defined vcells.
This is what the firewall is all about, for the
network: a way for the "security guys" to say "it         An item which is able to apply these rules
does not matter what you are going to do, mr              regardless of the security status of the
sysadmin, your packets are not going to get               infrastructure it is defending (or with an high level
there" or, in a slightly lesser BOFH way, "packets        of resilience): it should be able to work even if
from attackers are not going to get here even if          large     parts     of   the    infrastructure    are
you misconfigure our servers".                            compromised, in the same way firewalls still
                                                          enforce network restrictions if the servers are
We all know the rest of the history, the drift            compromised.
toward HTTP and the rise of the application
firewalls (even though they're still far behind           We will call this item with yet another arbitrary
when compared with network firewall in terms of           name: vGatekeeper.
actual usefulness).
                                                            Even if a vcell is completely compromised,
To make a long story short, what I am proposing              the vgatekeeper will prevent any action
is a logical enforcer for rules in virtualization            which has not been allowed (like migration
environments: a virtualization and cloud firewall,           of virtual machines) from occurring between
if you wish - not a virtualized firewall or a firewall-      vcells and, possibly. Inside the vcell itself.

You are doing it wrong – Failures in virtualization systems                                             3
   That is, the security breach will be limited to
   that vcell - this is what we will call the
   "cauterization" requirement.
  vCells could be placed anywhere and
   running on a number of platforms at a time,
   managed by different APIs. However, the
   vGatekeeper should be able to enforce a
   consistent set of security rules, translating
   them in the appropriate “dialect”. Of course,
   this will clearly create a problem of
   semantic. This is the "agnosticism"
  Access to the vCells should be allowed only
   through the vGatekeeper in a measurable
   and enforceable matter, in the same way
   traffic has to flow through a firewall. This is
   the "central" requirement.
  Moving to more functional requirements we
   expect the vGatekeeper to be able, at the
   very least, to limit the actions which can be
   performed on a given service or vcell.
   Mapping this to a more general approach,
   we want the vGatekeeper to provide
   Mandatory Access Control toward vCells,
   seeing as security principals either "the
   world" or other logical elements. Valid
   approaches include simple "time based"
   grants or more sophisticated systems, but
   this access and authorization approach is
   added to the existing authorization system
   provided by the virtualization infrastructure
   we're protecting.

To these specific requirements, we should add
those typical of any enterprise security system:
scalability, high availability, accountability and so
on. Moreover, since we are to interact with
various systems and cells over disparate
networks, we want to make sure that in every
step we can provide communication and storage

You are doing it wrong – Failures in virtualization systems   4
PART 3                                                    the status of the backend. Even if an intruder
                                                          was able to forge completely fake requests, he
                                                          would only be able to get those which the
IN WHICH WE ACTUALLY (TRY TO)                             vGatekeeper deems respectful of the rules
DO IT                                                     through. Similarly, VM or data migration has to
                                                          be initiated and pivoted through the gatekeeper,
“[…]you shall see, as I have said, great                  thus ensuring that even if a cell is compromised
difference betwixt our Bohemia and your Sicilia”          (as opposed to the management node) the
                                                          others are safe.
    -    The Winter’s Tale
                                                          AGNOSTICISM: Every API has its own sets of
We have now made it clear what we expect from
                                                          rules, and leveraging this simple feature allows
our vGatekeeper in terms of a set of
                                                          us to be completely platform agnostic.
                                                          CENTRAL: Passwords and authentication
So, how do we do it?
                                                          means to access the virtual cells, like the root
In this paper, I will not be presenting a                 passwords of hosts or authentication tokens for
production grade enterprise solution. Yet, a              cloud solutions, are always pivoted through the
simple proof of concept can be designed and               vGatewkeeper instead of being available on the
thought of to present the core idea.                      client, and authentication is performed between
                                                          the vGatekeeper and the user with different
In order to do so, we can simply leverage the             credentials (plannned)
fact that modern virtualization and cloud
computing systems all use simple SOAP based               Since every connection is performed through a
APIs to manage their infrastructure. They might           network, the position of our vGatekeeper can be
be fully documented or not (like in the case of           chosen freely: a vGatekeeper-in-the-cloud is
VMware's vCenter calls) yet this is a common              perfectly possible, and even a reasonable
factor for all the most widespread solutions.             approach since it would pose it outside the
                                                          administrative domain of those he is actively
Our sample vGatekeeper (VASTOKeeper) will                 restricting (the IT team).
thus simply be a rather simple web application
firewall based on standard, opensource                    With a vGatekeeper, Security has once again a
solutions: its goal will be to intercept any and all      tool to enforce system wide security policies on a
commands forwarded by the management point                given     environment,    protecting     the    IT
- be it a client (either with a single management         infrastructure from external threats… and, this
target or multiple targets as XenCenter) or a             time, also from itself!
central node like VMware vCenter - to the vcells
it will be protecting. In time, it will also be able to   CONCLUSIONS
monitor the flow of commands inside the vCells.
System design will also have to accommodate               In this paper we detailed a different approach to
this need, and shall be tweaked accordingly. We           virtualization security in real-world systems,
know that some architectures (most notably                calling for a new component (the vGatekeeper)
VMware also leverage older, legacy protocols,             and a new design concept (the vCell), which can
and will take them into account as well                   help applying security concepts in a segment
whenevere possible.                                       badly in need of new management tools.

How does this solution respect (most of) our

CAUTERIZATION        Since  every and       all
commands have to be forwarded through the
vGatekeeper, mandatory access control is
enforced through the vGatekeeper regardless of

You are doing it wrong – Failures in virtualization systems                                          5

To top