Strategies for Delivering Dynamic Security for Virtual Data Centers

CMP Virtualization Security Strategies for Delivering Dynamic Security for Virtual Data Centers Introduction Playbook 2 Securing Your Virtualized Data Center Fritz Nelson CMP Technology A Compendium Feature Articles of Recent Articles by CMP Editors InformationWeek Virtualization’s Next Frontier: Security Larry Greenemeier . . . . . . . . . . . . . . . . 8 Fighting Security Ghosts in the Virtual Machine Larry Greenemeier . . . . . . . . . . . . . . 10 Virtualization Security Heats up Joe Hernick . . . . . . . . . . . . . . . . . . . . 14 Network Computing Gartner: Virtualization Can Weaken Security Kelly Jackson Higgins . . . . . . . . . . . . . .3 A Look at Blue Lane VirtualShield Avi Baumstein . . . . . . . . . . . . . . . . . .21 Taking Virtualization Security Seriously Art Wittmann . . . . . . . . . . . . . . . . . . .12 Dark Reading VMs Create Potential Risks Kelly Jackson Higgins . . . . . . . . . . . . . .4 New Tool: Virtual Tip of the Iceberg Kelly Jackson Higgins . . . . . . . . . . . . . .6 From the editors of: Sponsored by: BACK TO HOME Securing Your Virtualized Data Center Virtualization has become all the rage for companies interested in optimizing their servers and storage systems. This shift in data center architecture allows IT organizations to reduce operational costs and increase flexibility and responsiveness to changing business demands. But in this rush to virtualize, security can fall by the wayside. And this lack of security is likely preventing many enterprises from experiencing the full benefits of this major network transformation. Organizations must be mindful that virtual servers require different security measures. Traditional, static security measures such as firewalls, VLANs and other specialized security appliances simply don’t make the grade in this new world of virtualization, so enterprises have to seek out alternatives if they want to create the next generation data center. Instead of rigid security walls, virtualized server environments need flexible virtual shields. Shields are dynamic. Shields can move with the servers they protect, so they can be brought quickly to the point of attack. And shields can be orchestrated ondemand by a management system. To get a third-party perspective on the topic, Network Computing provides “Gartner: Virtualization Can Weaken Security,” which presents research that suggest that virtual servers will be less secure than physical servers through 2009. According to the Dark Reading article “VMs Create Potential Risks,” there’s no guarantee that your security policies from physical servers will carry over to your virtual ones. The article looks closely at several of the biggest security risks and how to mitigate the threats to your systems. InformationWeek’s article “Virtualization’s Next Frontier: Security” elaborates on the pros and cons of virtualization security and some of the solutions currently on the market. All of this may sound grim for enterprise IT departments, but there is a bright side. A number of vendors are addressing exactly this issue of security for virtual resources. As we see in Network Computing’s in-depth review, “A Look at Blue Lane VirtualShield,” there are products on the market that will eliminate malicious content from network traffic before it hits your virtual servers. As you transition to a virtualized environment, turn to the Virtualization Security Playbook, brought to you by CMP Technology, AMD and Blue Lane, as your go-to resource for all the latest information about securing your most valuable IT assets against increasingly aggressive attacks. From the editors of: Fritz Nelson Senior Vice President CMP Integrated Marketing Solutions 2 SECURE VIRTUALIZATION PLAYBOOK BACK TO HOME networkcomputing.com, April 5, 2007 Gartner: Virtualization Can Weaken Security Research firms says companies that adopt virtualization without having best practices in place run the risk of jeopardizing enterprise security April 5, 2007 By Amy Larsen DeCarlo Gartner will present research later this month that suggests that companies that hurry to implement virtualization technology without first implementing best practices for security may be in for trouble. The analyst firm said the combination of immature security tools for virtualized environments and the failure of companies to set and carry out appropriate policies to protect virtual machines (VMs) means that these virtual servers will be less secure than physical machines through 2009. As is the case with any new technology that becomes an obvious target for security threats, Gartner said companies need to proceed with caution as they deploy VMs. The research firm suggested that too many businesses try to take the same approach to securing their virtual servers that they use to protect physical servers. This leaves VMs exposed to threats. Gartner said effective security for virtualized environments ideally should begin before an organization even picks vendors or products. Neil MacDonald, vice president and Gartner Fellow, will present Gartner’s findings at the Gartner Symposium/ITxpo 2007: Emerging Trends, being held in San Francisco from April 22nd to April 26th. Let’s catalog Gartner’s observations first: 1. Virtualization software—such as hypervisors—represents a new layer of privileged software that will be attacked and must be protected. 2. The loss of separation of duties for administrative tasks, which can lead to a breakdown of defense in-depth. 3. Patching, signature updates, and protection from tampering for offline VM and VM “appliance” images. 4. Patching and secure confirmation management of VM appliances where the underlying OS and configuration are not accessible. 5. Limited visibility into the host OS and virtual network to find vulnerabilities and assess correct configuration. 6. Restricted view into inter-VM traffic for inspection by intrusion prevention systems (IPSs). 7. Mobile VMs will require security policy and settings to migrate with them. 8. Immature and incomplete security and management tools. Their observations to fall into three categories: 1) Not new to virtualization (3,4—same as regular hardware appliances and other hotspare devices) 2) Useful, but obvious (1,7,8) 3) Only true if the admins aren’t paying attention, or the architecture is poorly designed (2, 5, 6) Incidentally, BlueLane—with their “virtual IPS”—has got to be ecstatic with No.6 since they’re the only vendors marketed squarely at that space right now. Jordan Wiens NWC Contributing Technology Editor 3 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME darkreading.com February 21, 2007 VMs Create Potential Risks Convenient and efficient, virtual machines can also increase your security exposure FEBRUARY 21, 2007 By Kelly Jackson Higgins machines. IBM won’t give specifics on its internal plans for the technology, but it has provided some elements of SHype to the Xen Open Source Project. And VMWare’s desktop Ace software lets you lock down virtual machines, even when they are moved around. Lin says it works like a network access control (NAC) for virtual machines. Thomas Ptacek, a security researcher with Matasano Security, says the move to virtualization is the biggest thing happening in IT today. “And every application running in a modern IT organization is on a path to being moved to one big iron [machine] running multiple VMs,” he says. “And its impact on security touches everything.” That has prompted some security vendors to step up. Allwyn Sequeira, senior vice president of product operations for Blue Lane Technologies, says virtualization is creating a tornado of forces that could blow away security as we know it: the new hypervisor layer creating a new security attack vector; VM sprawl, where virtual machines become unmanaged and unprotected, and rogue VMs emerge; and in the dynamic moves and changes model of VM, patching and testing cycles get disrupted, mismatched, and complex. “Some say virtualization of the OS doesn’t change Those tens of thousands of virtual servers spawned from your thousands of physical ones offer no guarantee your security policies will carry over, and can leave you with a security time bomb ticking away in your data center, according to vendors and some experts. “Virtualization is both an opportunity and a threat,” says Patrick Lin, senior director of product management for VMWare. “But one of the key things about hypervisors is their design is simpler than the modern operating system. As a result, they are simpler to harden and lock down, and there are not as many vulnerabilities.” “On the flip side, it’s a new layer that’s another opportunity for attack,” he says. Hypervisors are programs that allow multiple operating systems to use the same hardware. But these programs can also breed complexity, and with complexity comes security problems. Virtualization security solutions so far have been focused mostly on the hypervisor: IBM, for instance, recently unveiled SHype, a new secure hypervisor technology that ties security policies to virtual 4 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME darkreading.com February 21, 2007 anything, and others, that with virtualization, everything is broken,” Sequeira says. “I think it’s somewhere in the middle.” Sequeira wouldn’t comment on Blue Lane’s upcoming announcement in the virtualization space, but there’s a pretty big hint on its Website: “Coming soon: Virtual Security for VMware’s ESX hypervisor.” Security analysts caution against over-reaction. “We are still very early on in the whole virtualization thing,” says Michael Rothman, president of Security Incite. “The real threats end up being more theoretical than real now... But over time, this will become a real challenge.” “It also changes the definition of what is an application, where does the data reside,” he says. “And how do I end up securing it? To me, this is going to create another situation where you have to look at security from the outside in.” And virtualization technology itself isn’t inherently insecure. There are more vulnerabilities in your operating system than in your virtualization software, such as VMware, Matasano’s Ptacek notes. It’s more about how you configure your virtual architecture, where the virtual machine software is the main barrier among the different apps sharing the same physical machine. “So you design it the same way you design a secure network. You partition sensitive applications to sensitive servers, and consolidate the less sensitive ones onto the same hardware.” Ptacek says the biggest security risk with virtualiza- tion to watch out for are “guest-to-guest attacks,” where an attacker gets the root or administrator privileges on the hardware, and then can hop from one virtual machine to another. “This assumes the attacker has already broken into one of the machines,” he says, and it puts other apps at risk on the compromised machine. But this type of attack isn’t easy to execute, especially when cryptography is involved. But with a lot of math, an attacker can “infer some bits of the crypto secret,” Ptacek adds. There hasn’t been much activity in VM bug disclosures as yet, but that will change as more apps use virtualization, he says, and then attackers will have fresh vulnerabilities to prey upon as well. The underlying problem: Virtualization creates a set of dynamics in the IT infrastructure that traditional security approaches “don’t cope with well,” says Kevin Leahy, director of virtualization at IBM. “As you move virtual resources around, it adds another layer or dimension of complexity to it. You want to ensure that all security policies, access privileges, etc., that you put on the box originally will also be true with the new ’box’ you put it on.” VMWare’s Lin agrees that security should be part of the virtualization design. Aside from Ace, VMWare offers a virtual Layer 2 switch in its ESX Server – a hypervisor – with VLAN support, and the VMWare software lets you lock down virtual network adapters “from running in promiscuous mode,” he says. VMWare also offers resource management and the ability to set user permissions. 5 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME darkreading.com March12, 2007 New Tool: Virtual Tip of the Iceberg Blue Lane’s new security software addresses one of the gaping security holes in VMs, more tools expected March 12, 2007 By Kelly Jackson Higgins Starting to get worried about the exposure of those thousands of virtual servers in your data center? The bad news is no one knows for sure what security threats lurk in the virtual world. The good news is, security tools are finally starting to emerge. (See VMs Create Potential Risks.) Blue Lane Technologies on Thursday will release software that basically shields a virtual machine while it automatically detects vulnerabilities and applies patches. VirtualShield runs on VMWare’s ESX server, sitting between the hypervisor and the VMs. “They [Blue Lane] are taking their network-based shielding product and bringing it inside the virtual server,” says Neil MacDonald, a vice president at Gartner and a Gartner Fellow. “It buys you time and lets you bring it [the virtual machine] up and shield it.” But Blue Lane’s new VirtualShield doesn’t secure the hypervisor, the program that lets multiple operating systems use the same hardware. Hypervisors are considered a potential security hazard due to their complexity. “Its [VirtualShield’s] value is that it proactively shields VMs on that box,” MacDonald says. “One of the holes it protects you against is if you bring up a partition offline of disk-based VM, the chances are, you are not up to date in patches... It lets you bring it online and proactively shield it until you are able to apply the patches. “Blue Lane is not protecting the hypervisor, but the hosted workloads,” he says. Blue Lane joins a tiny group of vendors that provide security for VMs, including Reflex Security, which sells a network-based IPS for VMs. MacDonald says these smaller vendors are naturally the early birds in securing VM servers, but he expects larger players like Cisco, Juniper, TippingPoint, and others to jump in the game as well at some point. MacDonald earlier this month published a report for Gartner revealing that organizations rushing to virtualize their servers end up “unknowingly” weakening security, and that the offline patching of VMs and appliances has not been fully addressed by security vendors. The Gartner report says virtual machines may be convenient, but they also bring with them “embedded vulnerabilities and require special consideration for patching and updates.” Gartner 6 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME darkreading.com March12, 2007 recommends building security into VM implementations, and watching out for the common security “holes” in VM environments: • The separation of duties for administrative tasks, which can lead to opening security holes in VMs • Patching, signature updates, and protection from tampering with offline VM and VM “appliance” images • Limited view into the host operating system and virtual network, which prevents finding vulnerabilities Limited view for IPSes of inter-VM traffic • Security policies and settings don’t necessarily follow mobile VMs “With virtualization, just like with RFID and Web 2.0, security wasn’t baked in from the beginning,” Gartner’s MacDonald says. “It was an afterthought.” And if enterprises aren’t careful, they will be paying for that oversight. Virtualization security is still largely an unknown, too: “It’s definitely something a lot of folks, including the bad guys, are watching closely,” says Michael Rothman, president of Security Incite. And it’s still unclear just what a security virtualization product “needs to be,” he says. Blue Lane’s VirtualShield addresses a piece of the puzzle, he says. “I think Blue Lane is off to a good start because their inherent inline patching approach for the non-virtualized world is applicable to the virtualized world,” Rothman says. “Just think about a data center in a box, with the network fully contained in one, or many, chassis, and that’s virtualization. So the idea of fixing things at the network layer, even with a flexible definition of the network, is interesting.” But even this inline patching comes with risks. “Is ’inline patching’ the best way to do it? I can’t say definitively, when comparing to IPS or any other technology that can block attacks at the network layer,” he says. “It’s not clear how those other technologies will map to the virtualized world, so Blue Lane has a leg up there.” Rothman says the next 12 months will feature security vendors jockeying for position with products in this space, as the virtualization security problems become clearer. Gregory Ness, vice president of marketing for Blue Lane, says VirtualShield provides a plug-in to the hypervisor. “It’s a new kind of virtual ’appliance.’” VirtualShield automatically “discovers” the VMs, and shields them while grabbing updates and patches via Blue Lane’s subscription patch service. It secures VMWare ESX virtual servers, including OS and running applications, both online and offline, according to Blue Lane. Pricing for VirtualShield and VirtualShield Manager software is $499, which includes a one-year subscription to Blue Lane’s update service as well as online support. 7 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME InformationWeek, March 17, 2007 Virtualization’s Next Frontier: Security Virtual machines can improve a system’s security, but beware of the many pitfalls. March 17, 2007 By Larry Greenemeier The good news about virtual machines is that they’re easy to set up, they can run a variety of operating systems and applications on the same host, and they can isolate different workloads. That’s also the bad news, particularly when it comes to protecting your proliferating virtual servers from attack. Time to seize on virtualization’s ability to improve security while avoiding its security pitfalls. Blue Lane Technologies last week introduced the equivalent of an intrusion-prevention system for virtual machines running the VMware Infrastructure 3 platform. Its VirtualShield software, which sits between the host system’s hypervisor and its virtual machines, is designed to block malware from reaching the VMs, which are vulnerable if their applications don’t have the latest patches. VirtualShield “plays zone defense” for all of a server’s virtual machines rather than guarding each one individually, says Allwyn Sequeira, senior VP of product operations for Blue Lane. “We emulate the behavior of a patch so you don’t have to touch every server, although we’re not replacing the patch itself,” he says. About two-thirds of the 150 IT executives recently surveyed by InformationWeek say their companies are implementing server virtualization. Deployments will only grow as Linux players ratchet up their support. Red Hat has added the Xen open source hypervisor to its Enterprise Linux version 5, introduced last week. Also last week, Novell said that users of SAP NetWeaver and the mySAP Business Suite can implement instances of that software on virtual machines running on its SUSE Linux Enterprise Server 10, which ships with Xen. IBM has also contributed to virtualization security by 8 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME InformationWeek, March 17, 2007 developing an extension called sHype that ties security policies to virtual servers. In a virtualized environment, IP addresses change as virtual machines are created, disbanded, or moved from one physical server to another. Because most security is designed to associate an IP address with a location, it becomes harder for firewalls and intrusion-prevention systems to recognize the need to protect virtual servers, says Andreas Antonopoulos, an analyst with Nemertes Research. “That’s not a problem with virtualization; it’s a problem with security,” he adds. FEAR OF INFECTION A big concern for Paul Asadoorian, lead IT security engineer at Brown University, is the possibility that one compromised virtual machine could infect all VMs on a server. “So many people have their servers connected to a private network but still allow Web surfing from a virtual machine on that server,” he says, a situation that defeats the purpose of closing a server off to the public network. One product, Reflex Security’s Virtual Security Appliance, creates and enforces secu- rity policies between virtual servers and even virtual networks. Virtual machines can, in fact, improve a system’s security. When they’re set up to run different applications within a host server, they can keep buffer overflow attacks from bringing down the entire server. That’s because each virtual machine is allocated a certain amount of memory space and can’t steal memory from an application running in another VM. Virtualization also aids in disaster recovery by making IT environments more portable, says Burlington Coat Factory CTO Michael Prince. Another virtue of virtual server security is the ability to run multiple operating systems on the same server, creating a more diverse environment that can’t be shut down by malware that targets Windows or Linux. Blue Lane’s VirtualShield buys companies time until they can patch the applications and operating systems on their virtual servers. It may not solve all of virtualization’s security challenges, but it’s a step in the right direction. 9 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME InformationWeek, March 15, 2007 Fighting Security Ghosts in the Virtual Machine Blue Lane Technologies debuts an intrusion-prevention system for virtual machines running on the VMware Infrastructure 3 platform. March 15, 2007 By Larry Greenemeier every server, although we're not replacing the patch itself.” VirtualShield isn't the only virtualization security out there, but its timing is just right. About twothirds of the 150 senior IT executives recently surveyed by InformationWeek Research say their companies are implementing server virtualization. And IDC reports that more than two-thirds of all U.S. companies with 1,000 or more employees are currently deploying virtualization technology. That will only grow as key Linux players ratchet up their support for virtualization. Red Hat earlier this week released Red Hat Enterprise Linux version 5 with Xen's open source hypervisor. Also this week, rival Novell announced that users ofSAP( SAP) NetWeaver and mySAP Business Suite can now implement instances of that software on virtual machines running SUSE Linux Enterprise Server 10, which ships with open source Xen. IBM has also contributed to virtualization security by developing a security extension called sHype that ties security policies to virtual machines. IBM has also contributed some of its sHype code to Xen. In a virtualized environment, IP addresses change as virtual machines are created, disbanded, or moved from one physical server to another. Because most security is designed to associate an IP address with a location, it becomes harder for firewalls and intrusion- The unfettered growth of virtual machines means it's time to seize on the technology's capacity to improve security while avoiding new security pitfalls. That's exactly what Blue Lane Technologies had in mind Thursday when it introduced the equivalent of an intrusion-prevention system for virtual machines running on the VMware Infrastructure 3 platform. The company's VirtualShield software sits between the host server's hypervisor and its virtual machines and is designed to block malware from reaching virtual machines, which could be vulnerable to being exploited if their applications don't have the latest patches. “It puts a force field in front of server images; that was the 'a-ha' behind the product,” says Blue Lane president and CEO Jeff Palmer, adding that, although VirtualShield is currently available only for VMware, there's nothing keeping Blue Lane from developing VirtualShield for other vendors' hypervisors. Put another way, VirtualShield “plays zone defense” for all of a system's virtual machines rather than guarding each one individually,” says Allwyn Sequeira, senior VP of product operations for Blue Lane Technologies. “We emulate the behavior of a patch, so you don't have to touch 10 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME InformationWeek, March 15, 2007 prevention systems to recognize the need to protect virtual machines, says Andreas Antonopoulos, an analyst with Nemertes Research. “That's not a problem with virtualization; it's a problem with security,” he adds. A big concern for Paul Asadoorian, lead IT security engineer at Brown University, is the possibility that one compromised virtual machine could infect all virtual machines on a server. “So many people have their servers connected to a private network but still allow Web surfing from a virtual machine on that server,” he says, a situation that defeats the purpose of closing a server off to the public network. Reflex Security identified this problem and sells its Virtual Security Appliance, which creates and enforces security policies between virtual machines or even virtual networks. In fact, virtual machines can improve a system's security in several ways. Virtual machines set up to run different applications within a host server can keep buffer overflow attacks from bringing down the entire server. That's because each virtual machine is allocated a certain amount of memory space and can't steal memory from an application running in another virtual machine. Virtualization also aids in disaster recovery by making IT environments “more portable,” says Burlington Coat Factory CTO Michael Prince. Given that disaster recovery is increasingly being considered a part of a company's security program, virtualization improves security by making it easier to recreate an IT environment damaged during an emergency replace crashed systems in an emergency, he adds. Another aspect of virtual machine security that can't be overlooked is the ability to run multiple operating systems within the same server, creating a more diverse environment that can't be completely shut down by malware designed to specifically target Windows or Linux. VirtualShield's strength is that it buys companies time until they can patch the applications and operating systems on their virtual machines. It may not solve all of the security challenges that virtualization brings, but it's a step in the right direction. 11 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME networkcomputing.com, August 24, 2007 Taking Virtualization Security Seriously August 24, 2007 By Art Wittmann can fully protect virtual machines from one another. If it can simultaneously protect VMs and hosted applications against buffer, stack and heap overflow exploits, who wouldn't be interested in that? Determina's second product was called LiveShield. The idea behind it is to stop exploits on the fly - no need to reboot the server, just apply the patch in memory. Certainly this is right up VMware's alley as the technology isn't too far from its own binary emulation system, which rewrites parts of executable code as it loads. While the idea of patching a running OS or application sounds interesting, it doesn't alleviate the need to test patches before they're applied. Usually, it's that testing that slows down the process - and not usually the need to bounce the server. We've talked a lot about Blue Lane's patch emulation products (there's a physical appliance version as well as a virtual appliance for VMware). The idea is to catch incoming attacks and make the fix that an actual patch might do before the offending packet ever gets near the actual server. While the company has had its share of naysayers, the Blue Lane products performed as claimed when we tested them in Virtualization security has been on the minds of a lot of IT folks lately. There's no doubt that virtualization changes the security game - and because it involves new software - the potential for new exploits exists. The clever folks at VMware understand this and, as seems to be their practice, quietly bought a company that can help. Determina, which it bought a couple of weeks ago, had a couple of products; I say had because it looks like VMware was just after the technology. Rumor is that most of Determina, including sales, marketing and executives, was not retained after the purchase, and VMware won't sell the Determina products as stand-alone offerings. Its memory firewall protects against stack and heap overflow exploits. And while that's a pretty narrow protection goal, it's an important one. The problem is that for some applications, the Determina memory firewall could put a dent in overall performance. Still, where VMware needs to make a case is that it 12 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME networkcomputing.com, August 24, 2007 our Florida Real World Lab. It got another seal of approval this week from none other than Microsoft itself, which tested the physical appliance and found it fully interoperable with Microsoft's protocols. In the virtual world, the combination of the three products addresses many of the concerns currently expressed for security. The memory firewall will protect against overflow exploits, while Blue Lane's technology gives IT the time it needs to properly test patches and, once tested, LiveShield let's you apply them on the fly. There's another interesting angle here: the brewing battle between Microsoft and VMware, and how companies like Blue Lane could get caught in the middle. More on that soon. 13 SECURE VIRTUALIZATION PLAYBOOK BACK TO HOME informationweek.com, September 1, 2007 Virtualization Security Heats Up An attack that breaches the hypervisor is IT's new worst nightmare. Are you prepared? September 1, 2007 By Joe Hernick In March, Gartner ignited the blogosphere by stating the obvious: Virtualization creates new attack opportunities. There's still lots of smoke billowing around, but only time will tell how much fire is behind it, and who's fanning the flames. Vendors of new virtualized security “appliances” clearly have a stake. But many enterprises are realizing they rushed headlong into virtualization without considering the impact on their data protection policies, so IT pros do have legitimate concerns over the amount of real estate that could be consumed by a successful attack on a hypervisor. If you're squirming right now, the big question you want answered is: Just how risk-exposed are we today? After all, in that same report Gartner predicted that a patch-worthy hypervisor vulnerability would be discovered in a mainstream product before the end of 2008. These potential vulnerabilities fall into two broad categories. First, if you can escape a client OS and move into a host OS, you have access to the data on all the other client operating systems on that machine. And there are whole new realms of rootkits being designed to take advantage of virtualization technology. “People have been working on breaking out of the guest OS in VMware for some time now,” says Greg Shipley, CTO of security consulting firm Neohapsis and an InformationWeek contributor. “And having a hypervisor rootkit installed would be a serious threat to any org. However, I don't see the development of the rootkit being the big challenge.” It's the process used to deploy such a rootkit that really intrigues Shipley. “What's going to require more effort: Researching a vulnerability that allows us to break out of a guest OS and gain control of the hypervisor layer, or going after an administrator and hijacking the credentials required to install the rootkit, just like any other application? If the task was on my plate, I know which route I'd go.” 14 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME informationweek.com, September 1, 2007 servers, but there are added gotchas, from intrahost threats to vetting third-party hypervisor driver add-ons to new checklist items for corporate information security policies. Let's face it: If a traditional 1U server is compromised, you'll feel some amount of personal shame, regroup, assess damages, fix the problem, and move on. Most shops have strategies in place to localize internal damage, with secondary and tertiary lines of defense to safeguard against a cascade of compromised systems. Problem is, few network monitoring and management tools are up to the task of securing guest VMs. When a traditional server gets slammed and begins displaying erratic or suspicious behavior, alarms will go off. But how effective are your tried-and-true netmon tools if all machine-to-machine communication is occurring between VMs inside your “data center in a box”? How much time will the bad guys have to probe, test, and exploit intrahost weaknesses before you see what's happening? And is the current level of high security anxiety swirling around VM-specific environments justified? It's getting there, but that's the nice thing about smelling smoke--it warns that danger's afoot. FIRM FOUNDATION To weigh theoretical risks as well as where new applications of old attack points are feasible, you need to understand the underlying design of virtualized hosts. Virtualization creates an abstraction layer separating guest operating systems from underlying hardware, enabling multiple VMs to be hosted on a single server. Virtual machines may rely on trim hypervisors using small, privileged code bases as the foundation for this abstraction; the strength of this approach is that performance of hosted apps can reach near-native levels. As for breaking out of the client image, consulting company Intelguardians demonstrated just such an incursion into the host OS at last month's SANSFire show. Details of the vulnerability aren't public, so it's impossible to know what the attack was successful against, but you can bet these researchers aren't the only ones in this race. The lesson is that organizations now need to assume that a sufficiently motivated attacker is capable of such an exploit, and plan accordingly. Defense in depth and proper virtual machine layout and design, including not mixing VMs with different security postures and requirements on the same host system, are crucial. To find out how prepared our readers are, we fielded a survey--and got some eye-popping results. We can't help thinking that the 43% saying they believe virtualized machines are just as safe and secure as traditional environments are whistling past the graveyard. Of the 384 IT operations and security professionals responding, a mere 12% have put formal strategies in place to protect their VMs. Now, many say they're relying on their current IT policies and toolsets to manage and protect virtual servers, and that makes sense ... to a point. Virtualized environments do face the same operational threats and risks as traditional 15 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME informationweek.com, September 1, 2007 technology, and XenEnterprise has endured the pokes and prods of the open source community, earning a Common Criteria Level 5 rating. Chip designers and VM software vendors also are working to stay on top of the security struggle. Steve Grobman, Intel's director of business-client architecture, says Intel VT-X server and desktop virtualization offerings are designed from the ground up to strengthen security. For example, Intel's current VT-enhanced server chipsets offer three new layers of code privilege for virtualization on top of the traditional three layers of CPU code privilege. Products targeting the enterprise server market, including VMware ESX, Intel VPro, Virtual Iron, and XenEnterprise, favor a hypervisor design. Alternatively, desktop VMs and Microsoft's virtual server offerings use a traditional “fat OS” model, where guest VMs ride atop full-fledged hosting operating systems. While hypervisors provide optimized performance and a reduced attack surface, they also bring new vulnerabilities to the party and so need to have security baked in from the beginning rather than added as an afterthought. The million-dollar question here: Is it safer to rely on the open source community to vet and test Xen, or are VMware and other vendors of proprietary hypervisors the best path to secure hosts? “From what I've seen, VMware's QA is pretty darn good,” Shipley says. “They look like rock stars compared with many other companies. How many patches has Oracle come out with this year? I lost track as they approached triple digits.” Meanwhile, XenEnterprise's upcoming 4.0 hypervisor will weigh in at a trim 60,000 or so lines of code, XenSource CTO Simon Crosby says. Less code equals fewer potential bugs. Moreover, XenSource, which recently agreed to be acquired by Citrix Systems, uses IBM's secure hypervisor Of course, VMware owns the enterprise virtualization market, and the company is feeling pretty secure. “Design, testing, and implementation of VMware ESX server contrasts with traditional, larger-platform operating systems,” says Mendel Rosenblum, a VMware co-founder and chief scientist. “VMware has been focused on security concerns from our first line of code. I am 100% confident that we will not have a hypervisor compromise due to a design flaw.” We certainly hope his certitude proves warranted, and indeed, vendors have been successful thus far. That noise was us knocking on wood. DOOMSDAY TIME The worst-case scenario in a hypervisor-based hosted environment? Hyperjacking, where an exploit leads to a compromised platform, allowing criminals full access to all hosted guests on a given machine. In subverting the hypervisor, malicious software could easily disguise its presence from traditional security tools that reside in either hosted-OS partitions or on any software layer above the hypervisor. The exploit situation is analogous to the threat of cloaked rootkits compromising a standalone 16 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME informationweek.com, September 1, 2007 server OS. If you own the hypervisor, you own all data traversing the hypervisor and are in a position to sample, redirect, or spoof anything you please. Without some form of fail-safe, guest operating systems would have no way of knowing they're running on a compromised platform. This is the stuff of nightmares when you're talking large-scale virtualization platforms that offer 10, 50, even hundreds of hosted servers running on a single piece of hardware. The potential risk for loss of control and revenue is enormous. The answer is to maintain the integrity of the hypervisor while building in multiple fail-safes so hosted operating systems can ensure that they're communicating with an untainted hypervisor as a bridge to the underlying hardware and external connections. To run an unmodified OS outside Ring 0, the hypervisor must intercept “forbidden” Ring 0 instructions and emulate them elsewhere–without the guest OS recognizing what's going on. Silicon makers are looking to help here; for example, newer Intel and AMD chips targeting the virtualization market are able to insert a new privilege level beneath Ring 0. Both provide new machine code instructions that work only at Ring -1, intended to be managed by a hypervisor. In this way, a guest OS doesn't have to be modified, and the performance penalty from emulation is reduced. It's the hypervisor's job to convince each guest OS that it and it alone has access to the host server's physical resources, while juggling access to ensure that programs and data don't leak between operating systems. Additional layers of code privilege for virtualized platforms on modern chipsets allow vendors to reduce the impact of a misbehaving guest OS in the event of a security breach or errant application. To further minimize the risk of a compromised platform intercepting guest communications to the underlying hardware, some form of transaction confirmation needs to be implemented. The Trusted Computing Group's most widely adopted standard is TPM, or Trusted Platform Module. The TPM is a critical element for a trusted hypervisor, providing hardware-based trustable root certificates, a trusted location for performing measurements, and several registries where trust measurements can be stored. TPM hardware encryption provides a guaranteed method for guest operating systems to vet communications with the hypervisor. The goal of TPM is to provide tamper detection and prevention; Intel's implementation, for example, offers trusted VM monitor whitelists on a hosted platform. TPM is enabled before any software is loaded and can provide owner confidence over the boot sequence and ensure the authenticity of each system element as it loads. In a nutshell, the TPM hands control of the platform to the hypervisor only after the hypervisor has been loaded into a known, trusted state. These concepts sound familiar? In higher-end versions of Vista, Microsoft relies on chipsetbased TPM to provide BitLocker functionality for encrypting data stored on local drives. Future Intel and Advanced Micro Devices hardware platforms also are slated to use TPM to forge trusted paths to attached peripherals, relying on it to create and store unique keys for hardwarelevel encryption of data paths. This encryption, in combination with validation of virtualization components, should make intercepting the TPM/hypervisor handoff more difficult, increasing IT's confidence that OS communications to and from the hypervisor are untainted. THE SIMPLE THINGS IN LIFE Like a person who obsesses about being hit by lightning while driving without a seat belt, it's the mundane dangers associated with virtualization that are most likely to bite you. For example, both fat and hypervisor-based hosts are at risk of having a guest OS compromised via traditional threat vectors and exploits. An unpatched or poorly protected public-facing server is at risk, period, whether it's running on a standalone box 17 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME informationweek.com, September 1, 2007 or as one of many VMs on a large hosted platform. However, common sense dictates that an organization's exposure increases in tandem with its reliance on virtualization and server consolidation--the more VMs per platform, the greater the danger of an undetected intrahost problem spreading. For all practical purposes, intrahost threats are invisible to traditional offbox safeguards. External firewalls and other security tools cannot inspect or control intrahost traffic, where packets never leave the host to t raverse wired infrastructure. Concerns common in the real world that are tough to catch in a complex hosted environment include extraneous or suspicious intrahost cross talk posing as legitimate traffic, which is indicative of port scans, virus behavior, or other malware, and direct (targeted) or incidental denial-of-service attacks impacting other guest VMs because of consumption of CPU cycles, input/output resources, or virtualized network bandwidth. “The 'more eggs in one basket' risk, from a pure operational perspective, has less to do with evolving threat vectors than simply being no-duh IT,” Neohapsis' Shipley says, adding that IT groups saw the same dynamic with early storage area networks. “Most organizations can manage this risk by designing for extra capacity, running through virtual server migration drills, and keeping up with patching.” The last item is a lesson worth repeating. “Even though I think VMware has done a good job in reducing the attack surface, ESX/VI3 still an operating system, derived from Linux, and as such it needs to be patched,” he says. “Problem is, patching ESX servers is a riskier and more intrusive proposition because you're not just taking down one OS, you're taking down all the OSes it hosts, too.” The good news is that we've thus far seen relatively few critical VMware patches. LIVING SAFELY IN A VIRTUAL WORLD For now, we're all biding our time, waiting on the first successful compromise of a production hypervisor or VM monitor. To make sure your 18 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME informationweek.com, September 1, 2007 network doesn't become a poster child, architect your host implementation to keep the potential attack surface as small as possible. Find out where third-party device drivers reside–within the hypervisor to improve performance or at a higher layer, taking a slight hit while reducing the security risk. Disable unnecessary emulated devices and lock down extraneous features and unused services on both the host platform and guests. Remember, a virtualized machine is still a machine. While that may seem obvious, IT needs to approach VMs with the same diligence and care offered to traditional servers, including adherence to security policies and guidelines. Thirty-six percent of our survey respondents admit to having no IT security or protection plan in place, with 23% fessing up that their policies are works in progress. Considering that upward of 70% of respondents have deployed at least one host platform, it's clear that unpatched or unprotected virtualized servers represent vulnerabilities just waiting to be exploited. Ensure that security concerns, permissions, and environmental settings are properly configured to follow VMs to new hosts--while environmental flexibility is a key advantage of enterprise-class offerings like VMware ESX, without proper planning the ability to move VMs on the fly can be a curse. “Along the lines of reducing attack surfaces and general exposure, I've seen organizations moving their VMware management segments off from the rest of the network and restricting who and what can gain access,” Shipley says. “Clearly, firewalls in the data center is a newer trend, but certainly not one that's being driven solely by VMware. The more progressive IT teams I've seen are really starting to think about the concept of 'least privilege' models when it comes to network segmentation, and organizations can reduce their risk profiles by being diligent about restricting access to the VMware management infrastructure.” Shipley also stresses that IT should never put a virtualization host machine in a position where it has to enforce network zones--for example, having an ESX parent host guest VMs in and out of a DMZ. CAN YOU BUY YOUR WAY SAFE? Innovative vendors of dedicated virtualized security appliances, including Reflex's VSA and Blue Lane's VirtualShield, are focusing on virtualization as a solution to security problems, rather than just another attack vector. While we prefer to reserve the “appliance” moniker for things with three-prong plugs, we realize we're fighting a losing battle: VMware is pushing the concept of “drop-in” dedicated VMs that are purposebuilt and preconfigured to address specific security or management needs, and many others are rushing to this nascent space; while no big player has given formal notice, we predict traditional security vendors such as Symantec will soon enter this market in force with tailored offerings. BUT IS THAT A GOOD THING FOR IT? “I can't help but wonder if some vendors are simply looking at all the virtualization going on and saying, 'Hey, how do I sell security to all these VMware shops?'” Shipley says. “Part of the burden on users/ consumers is to discuss what the true threat vectors are in their networks, and then look to tools.” 19 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME informationweek.com, September 1, 2007 Still, no one ever got fired for buying something that keeps an organization from being the next TJX. The clever folks at VMware understand this and recently bought Determina, a company that sells a memory firewall that can protect against stack and heap overflow exploits. While that's a pretty narrow protection goal, it's an important one. The problem is that for some applications, the Determina memory firewall could put a good-sized dent in overall performance. VMware also absorbed Determina's LiveShield, which can apply patches on the fly–no need to reboot the server, just apply the patch in memory. Certainly this is right up VMware's alley as the technology isn't too far from its own binary emulation system, which rewrites parts of executable code as it loads. While the idea of patching a running OS or application sounds interesting, it doesn't alleviate the need to test patches before they're applied. Usually, it's that testing that slows down the process, not the need to bounce the server. This is where Blue Lane comes in with its patchemulation products (there's a physical appliance version as well as a virtual appliance for VMware). The idea is to catch incoming attacks and make the fix just as an actual patch might do, before the offending packet ever gets near the server. While the company has its share of naysayers, VirtualShield performed as claimed when we tested it in our Florida Real-World Labs. And it got another seal of approval this week from none other than Microsoft itself, which tested the physical appliance and found it fully interoperable. In the virtual world, the combination of these three products makes a pretty good safety net. The memory firewall will protect against overflow exploits, while Blue Lane's technology gives IT the time it needs to properly test patches and, once tested, LiveShield lets you apply them on the fly. Finally, expect VMs to take a growing role on the desktop, with notebooks and PCs designed from the ground up to support admin-locked VM partitions constantly monitoring all running areas, safely removed from user-installed malware or human-error compromises. If we leave you with one piece of advice, it's to work to raise awareness. The last question in our reader survey was open-ended, asking if readers had additional concerns or opinions on virtualization security. Sure, we got the expected rants and raves for or against specific vendors, but a recurring theme was, “I didn't have any concerns ... until I completed this survey.” If knowledge is power, consider yourself armed. 20 SECURE VIRTUALIZATION PLAYBOOK BACK TO HOME networkcomputing.com, May 28, 2007 A Look at Blue Lane VirtualShield We put Blue Lane VirtualShield to the test and found its unique patching approach an effective way to protect against remotely exploitable vulnerabilities targeting VMware. May 28, 2007 By Avi Baumstein In many ways, security in a virtualized environment IS no different from security in the real world: You plan for defense-in-depth with hostbased and network-access controls, and set security systems to monitor traffic where appropriate. For added protection, there are a few security “virtual appliances” available at VMware's Virtual Appliance Marketplace. We decided to test one, Blue Lane Technologies' VirtualShield, in our University of Florida RealWorld Labs®. According to its billing, VirtualShield removes malicious content from network traffic before it reaches your virtual servers, a technique the company calls “inline patching.” This guards against new vulnerabilities, often well before vendors release fixes, and lets IT safely run legacy apps for which patches may no longer be issued. At press time, VirtualShield was one of just two virtual appliances (VAs) we've seen intended to protect virtual machines (VMs) running under VMware. The other, Reflex Security's VSA, is an IPS (intrusion-prevention system) that runs in ESX and protects virtual servers. Blue Lane distinguishes itself by taking the approach of patching network traffic, rather than just blocking the evil stuff. Although still new and needing some polish, VirtualShield is innovative and well-executed. The core functionality works as advertised, and Blue Lane, a four-year-old pre-IPO start-up, seems committed to refining its technology. The company's willingness to rapidly correct problems discovered during our tests makes us feel very comfortable recommending VirtualShield, especially since the product brings the capability of Blue Lane's two-year-old PatchPoint appliance inside VMware ESX server at an attractive price: $599 per year for a dual-processor server, compared with a $7,500 cost of entry for Blue Lane's physical appliance. Keep Your Guard Up Moving to a virtualized environment doesn't put conventional security measures out of business; however, a few factors are worth considering. First, you can't always deploy a tap or span port on virtualized systems, as you can on conventional devices. Fortunately, IT can use VMware's 21 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME networkcomputing.com, May 28, 2007 ability to create vSwitches to its advantage—by putting security right next to virtual server instances, you decrease the perimeter that must be protected. The main thing to remember: Don't treat a large virtualized infrastructure as a network black box. Security systems must be able to look inside the virtual infrastructure. If it's treated as one solid “box,” or system, you might find that an attacker who compromises one VM has a large sandbox in which to play. Fast Fixes We were intrigued by the concept of inline patching. When a vendor releases a patch to an OS or application that VirtualShield protects, Blue Lane tears apart and analyzes the patch, deciphering exactly how it changes the OS's behavior to eliminate a vulnerability. The company promises to produce an inline version in less than 24 hours for critical patches and in no more than 72 hours after the original is released for lower-priority patches. Consider a buffer overflow: VirtualShield recognizes there's too much data in an API call and truncates the data, the same way the vendor's patch does. This technique renders network traffic at least as safe as the original patch would have, before it reaches the server. Blue Lane maintains inline patches for server versions of Microsoft Windows; Red Hat Enterprise Linux; SUSE Linux; FreeBSD; MS Exchange and Sendmail; Cyrus and Courier IMAP; MS SQL; the Apache, IIS and iPlanet Web servers; and apps such as Samba, Bind and WuFTPd. The company partners closely with Oracle and VMware, and its physical PatchPoint appliance offers protection for Oracle databases. One key feature Blue Lane offers: inline patches to protect software, such as Windows NT, for which the vendor is no longer issuing patches. This lets IT run outdated but business-critical apps with fewer security concerns. You may wonder, why analyze suspicious network traffic at all when you could just block it? There are occasions when simply dropping packets on the floor will cause problems. Consider e-mail passing between two Exchange servers. If a message contains malicious content, such as a buffer overflow, an IPS could recognize that fact and dump the packets, or maybe even be nice and send a TCP Reset. But then the sending server will try to send the traffic again, get blocked again, and so on. Another problem could arise when a Web server is using pooled connections to a back-end database. Say someone tries to pass a SQL injection attack. An IPS between the Web and database servers would kill the connection ... but that would take out all Web transactions, not just the malicious one. Protection You Didn't Know You Needed We were impressed that Blue Lane develops inline patches for vulnerabilities that have been announced, but for which no vendor patch yet exists. By analyzing vulnerabilities discussed on venues like Bugtraq and Full-Disclosure mailing lists, the company determines the underlying problem and develops virtual patches for these zero-day exploits. Once the vendor releases an official patch, Blue Lane analyzes it as it would any patch, makes needed changes to the initial inline patch and distributes it. A good example of this process is the recent Microsoft DNS RPC buffer overflow (CVE-20071748). Microsoft released an advisory on April 12, a CERT advisory was issued April 13 stating that exploits were available, and Blue Lane provided an inline patch on April 13. Microsoft finally released the official patch on May 8—leaving servers vulnerable for more than three weeks. With VirtualShield, Blue Lane also has started offering policy-based filtering. This lets the product detect and rectify some problems with network traffic that aren't normally addressed by vendor patches. For instance, Blue Lane claims to protect servers only—it makes no effort to replicate vendor patches for client applications. But the company also realizes there are times 22 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME networkcomputing.com, May 28, 2007 when IT needs to run a Web browser on a server, maybe to download patches or register software, a practice that could expose the server to malicious network traffic. To counter this risk, Blue Lane offers a policy that limits Web browsing to only sites that have been whitelisted by the administrator. Other included policies can block connection attempts to a Microsoft SQL database with the user name “sa” and a blank password (some versions of SQL Server and the Microsoft Data Engine had a blank sa password as the default) or block duplicate DCERPC bind requests. Because these policies won't apply in all situations the way patches would, they're not activated automatically. IT must choose which to enable. During our tests, to better understand its capabilities, we asked whether VirtualShield could block a range of scenarios. Many times the answer was yes, but our contact furiously scribbled notes to take back to the development team when we brought up situations its policies didn't cover, including suggestions to block PHP remote includes or Windows login attempts after a number of failed tries, to shut down bruteforce password guessers. This is an area where Blue Lane says it plans to expand, and the company promises to include the ability for admins to write custom policies in the next major version. Specialized Protection VirtualShield comprises two parts, both of which run as VMs under VMware. By monitoring traffic at the hypervisor level, VirtualShield Gateway brings protection as close as possible to your virtual servers. In the simplest case, this is implemented in VMware ESX by creating two virtual switches, or vSwitches. The first accepts all incoming traffic from a physical NIC and passes it along in promiscuous mode to the VirtualShield VM. VirtualShield does its magic, then passes the traffic out through a second vSwitch, to which all other VMs are connected. Note that in this design, traffic between VMs does not pass through the VirtualShield Gateway. Other configurations are possible, including multiple VirtualShield Gateways or passing a VLAN trunk through the gateway, so that the gateway sees the different networks. The VirtualShield Manager runs as a separate VM and should use the management vSwitch rather than the one connected to the VirtualShield Gateway. The Manager appliance provides a UI for working with as many as 100 gateways, both virtual and Blue Lane's physical appliances. It's responsible for updating and rolling up reporting from multiple gateways. The Manager provides a number of convenient features, including the ability to create multiple VirtualShield admins and assign rights to configure and monitor the inline patching of specific servers. It also offers alerts in e-mail and SNMP traps. Automation Situation VirtualShield automatically discovers servers by monitoring its network; when it detects traffic to a server, it starts probing to determine the server's OS and what services it's running that can be protected. For a small collection of servers, this automatic discovery seems a bit of overkill, especially given the hand-holding it took (see “How We Tested”). However, as the number of virtualized servers grows, autodiscovery will save significant amounts of time. Once VirtualShield discovers the services running on a given server, it configures the appropriate set of inline patches. Some patches are mandatory—Blue Lane calls these “kernel-level” patches—and they cover basic network stack issues. Other patches can be selected and unselected on a per-server basis, though by default VirtualShield selects all applicable vendor patches—a good starting point. Nonpatch policies aren't applied by default, but VirtualShield lists policies that apply to each particular server, along with descriptions and CVE references; we could choose which ones we wanted to apply. Once running, the VirtualShield Manager offers a number of monitoring and reporting tools. Displays show the status of its server discovery 23 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME networkcomputing.com, May 28, 2007 activities as well as its detection of exploit traffic on the network and what action it took (usually “Apply Fix”). It has reports that let us slice and dice its activity every which way; for example, number of inline patches per server or exploits prevented. An Executive Report presented graphs showing the number of virtual patches enabled for discovered OSs and apps, the number of times those patches have been used over the past month, and a list of managers and gateways. When we pointed out that the Executive Report—the first output IT will likely want to show the CIO—was the only one without an option to print, Blue Lane promised to add that capability. In a world in which teams of researchers promise to expose a new vulnerability each day—March is brought to you courtesy of PHP bugs—security products must stay up-to-date. Virtual Shield delivers here as well, providing an online-update feature. Updates can be automated to occur every night or when you provide the OK. We tested a full update of the system, which took about 20 minutes to download and replace all the software on the manager and one gateway—the equivalent of flashing the firmware on a hardware appliance—and required a reboot of the Blue Lane appliances. During the reboot of the gateway was the only time our protection was impacted—for 20 to 30 seconds, as it restarted. VMware can be configured to pass all traffic or no traffic when the gateway is not functional; this provides a fail-open or fail-closed capability for updates, or in the event the VirtualShield Gateway should crash. Blue Lane says it expects to release full updates approximately every six months. Patch and policy updates could occur as frequently as every day, depending on need, but fortunately, these are extremely quick, and require no downtime or loss of filtering capability to install. Shortly after we completed the full update, we noticed that reports were not updating accu- rately and, even worse, that servers were not being discovered and protected as they should. This is when we exercised VirtualShield's support feature, which creates a secure outgoing SSH tunnel on Port 80, 53 or 443 (whichever is open) to Blue Lane. Once connected, keys were generated so we could let a Blue Lane engineer log in to our Manager and diagnose the problem. It turns out that the update process had created a full backup of the system before installing, but because Blue Lane keeps the disk size of the VMs small, there wasn't enough room for the entire backup. This feature, used in the physical appliances, lets Blue Lane quickly revert back to a known-good configuration in case of update problems—but those have 200-GB disks, so a few gigabytes of backup don't cause many problems. Again, Blue Lane says it will fix this bug before it impacts users. Engineers also can troubleshoot problems the old-fashioned way, over the phone, in environments where an outgoing tunnel would not be feasible. Should You Buy? We've established that VirtualShield is cool, but do you really need it? If your company has excellent procedures for patch management on servers and the workstations that access them, downtime windows that let you get patches installed promptly, and good firewall separation from the Internet, probably not—though for the price, you may want the added peace of mind. Another crucial factor to ponder is whether Blue Lane's list of protected OSs and apps covers your environment, keeping in mind that policies can be used to protect many otherwise uncovered vulnerabilities, especially common problems in PHP Web apps. If you run critical servers on VMware and have uptime requirements or strict change-control processes that make it difficult to patch in a timely manner, Blue Lane's VirtualShield may be just the ticket. It could also be useful in environments with a large number of virtual servers managed by different groups, but housed together, and where it's hard to ensure that each group is keeping its servers well maintained. With the bad guys planning to release new 24 SECURE VIRTUALIZATION PLAYBOOK continues >> BACK TO HOME networkcomputing.com, May 28, 2007 exploits right after Microsoft and other vendors ship their monthly patches—thus ensuring almost a month without protection—Blue Lane's quick action on newly discovered exploits could really save your servers. Other Virtual Bargains Most offerings at VMware's Virtual Marketplace are meant as a quick way to try out a product or technology. Two we noted are StillSecure's StrataGuard and the SVIDS intrusion-detection system, a replacement for a physical network device. The former takes an innovative approach to packaging software that we found almost effortless to set up and get running, making it easy to try out a number of options before you buy. The latter is a way to run services that would normally require dedicated hardware in the virtual environment you already have. Obviously, you wouldn't run a high-traffic router or firewall on a virtual appliance, but for lowertraffic situations, or when the devices being protected also are virtualized, it can drastically reduce your cost and complexity. Old Vs. New Conventional IPSs use signatures to detect network traffic that matches patterns known to be malicious, like the specific sequence of characters used by the Blaster worm. This technique works well against the attack it's designed for, but any variation or new attack against the same vulnerability requires a completely new rule. Additionally, evasion tactics, such as fragmenting packets in unusual ways, can fool an IDS or IPS into not detecting an exploit attempt. Security vendors are catching up, using more sophisticated techniques to monitor connection states and search for malicious traffic using rules that match behavior, not just patterns; this improves detection rates and reduces false positives. Blue Lane employs all these methods, plus a few more: Custom code built atop a modified Linux kernel emulates the network stack of the OS VirtualShield is protecting. This way, it can reassemble packets exactly as the would-be-victim OS would, thus stopping many evasive attacks cold. Blue Lane told us this processing adds only .25 ms to .5 ms of packet delay, which is typical for firewalls. How We Tested Getting virtualshield running was as simple as extracting files to our ESX storage volume, adding them to our VM inventory and configuring some vSwitches. We used VMware's Infrastructure Client to set up our network configuration parameters, then connected to the VirtualShield Manager via Web browser. VirtualShield automatically discovered our servers and set out to probe the devices to determine OS and what running services it could protect. We generated some network traffic by scanning our servers' IP address range with Core Impact's Information Gathering module, and VirtualShield went to work doing discovery. VirtualShield couldn't detect much about one of our servers, a Red Hat Enterprise Linux device, because the portmapper had been firewalled. We used the Web interface to provide credentials that VirtualShield could use to connect to the RHEL server via SSH; it was then able to discover all of our services. We could have provided Windows login credentials to help discover services on Windows servers, and for servers that don't respond to ICMP queries, we could manually add IP addresses; Virtual Shield will then attempt to discover their services. All NETWORK COMPUTING product reviews are conducted by current or former IT professionals in our own Real-World Labs®, according to our own test criteria. Vendor involvement is limited to assistance in configuration and troubleshooting. Network Computing schedules reviews based solely on our editorial judgment of reader needs, and we conduct tests and publish results without vendor influence. Avi Baumstein is an information security analyst at the University of Florida's Health Science Center. Write to him at abaumstein@nwc.com. 25 SECURE VIRTUALIZATION PLAYBOOK