In The Four Horsemen of the Virtualization Security Apocalypse, Chris Hoff shines a great big spotlight on the dark side of virtualization security (or virtsec, as its increasingly being known). To quote from Hoff’s article:
Short of the notions I’ve discussed previously regarding instantiating the vSwitches into hardware and loading physical servers with accelerators and offloaders for security functions, there aren’t a lot of people talking about this impending set of challenges or the solutions in the short or long term.
This should be cause for alarm.
These issues are nasty. Combined with the organizational issues of who actually owns and manages “security” in the virtualized context, this stuff makes me want to curl up in a fetal position.
I agree with what Hoff has to say and I’m glad he’s taking the time to boil down the issues so that non-security-minded IT pros can really understand the problems. However, Hoff, I have to take you to task for one thing in your article: the kitten thing was just too much. Poor little kitten…
I particularly agree with Hoff’s #1 point (”Virtualized Security Screws the Capacity Planning Pooch”). The idea behind VMsafe and all these virtsec appliances is a great idea and all, but what about the overhead? At what point does having all this “extra” security so greatly bog down our virtualization engine that it’s no longer worth it to virtualize? And how do we actually, realistically begin to address this issue? Do we move the security functions into the hypervisor itself? And while this might address the performance concerns—although I don’t think so—isn’t this just instantiating Hoff’s vUTM?
One of the interesting things that I hope to be able to do soon is try to measure the overhead of some of the virtsec appliances that are currently available on the market. Not to publish any results or hit any vendors over the head with the information, but just to have a better idea for myself and my customers about how this stuff actually behaves in the real world. If anyone has already done that sort of thing and is willing to share their information with me, I’d be mighty appreciative.
I am curious about something—how many organizations are using a single physical host with VMs across different security zones? See, this is something that I would never recommend, and to me it seems like physically segregating your security zones into different virtualization environments solves a fair number of the concerns about the “dynamic data centers” created by VMotion, VMware DRS, and VMware HA. Or am I overlooking a critical aspect?
Tags: ESX, Security, Virtualization, VMotion, VMware, VMwareHA
7 comments
Comments feed for this article
Trackback link
http://blog.scottlowe.org/2008/04/16/the-dark-side-of-virtualization/trackback/
Thursday, April 17, 2008 at 2:07 am
Osama Salah
I would like to see a security virtualization discussion that differentiates between new security threats added due to virtualization that do not have any parallel in the physical world.
I believe that conceptually there is not much new here. True, Virtualization has produced new challenges but fundamentally they are not much different from the challenges we face in the physical world. The current virtualizaton security discussions seem to lack focus and sometimes it just feels like FUD.
As for the overhead, I’m not convinced that it is such a great concern. In the physical world we add dozens of hardware and software solutions that add an overhead. They do cost money and they do impact performance. In case of the virtual world the overhead might just be resolved by adding one more CPU or a little more RAM.
Thursday, April 17, 2008 at 9:11 am
slowe
Osama,
Thanks for your comments.
One key thing that jumps to mind is a lack of visibility. In the physical world, there are almost always ways we can see the traffic that is flowing across the network–via span ports, inline IDS/IPS, network sniffers, etc. How does one accomplish this in the virtual world? Sure, you could run a network sniffer on a vSwitch in promiscuous mode, but then you’re back to considering the overhead again. It’s not an impossible problem to solve, but it is a problem to be solved (and solved well).
As for the overhead, I do think it’s too early to tell for sure. However, isn’t this something we should address before it becomes a problem? In the physical world when we add appliances and hardware solutions, they don’t detract from the CPU/RAM resources available to the workloads. That inline IDS/IPS has its own RAM and CPU processing power. That hardware firewall has its own RAM and CPU resources. Now we are going to have to collapse those functions onto the same hardware that is running your applications. Of course there’s going to be a performance hit. Adding one more CPU might fix it; adding one more CPU might not. The point is, who out there is telling us how much overhead we should be planning for? The virtual security vendors aren’t going to say because that might cause people not to buy their products. So, in my mind, anything we can do to bring this issue to people’s attention is worthwhile.
Thursday, April 17, 2008 at 9:27 am
Christofer Hoff
@Osama:
The key point of capacity planning here is that as you continue to add VM’s to a host to “scale” security functionality, you remove the capacity to consolidate P to V which is the primary goal.
If one used to be able to consolidate 10 P servers down into a single V host, and now you’re having to load that same server with 3-4 Virtual appliances for security, you’re not going to have the same ROI and the capacity issue — while similar to the physical world — becomes much more difficult for the other reasons I stated.
The VirtSec providers will start to give you stats shortly that promise performance/throughput, but they will be confusing due to the example above…esp. when these functions are placed inline with VM’s AND the ping/pong of interacting with external security functions.
My opinion is that it’s not going to be as straightforward as people think. I’ve spoken to most of the emerging providers and in confidence, they’ve told me the same thing…
YMMV.
In terms of threats and vulnerabilities — it’s an evolving landscape. Looking for the answer (is virtualization more secure) before the question can even be properly framed is going to give a false answer. What I mean is that as the solutions evolve, so will the T&V’s…we’ll react to them as we always do. There will be classes of new T&V’s that are virtualization-specific.
Right now, the risks are mostly centered on organizational and operational role shifts, separation of duties and virtual networking issues as Scott and I continue to write about.
/Hoff
Thursday, April 17, 2008 at 9:49 am
Gabriel Maciel
“I am curious about something—how many organizations are using a single physical host with VMs across different security zones? See, this is something that I would never recommend, and to me it seems like physically segregating your security zones into different virtualization environments solves a fair number of the concerns about the “dynamic data centers†created by VMotion, VMware DRS, and VMware HA.”
I agree with that 100%. So, if you can keep your Virtual environments / traffic (Dev, Test, etc) without making them cross your security zones (using different ESX Hosts and vSwitches plus physical Nics going to dedicated vLans), most of your security problems will be addressed (at least at the network level).
Friday, April 18, 2008 at 3:42 pm
Paul
So what’s the verdict?
Does this darkside warrant that we ‘turn out the lights’ on virt as we knew it? Or are we beFUDdled just enough to drive another technology breakthrough within InfoTech “feature” that is virtualization. (eluding to recent financial analyst discounts of virtulization as a market and VMWare as a market leader).
My take is that the Hypervisor will at some point become the network, the security intelligence and the policy enforcement that is currently handled by dedicated devices.
The gap is closing fast. So fast that Moore himself can’t see it in motion.
The distributed vs. centralized dilemma is waning in the face of geo-redundancy and local-instance networking.
And finally, if we can get over this hump that is IPv4, we’ll have 64-bit optimized networking and much improved layer 2 negotiation to ice the cake.
Oh, and for those that may not know, FUD is the the loving acronym for the use of Fear, Uncertainty and Doubt to drive an agenda.
Saturday, April 19, 2008 at 7:29 am
piyush bakshi
Security with virtualization is of key importance along with the physical security of servers and storage boxes. One should check if vendors offering virtualized storage are also offering native security with it.
Wednesday, April 30, 2008 at 12:22 am
Christofer Hoff
@Paul:
No, it doesn’t mean we “turn out the lights.” In fact, just the opposite. It means instead of allowing these issues to hibernate in darkness and wait to be bitten, it means shining the light on the issue(s) and pushing for solutions.
I’m not sure if you were alluding to some super-secret agenda I supposedly have based on what you classify as FUD, but you don’t seem to dispute my assertions, so I’m confused.
The only agenda I have is to point out what is quite obvious to me as an impending issue that I’ve not seen discussed elsewhere and offer solutions.
Last time I looked the hypervisor is becoming a commodity and IPv6 will not rescue orphans from a burning building, either
/Hoff