The news has hit the Internet in various places, but I wanted to point it out here because it does help to debunk the myth that virtualization can’t handle all workload. What’s the news? EMC and VMware have jointly demonstrated that a single VMware vSphere host running just three virtual machines can drive just above 350,000 I/O operations per second (IOPS).
I’ll let that sink in for just a moment. In case you don’t understand just how significant that number is, consider that a typical Fibre Channel drive can sustain somewhere just below 200 IOPS (and that’s being a bit generous). At 200 IOPS per drive, driving 350,000 IOPS would require 1,750 drives. (Fortunately, EMC used Enterprise Flash Drives (EFDs), so far fewer drives were required.) I would wonder how many of us have actually seen a storage array with that many drives.
Chad Sakac of EMC covered the tests on his blog here; the VMware Performance blog also discussed the results in detail as well.
So, next time you are thinking that VMware vSphere can’t handle your database workloads, keep these figures in mind. Or, if you’re a consultant like me, use these figures next time your customer says that virtualization can’t handle I/O-intensive workloads. This looks like pretty definitive results to me.
Tags: EMC, ESX, SAN, Storage, Virtualization, VMware, vSphere
-
Those numbers prove nothing though, they’re in a bubble and insignificant as far as I’m concerned.
Show me workload X on raw hardware vs. workload X in VMware, using a typical setup consisting of FC drives (no EMC, I’m sorry, as much as you proclaim otherwise, flash drives to back a VMware farm is nothing remotely resembling a typical setup). The reason people say “it can’t push I/O” is because of the overhead vs. raw hardware, not because they literally think VMware can’t generate I/O.
-
TimC,
I do agree with you that using EFDs to back a VMware installation is nothing like your typical setup. If someone were to create a “workload X on raw hardware vs. workload X in VMware”, we’d have to be careful to make sure that it is an apples-to-apples comparison. Like I said in my recent Virtualization Congress presentation, if you wouldn’t run that database on a 5-disk RAID-5 set when it’s physical, you shouldn’t run that database on a VMFS datastore sitting on a 5-disk RAID-5 set after it’s been virtualized. Just because it’s virtualized doesn’t mean the underlying disk I/O requirements magically change.
Personally, I believe that is the key reason why people think “it can’t push I/O.”
-
TimC,
There are case studies where application performance on virtual platform is within 85 to 90% of native. In some cases, aggregate performance from virtual machines on a single host beats the aggregate performance on the same hardware on native.
Checkout the following links for more details:
http://blogs.vmware.com/performance/2009/04/d.html
http://blogs.vmware.com/performance/2009/02/vmware-sets-performance-record-with-specweb2005-result.html
http://blogs.vmware.com/performance/2008/02/16000-exchange.htmlSome of these applications do require high I/O bandwidth (link# 1). vSphere can comfortably handle that and do much more as shown above.
-
I agree with both of you in a sence.
What I would like to see is the “cost” of virtualizing, that is comparation of the same workload on pure hardware and running in the virtualized envirovment. That way I would estimate the probable results in my own datacenter. -
“…if you wouldn’t run that database on a 5-disk RAID-5 set when it’s physical, you shouldn’t run that database on a VMFS datastore sitting on a 5-disk RAID-5 set after it’s been virtualized. Just because it’s virtualized doesn’t mean the underlying disk I/O requirements magically change.”
Very well said!
-
Scott, et.al.,
I believe the point of the effort from VMware and EMC was to show that VMware CAN handle incredible amounts of I/O without falling down. There are a very few organizations that will generate that amount of I/O with 1 host and 3 guests.
I believe TimC is correct in that EFDs are atypical, however, media is not relevant to the objective of this specific test.
Scott, you make a good point regarding storage and volume types. I think what is often a “lost art” in virtualizaiton efforts (and non-virtualization SAN deployments) is the carving and provisioning of the storage.
Just because you have a NetApp, EMC, Lefthand or IBM SAN doesn’t automatically make your storage optimal. Storage tuning is something that takes time and effort and planning that most clients and a good number of “engineers/consultants” just don’t do.
-
@TimC – I hear you – but remember the goal of this exercise was “how fast can it go”. We had the same feedback last time: “this isn’t a real world test” – no guff. The idea of running that type of load on a single vSphere ESX host won’t happen anywhere – it’s an insane configuration.
The goal is simple – by showing upper limits – we can change the question from “what can I virtualize” to “what CAN’T I virtualize”, and then move to “ok, what’s a reasonable way to move forward and be as successful, low risk, as low cost, and efficient as possible”. BTW – guys, EFDs are very real. We’ve sold out for 4 consecutive quarters now – we can’t literally meet the demand fast enough. Every customer has workloads that are gated by performance, not just capacity.
We ALSO produce a TON of content that are real-world configurations, real world workloads (often with side-by-side with physical, and the same platform, so you can do a direct comparison).
For example, in the last 24 hrs:
http://virtualgeek.typepad.com/virtual_geek/2009/05/more-on-exchange-on-vsphere-including-ft.html
@Scott:
You’re 100% right. The other “app on VMware” crime is the same one that happens in the physical world – talking in GB in configurations, when it’s about GB/MBps/IOPs/ms. Often – the VMware administrator only “sees” GB (I mean “capacity”), and therefore has no idea about the other factors.
-
A simple question – perhaps naive sounding…
How many I/O “PORTS” were used to generate this level of I/O activity?
Then one can calculate IO per PORT !
Hyper-V, based on QLogic experiments with Solid State storage indicated about 170,000-180,000 IOPS from Hyper-V using a single 8Gb Fibre Channel HBA port.
Ergo, about 2 HBA ports would deliver 340,000 IOPS.
How many such 8Gb FC ports would VMware need ?
Site Sponsors
8 comments
Comments feed for this article
Trackback link: http://blog.scottlowe.org/2009/05/18/vmware-vsphere-generates-insane-amounts-of-io/trackback/