You might have read the article I wrote here titled vSphere Virtual Machine Upgrade Process, in which I described a process whereby you could upgrade your VMs to VM hardware version 7 (the version used with vSphere) as well as use the latest paravirtualized network and SCSI drivers (VMXNET3 and PVSCSI). Both PVSCSI and VMXNET3 offer greater performance with the same CPU utilization.
Rightfully so, some readers and other bloggers pointed out that PVSCSI isn’t supported for boot disks (Rich Brambley put up a really good post, for example). Rich, among others, suggested moving virtual machines back to a “two disk model,” with a boot disk and a separate data disk; this would allow for the greater performance of the PVSCSI controller on the data disk. This seemed to be a reasonable workaround. I don’t recall hearing about any significant issues with VMXNET3. Using the newer network driver seemed to be a good move all the way around.
Unfortunately, there is another drawback to both of these devices. Rich caught this drawback in his article, but relegated it to a small mention at the very end of the article that even I overlooked at first (emphasis mine):
There are some other factors to consider as well. For example, vSphere Fault Tolerance cannot be enabled on a VM using PVSCSI.
That’s right—you cannot use VMware Fault Tolerance (FT) on a virtual machine that is using the PVSCSI device. However, this restriction doesn’t just apply to the PVSCSI device; it also applies to VMXNET3! VMware FT cannot be enabled on a virtual machine using either the VMXNET3 or PVSCSI devices; vCenter Server will simply report an error that the network interface or disk controller isn’t supported for VMware FT.
In my opinion, this is a significant enough limitation that I felt it warrants its own post. If you are planning on using VMware FT in your environment, be sure not to configure any virtual machines to use VMXNET3 or PVSCSI if they might need to be protected with VMware FT. In this case, you’ll have to choose from either maximum performance or maximum protection—you don’t get both.
UPDATE: Rich Brambley shared links to two resources that describe the incompatibility between VMware FT and PVSCSI and VMXNET3:
VMware Communities: Unable to configure FT with error “Unsupported virtual machine configuration for Fault Tolerance. Device ‘Network adapter 1′ is not supported”
VMware Fault Tolerance Requirements and Limitations
Tags: Virtualization, VMware, VMwareFT, vSphere
-
crap!
-
Sorry, that was my visceral instant response to this, since I’ve already been building new templates for our migration process.
Now to go undo it.
-
Scott,
Some VMTN Communities links for your readers on these FT limitations-
Error received when VMXNET3 in use:
http://communities.vmware.com/thread/217845FT requirements and limitations:
http://communities.vmware.com/blogs/vmroyale/2009/05/18/vmware-fault-tolerance-requirements-and-limitationsThanks for the reference to vmetc.com!
-
Thank you for the information Scott. Keep up the excellent work.
-
Hello!
“be sure not to configure any virtual machines”
Sorry, why for all. Only for those what planing put to the FT. Right? -
It makes sense that VMXNET3 (which can be used with VMDirectPathIO) and PVSCSI don’t allow for FT – any VM using those devices is “tied” to that ESX host.
It’s the same principle as not allowing VMotion of a VM with a CPU affinity set, or of a VM with it’s CD-ROM connected to an ISO image on the local VMFS.
-
Scott V,
VMXNET3 currently does not support VMDirectPath yet, so VMs using VMXNET3 are not tied to the physical host.
Virtual devices need enhancement to support FT and such enhancement does not exist in VMXNET3 yet, that’s why you cannot enable FT when the VM is using VMXNET3.
-
Scott,
Thanks for taking the time to share your knowledge. As a person responsible for the successful deployment of a virtual initiative I’m grateful to have resources like yours where I can get up to speed fast on the ups and downs of this technology.
While there’s no substitute for trial and error on the actual equipment, it’s very helpful to have a solid idea of where we’re going and how things will work.
Cheers!
-
Stu, I’m with you.
This post should be renamed…
More limitations of FT.
All our VM guests hosted on VI3 are being switched over to VMXNET2 for improved performance.
As we move guests to new vSphere clusters the vmware tools are being updated as well as the NIC types being changed. We’re considering updating the SCSI driver type too. Some guests still have buslogic and are being change to LSI Logic. I noticed now there are multiple choices, SAS, SCSI, etc…
For the few use cases where FT is needed or desired seems like it would be fine leaving the NIC at VMXNET Enhanced (VMXNET2).
Any thoughts on the performance advantages (or disatvantages) of the following SCSI controller types:
BusLogic Parallel
LSI Logic SAS
LSI Logic Parallel
VMware Paravirtual -
Scott – Have there been any updates from VMware since you originally posted this article to fix the incompatibility between VMware FT and PVSCSI and VMXNET3?
Thanks again for the good info and I’m glad that I read this when moving to vSphere as I’ve decided to stick with VMXNET2 (Enhanced) for the time being.
-
Although maybe not “supported” something else to note is the VMXNET3 Adapter does not allow the MAC address to be specified via the Windows Guest OS Driver.
-
Just to update this post, VMXNET3 supports FT now in ESX 4.1.
-
also PVSCSI supports FT as well in ESX4.1, so both impossibilities solved.
27.ParaVirtual SCSI
VM configured with a PVSCSI adapter can be part of an Fault Tolerant cluster.
PVSCSI adapters already support hot-plugging or hot-unplugging of virtual devices, but the guest OS is not notified of any changes on the SCSI bus.
Consequently, any addition/removal of devices need to be followed by a manual rescan of the bus from within the guest.




20 comments
Comments feed for this article
Trackback link: http://blog.scottlowe.org/2009/07/05/another-reason-not-to-use-pvscsi-or-vmxnet3/trackback/