Using VAAI to Maintain Control?

Chris M Evans (@chrismevans) aka The Storage Architect recently posted an article on his site titled “VAAI – Offloading or Maintaining Control?” The article was a follow-up to a discussion with Nigel Poulton (@nigelpoulton) at HP Discover 2011.

The basic premise of the article as I understand it was a discussion of whether the introduction of VAAI (vStorage APIs for Array Integration), while it does bring benefits to the storage layer, also gives VMware more control. Quoting from the article:

In comparison to NFS, it seems that by implementing VAAI features, VMware are looking to retain even more control over the storage layer than before.

At first glance, it would seem that this conclusion is logical: VMware is using its popularity and large market share to force storage vendors to write to their proprietary “array integration APIs”, thus giving them another layer of control over the storage layer. VMware is the one pushing these integrations, so it seems like a reasonable conclusion, right?

The problem with this conclusion is that it overlooks one simple fact: the vStorage APIs for Array Integration aren’t APIs at all; in fact, they are nothing more than standard SCSI commands. That’s right: VAAI is composed of standard SCSI commands that have been defined and approved by the INCITS T10 committee. (If you’re interested, you can get details on the commands in the SBC-3 standard available from the T10 website.) Any hypervisor or OS running any file system on any array could leverage these commands and their functionality. There’s absolutely nothing specific to VMware or VMFS, VMware’s proprietary clustered file system, that would grant VMware any level of control. Microsoft could add support for these T10 commands and Hyper-V would reap the same benefits as VMware. The open source community could add support for these commands to Xen or KVM, if they so desired, and they would receive the same benefits that vSphere receives.

The flip side is true, too: there’s nothing in any of these SCSI commands that grants a storage vendor a level of control, either. NetApp, EMC, HP, HDS, Dell/Compellent/EqualLogic—anyone of these vendors could (and many did) implement the commands and would support them regardless of the software running on the host.

In my mind, the fact that these commands are completely hypervisor- and OS–agnostic (meaning that any hypervisor or OS could implement them) completely deflates the theory that VAAI gives VMware another form of control at the storage layer.

What do you think? Speak up in the comments.

Tags: , , , ,


  1. Glenn Sizemore’s avatar

    Yeah he was a little off base with that statement, but… what about the real meat of the article. Why choose VAAI over NFS if there is a negligible performance delta? Even though VAAI is completely standards based, VMFS is still proprietary. NFS is also standards based, and far more flexible than NFS. NFS also offers the ability for the array to go above and beyond what’s available in VAAI.

    IMO, VAAI makes block almost as compelling as NFS… now if we could only combine the two wouldn’t that be awesome!


  2. Brandon’s avatar

    I agree with your conclusion. Then again, why VMware chose to call it APIs is interesting on its own.

  3. slowe’s avatar

    Glenn, VMFS is no more proprietary than NTFS. Yes, the specifications for NTFS are more available, but “proprietary” does not equal “not well documented.” As for the real meat of the article, NFS vs. VMFS isn’t a debate—it’s a decision, and an “and” decision at that. There are plenty of reasons to use either or both, depending upon the needs of the environment. Finally, as for “NFS allowing the array to go above and beyond what’s available in VAAI,” well that would be proprietary to the array, would it not? NFS itself, as you pointed out, is a standard. It would be an array vendor’s proprietary file system (like WAFL for NetApp) that allows the array vendor to do interesting things underneath NFS. This is not to say that these “interesting things” aren’t useful and valuable, but that’s not a function of NFS—that’s a function of the fact that NFS plays well with the array vendor’s functionality when block storage does not.


    Absolutely agreed! The fact that these SCSI commands are called “APIs” is at the root of the misperception around VAAI.


    I’ll once again remind everyone to be sure to provide full vendor disclosure with all comments, where applicable. Thanks!

  4. Glenn Sizemore’s avatar

    Great points all, and you hit the nail on the head it is often an “and” decision. Keep it up we were all better off when you blogged more.

    I apologies I omitted my full vendor disclosure, I work for NetApp baby!


  5. Andreas Peetz’s avatar

    Scott, I was not aware of that VAAI really only consists of already standardized SCSI commands. This is very interesting information and I totally agree with your conclusion that this is NOT an unfair way to gain a competitive advantage then.
    However, I think, the real reason why Citrix and VMware do not come out with a VAAI equivalent is that they do not have a true cluster file system like VMware has with VMFS. And without that features like accelerated Storage VMotion and offloading of lock handling just do not make much sense.

    - Andreas

  6. slowe’s avatar

    Andreas, I personally don’t think that the presence or absence of a clustered file system really makes any difference. Using the EXTENDED COPY command (which is the command that supports hardware copy offload) would still provide benefits regardless of the presence of a clustered file system.

    Glenn, thanks for clarifying your affiliation—I appreciate it.

    Keep up the great comments everyone!

  7. Derek’s avatar

    Ya I was aware VAAI was just really SCSI commands (Opcode 0×83, 0×93 and one I forget), and would hope that MS adds support in Hyper-V 2012. The extended copy (0×83) would, as you point out, benefit any operating system doing large block copies. I figured one reason MS isn’t supporting yet is more would seem like MS is using a VMware feature and admitting Hyper-V could benefit from some VMware innovation.

  8. Kyle’s avatar

    Okay, I’ve been reading about and paying attention to VAAI for awhile now, before it was incorporated into version 4.1. Now here’s the gist, the original version required a plugin from your hardware vendor. Which basically took vmware’s proprietary advanced drive commands and a subset of the sbc-3 and converted them into that vendor’s internal commands.
    99% of the SAN vendors support SBC-3, but they also have their own lower level way of talking to the san, that has a bit more control, and just as fast.
    The new VAAI in 5.0 will fully support SBC-3 therefore all of the Storage vendors that support sbc-3 will be able to use VAAI.

  9. slowe’s avatar

    Kyle, thanks for your comment. I could be wrong, so I’ll dig in a bit and see if there’s more to this than there seems to be. In the meantime, would you mind sharing the source of your information?

  10. Andreas Peetz’s avatar

    Scott, you are right. The extended copy command is something that would have benefits for every vendor and file system. However, I consider the lock offloading the most important VAAI feature because it dramatically improves VMFS’ scalability (more hosts and VMs per LUN).

  11. Duncan’s avatar


    Reality is that many customers have already invested heavily in storage, fabric, education/experience/ops… VAAI enables them to be more efficient.

    And even NFS could use VAAI :)

  12. Chad Sakac’s avatar

    Disclosure – I’m an EMCer.

    Great comments all. FWIW – Microsoft and other OSes are working hard to implement support for these commands. TRIM support is likewise, an OS support for disk-level reclaim behavior (the SCSI equivalent is UNMAP). You should expect to see these things become more prevalent over time.

    @Glenn – I’ve talked about the topic of NFS/VMFS often, it’s always a religious debate. There remain advantages on both sides today (performance is not the primary one in my mind), and some use cases have one as the sweet spot, and the other has other use cases as sweet spots. Engineering teams at VMware and at the storage vendors are working hard to “strengthen the weaknesses”.

    @Scott/@Kyle – yes, the vendor plugins “munge” vendor-specific weirdness so that VMware has a common API structure.

  13. Chris M Evans’s avatar


    I didn’t mention VAAI as I’ve mentioned it many times before, including in the context of using the underlying SCSI commands for other purposes. For example:

    So I’m more than aware of how the technology is implemented.

    In fact, having read the T10 documents, I am still of the opinion that the SCSI commands that enable VAAI also open a security loophole. I’ve seen no response to my suggestion that the EXTENDED COPY command appears to have no security controls. If that is so, then an unscrupulous user could use the commands to copy data into a LUN they own – or worse, copy data back into a LUN and change data. I discuss this situation here.

    So, do any of your readers have a comment on that?


  14. slowe’s avatar

    Hi Chris, great comment—thanks for responding.

    Clearly, my post was not intended to insinuate you don’t know the facts about VAAI. I think you’ve proven that you are well-versed in many more storage technologies and products than I am!

    My post was merely to address the viewpoint that VAAI was a level of control. I suppose that because VMware did/does require some vendor plug-ins due to the fact that their implementations weren’t completely T10-compliant could be construed as a means of retaining control. Assuming VMware intends to make their implementations completely T10-compliant, even that will go away.

    Regarding your security concerns, I’m not nearly as knowledgeable as you are, so let me ask a stupid question—what security controls exist for the standard SCSI copy command that don’t exist for EXTENDED COPY?

  15. udubplate’s avatar

    Something to also consider…if the current functionality of VAAI uses standard SCSI commands it doesn’t necessarily mean future planned functionality will all necessarily use standard SCSI commands. Maybe some will and some won’t. Doesn’t have to be a black and white thing. VAAI is simply a marketing term for rolling up certain functionality.

Comments are now closed.