Fate, Free Will, Virtualization, and Storage

You might wonder what fate and free will have to do with virtualization and storage. The title of this post is a reference to the debate of Fate vs. Free Will, which in turn is a reference to Stephen Foskett’s recent post VMware as Oedipus: How Server Virtualization will Change Storage Forever. I won’t provide all the details here (go read the post), but the basic idea behind the post is that VMware’s drive to add storage features to the virtualization stack puts it on a collision course with EMC, a leading storage vendor. The twist here is the fact that EMC has a majority ownership in VMware, thereby earning EMC the term “parent company” and creating the Oedipal conflict to which Stephen alludes in his post.

First, let me sum up Stephen’s points:

  1. VMware is causing users not to purchase storage arrays.
  2. VMware integration is “leveling” the playing field.

Let’s take a look at each of these points.

A Decrease in Shared Storage?

Stephen makes this statement in his article (emphasis his):

VMware is rapidly innovating in this area. Integrating and developing snapshot, replication, thin provisioning, and other features in VMFS enables everyone to have advanced storage functionality, regardless of which storage device they use. In this way, VMware is already causing many users to forego an enterprise storage array purchase.

Perhaps it’s the specific term “enterprise storage array,” but I have a hard time believing that the adoption of VMware is causing users to forgo array purchases. Think about it: to even use many of the advanced features of vSphere like vSphere HA, vSphere DRS, or vSphere FT, shared storage is a prerequisite. Users literally cannot use these features without shared storage, and—today, at least—shared storage in almost all cases means an array.

If, however, the statement is intended to say that VMware users are buying less feature-rich arrays because of the features being added into vSphere—features like snapshots, replication, and thin provisioning—then I suppose I can see that. This is why array vendors are (or should be) driving innovation in other areas, such as dynamic auto-tiering, more robust snapshotting functionality, higher availability, and higher levels of performance.

Additionally, this is an opportunity for both virtualization experts and storage experts to help customers understand the differences between the features provided by the hypervisor and features provided at the storage layer. While these features share names, they can be very different! Here are a couple specific examples:

  • VMware’s snapshots are fundamentally and dramatically different from the snapshot features offered by many storage vendors. Not only are they different in how they work, they are also different in their uses and usage patterns. Storage administrators use array-based snapshots for different purposes and in different ways than vSphere administrators use snapshots.
  • vSphere’s replication functionality is a nice “check box” item, but lacks many of the features that array-based replication offers. For example, there’s no compression, no deduplication or WAN optimization, and no idea of consistency groups.

As you can see, while it’s true that VMware is offering features that are similar in name and purpose, these features often are not true competitors to the features that storage array vendors offer, EMC included. Looking ahead, I anticipate that will continue to be the case, and storage vendors will continue to have ample opportunities to offer functionality above and beyond what the hypervisor can or will offer.

Homogenization of Storage?

The second point of the article is the assertion that “ever tighter integration serves to anonymize and homogenize enterprise storage devices.” I don’t agree here, for a couple of reasons:

  1. This statement assumes that all integration is the same, which is not the case. One vendor’s level and type of integration with VMware can be very different than another vendor’s level and type of integration. A vCenter plug-in is not the end of the story. What about integration with your replication solution? What about integration with your snapshot functionality? What about the quality of your VAAI implementation? One vendor’s implementation of VAAI might behave quite differently than another vendor’s implementation. What about your support of VMware’s multipathing APIs? I could go on and on, but you get the idea.
  2. This statement excludes the value of innovation in other areas, implying that VMware integration is the sole factor that levels the playing field. As I’ve stated on many occasions, every storage solutions has its advantages and disadvantages. The way that EMC does things gives it an advantage over NetApp in some areas; at the same time, the way that NetApp does things gives it an advantage over EMC in other areas. If all arrays were the same and were measured only on their VMware integration, then I could see this statement. That’s not the case. And even if it were, then as I’ve just shown you, VMware integration can take many forms and many levels. Despite ever increasing levels of integration, vendors still have plenty of opportunities to differentiate themselves from other vendors through price, performance, data protection, scalability, reliability, and availability.

Conclusion

I don’t disagree that VMware will change the nature of enterprise storage; in fact, I would argue that it already has changed enterprise storage. But to say that VMware will completely anonymize and homogenize enterprise storage is, in my humble opinion, a bit of a reach. There are still plenty of areas in which storage vendors can innovate and differentiate, both in addition to as well as in spite of VMware’s own storage-related ambitions.

Disclaimer: It’s probably well-known anyway, but it’s important to state that I do work for a storage vendor (EMC), although—as my site-wide disclaimer indicates—content here is not sponsored by, reviewed by, or even approved by my employer.

Tags: , , ,

  1. Will Huber’s avatar

    Scott,

    Nice post. I agree with everything you wrote as it pertains to enterprise storage. Obviously that was the focus of the article, but I think it is worth pointing out that there may be some validity to the “anonymize and homogenize” argument for small and maybe even some mid-sized organizations. VMware is doing a lot in this space and I think that will probably have more of an impact on smaller environments, not the enterprise.

    As always, thanks for the write up!

  2. Ed Grigson’s avatar

    I completely agree Scott. Today Oracle is the technology which ‘the business’ care about (in my case) – VMware is well understood as a platform but the storage array vendor was largely chosen due to the integration with our application stack and functionality outside of VMware related storage.

  3. Donny Parrott’s avatar

    Actually, to the contrary, the virtualization push is moving all physical assets toward comoditization. This is evidenced by the progression through silos. First, it was the compute layer. The server platform is near irrelevance as abstraction makes reliability and performance the primary per dollar the primary driver.

    Next comes networking. As products like the Nexus 1000v and vShield push networking into software, openflow (or similar) will consume networking for all but edge devices (maybe those one day).

    Now storage. Do we even need arrays any more. The grid is coming and companies like Nutanix are leading the charge to diminishing the array requirement.

    Soon, the orchestration of the physical assets will have greater importance than the assets themselves as the software will distribute feature sets to any number of compatible assets.

  4. slowe’s avatar

    Will, I agree that in the SMB space the phenomenon that Stephen outlines could be more pronounced. In that space, cost is generally the primary factor and features/function are less important. In that case, VMware’s innovation could produce a homogenization of storage. Good point!

    Ed, agreed—it’s about more than just VMware, although VMware is (today) a large part of it and will be (in the future) an even larger part.

    Donny, I can’t say that I agree completely. There are still enough differences in hardware architectures that allow vendors to differentiate and show value. Until those hardware differences melt away—as they did in the PC space, where everyone was using the same processors and the same chipsets—the physical assets won’t be completely commoditized. Further, even if the hardware were to be commoditized, this still doesn’t support Stephen’s arguments; storage vendors would *still* have numerous ways to differentiate themselves through varying levels and types of VMware integration and features/functionality.

  5. Stephen Foskett’s avatar

    Thanks for reading and your thoughtful reply, Scott! I can always count on you for that!

    I was indeed referring to high-end arrays of the sort that have become EMC’s bread and butter. I already see many VMware admins (admittedly at smaller shops) turning towards “fast and cheap” storage from the likes of Xiotech (I won’t use that new name), Nexsan, Overland, DataDirect, and so many others. Yes, it’s a storage array, but it’s really a different animal. In other words, they’re skipping the whole storage world and focusing on VMware. This is half the equation.

    The other half is the enterprise space, where arrays from EMC, NetApp, HDS, and others are becoming mere vessels for VMware functionality. End users are leveraging their features through VAAI but knowingly and intentionally skipping anything else. It’s all about the VMware.

    The goal of that post wasn’t to suggest VMware is killing EMC on purpose. It was to say that VMware is destined to harm and perhaps kill the storage industry by commoditizing hardware. And, as Sophocles said, one cannot escape fate.

  6. slowe’s avatar

    Stephen, I’m far from being a storage expert like yourself and probably don’t have the same visibility into the market that you do. While this trend of “fast and cheap”—by your own admission—is primarily in the smaller shops, I have to question whether this is a byproduct of VMware’s presence or if it is simply a byproduct of increasing economic pressures. It doesn’t seem to me that we can clearly delineate exactly *what* might be driving these buying patterns.

    As for the enterprise space, I certainly can’t argue that all the major players are focusing quite heavily on the VMware use case. But are the vendors focusing on it to the exclusion of other use cases? And are users intentionally skipping the array’s features? From what I’ve seen—and again, I probably don’t have your visibility into the market—users are still very interested in array-bound technologies like thin provisioning, snapshots, replication, and auto-tiering. I would further say that if storage companies are being challenged by VMware to show their relevance and differentiate themselves, this is no different than many other competitive challenges. A storage company that doesn’t innovate and evolve will grow stagnant and die. That’s not VMware’s fault—that’s the market’s fault.

    Finally, I understand the point of the post. The crux of my response is that I don’t believe the trends you’ve identified necessarily point in the direction of an Oedipal confrontation between VMware and storage vendors. Besides, I don’t believe in fate. God gave Man free will. :-)

  7. Michael’s avatar

    Great post. That’s quite an interesting situation and I wonder if EMC ever foresaw this when they purchased VMware almost 8 years ago.

    It will be interesting to see what EMC’s approach to this will be over the next 12 months or so. Do they feel threatened by this homogenization (as far as it goes)? Will they try to exert pressure on VMware to stem it (or, as I expect, is it too late to stop that)? I can’t help thinking that to prevent VMware innovating would be a gift to other virtualization vendors who are not tied in the same way and a bit of an “own goal”.

    Personally I think that the momentum of change is too big to stop. EMC et al will roll with it and continue to innovate and differentiate in other ways. Some of the newer storage vendors will gain traction, some will not – that’s the way it often is with technology companies.

  8. Paul C’s avatar

    I’ll agree that VMWare is removing array requirements for smaller shops, especially if they can drag a clusterfs into the base hypervisor … functionality like provided by gluster or ocfs inside the hypervisor would potentially allow users to take advantage of the advanced features (DRS,etc) without needing an expensive SAN array.

    However as a heavy Compellent user ( auto-tiering FTW ) I think for larger shops/Enterprise will still have a requirement for the benefits of Enterprise SAN and those SANs need to move their feature sets to complement virtualization, not compete. Features of which have already been mentioned Auto-Tiering is probably the one killer feature right now. I can have 15k RAID10 performance for little more than the cost of 7.2k RAID5 with Auto-Tiering which is a really big deal when you’re talking hundreds of terabytes.

  9. Donny Parrott’s avatar

    Scott, you are correct in identifying the differences in capability between VMware and array vendors. There is far more capability within the array space; but is it needed?

    VMware is driving both the commoditization and isolation of capabilities. By this I mean that functionality is becoming independent of hardware and selectable by use case. For instance, a recent discussion focused on network requirements in a virtualized environment. The Nexus 1000v, vShield, and vSwitch were all candidates. This allowed the customer to identify the exact feature set required and best tool to match.

    I believe, like above, that storage is headed the same way. Specific requirements, specific solution. As VMware’s capabilities expand, the requirements of the array shrink…

  10. slowe’s avatar

    Michael, personally I think there are ways that storage vendors (including EMC) can embrace this change and continue to offer value to customers above and beyond what features/functions are rolled into the hypervisor stack. Historically, the storage features/functions have been limited in scope and scale, which leaves one area of value (among several) for storage vendors.

    Paul C, you and I agree that auto-tiering is just one example of how storage array vendors can continue to offer value that complements the features and functions in the hypervisor.

    Donny, as with all technologies, there are some features and functions that eventually get adopted by all vendors in a space; that commoditizes that functionality and forces the vendors to continue to innovate. In my opinion, VMware is a similar influence. Storage vendor value won’t disappear; instead, storage vendor value will appear in different ways. For example—and forgive me for using an EMC-specific example here—consider VPLEX. The ability to have read/write storage in two locations simultaneously is incredibly powerful. This is just another example of how a storage vendor can add value in ways that complement the functionality VMware builds into the hypervisor. Further, as I mentioned earlier in this comment, VMware’s storage features/functionality have historically been limited in scope and scale, which provides yet another area for storage vendors to differentiate themselves.

  11. FredK’s avatar

    Hi,

    Nice analyze, and i agree to almost everything :)

    About :
    “vSphere’s replication functionality is a nice “check box” item, but lacks many of the features that array-based replication offers. For example, there’s no compression, no deduplication or WAN optimization, and no idea of consistency groups.”

    Can’t we state that the vSphere Replication utilize optimization too with the PSF (persistent state file) ? I mean, if we consider that PSF make the bigger part of the work, compression and deduplication on the physical arrays will have less work to do in fact, because very few datas will have to be replicated.

    But this is just to argue with you :) I agree with your point of view in fact.