Interesting New VMware Storage Product

This little blurb popped up in my virtualization pipe this morning and, intrigued by what this might mean, I followed the link.  It turns out that LeftHand Networks is introducing an virtual appliance that takes unused local storage on servers running ESX Server and turns it into reusable iSCSI SAN storage that can then be re-presented to those same ESX servers for use in storing virtual machines.  Very interesting!

One question that pops up in a lot of ESX deployments is the question of how to handle local disk storage on the ESX servers themselves:

  • Should the ESX Servers even have local storage, or should they boot from SAN?
  • Should we create a local VMFS datastore?  If so, how large should it be?
  • What can local VMFS datastores be used for?

Personally, I generally recommend local storage unless the customer has strong feelings or a good business case otherwise (it eases the complexity of the implementation), and I recommend creating a local VMFS datastore.  (Like me, some of you out there have probably seen strange things that occur when a local VMFS datastore is not created.)  If nothing else, these local VMFS datastores can be used as P2V destinations, destinations for cloned test VMs, etc., when you don’t want to use (generally expensive) SAN storage.  Now it appears that LeftHand Networks will let us use the extra storage on the ESX servers to create a SAN.

This is an interesting announcement, and it brings up additional questions that haven’t yet been answered (Will this SAN allow for the use of VMware technologies such as VMotion, DRS, or HA?  What happens to the SAN when a member host whose local storage is being used in the SAN fails?  How will the SAN maintain redundancy and data consistency in such sitautions?  How will the performance of this solution compare to a dedicated SAN?  How does the pricing of this solution compare to a dedicated SAN?)  Hopefully I’ll be able to get some answers from LeftHand, since they’ll be here in San Francisco this week for VMworld.

Tags: , , , ,

  1. John Spiers’s avatar

    This is an interesting announcement, and it brings up additional questions that haven’t yet been answered (Will this SAN allow for the use of VMware technologies such as VMotion, DRS, or HA?

    Yes, LeftHand’s Virtual appliance works with VMotion, DRS & HA.

    What happens to the SAN when a member host whose local storage is being used in the SAN fails?

    If you have LeftHand’s VSA running on more than one physical server, you can cluster & virtualize the local storage together from any number of servers and run Network RAID on the iSCSI volumes across those servers. If any server fails the remaining servers still have access to the SAN, and all volumes are still on-line (similar to a disk failing in a RAID set), and the failed server’s disk can be incrementally rebuilt (outage with data intact), or completely rebuilt (hard failure of server and disks) from the other servers. VSA appliances can also be moved with VMotion and provide seamless failover and failback with DRS & HA.

    How will the SAN maintain redundancy and data consistency in such situations?

    LeftHand’s Virtual Storage Appliance is based on the SAN/iQ software, which provides redundancy with Network RAID across servers and guarantees data consistency using a 3-phase commit protocol across servers.

    How will the performance of this solution compare to a dedicated SAN? How does the pricing of this solution compare to a dedicated SAN?)

    Performance depends on the type of server and its internal disk system, i.e. CPU, memory, drive type, controller, etc. LeftHand has performance metrics for a variety of servers and disk systems that we’ve ran this on and it’s very impressive. Pricing is less than a complete LeftHand SAN (HW+SW) because you’re not buying the hardware.

    John

  2. Dan’s avatar

    Should the ESX Servers even have local storage, or should they boot from SAN?

    I Say NO

    ESX virtualization hardware appliance using flash storage boot device is the right direction.

    Just add Power, Network and Storage.

    DanFS Direct Attached NFS is my prediction for Storage. Simple. No VMFS, no ISCSI, no FC, no Problems… 10Gb DanFS!

  3. slowe’s avatar

    John Spiers,

    Thanks for the response. I plan to stop by and talk to LeftHand Networks while I’m here in San Francisco at VMworld 2007. It would be great to be able to look at some of those numbers and discuss implementation details.

    Dan,

    VMware on NFS is getting lots and lots of attention these days, so you’re not alone in liking NFS, although I think some people may take exception with your choice of names. :-)

    Of course, with the public announcement of ESX 3i this morning at VMworld Partner Day, the move toward no local storage will definitely pick up steam.

  4. John Spiers’s avatar

    I’ll have to chime in one more time. I’ll have to agree with Dan here. The benefits of having all your storage out on a SAN, all centrally managed, are numerous. Booting from the SAN takes it one step further, eliminating the need to manage local disks in your servers all together. This is why LeftHand sells complete IPSANs, and has 100s of customers booting off them as well.

    This being said, there are some customers that may not want to invest in a SAN, or may have already invested in Direct Attached Storage. These customers could benefit from creating a SAN out of their current islands of storage attached to servers.

    The good news with LeftHand is that you can have it either way or both.

    As far as NFS goes, it’s a valid point in some respects, but it opens up a whole new can of worms, including performance, open file security issues, metadata locking, and the requirement for an expensive Distributed File System if you want to virtualize, cluster and load balancing across storage systems like LeftHand does. It also requires parallel extensions for NFSv4, which are not cooked yet. Please, don’t get me started on NFS.

    John

  5. slowe’s avatar

    Well, John, I’d be surprised if you didn’t agree with Dan, but you did at least recognize that some customers have already invested in DAS. That’s better than I can say for some vendors…

  6. PaulM’s avatar

    Hi all,

    I’ve never posted before but have found the site an invaluable tool for technical issues, and just for passing the hours.

    I have been asked to install ESX 3.5 into a new cluster (only two ESX hosts at initial build). The customer requires VMotion and DRS but I have only jsut been involved in the project and the kit has already been ordered.

    The problem I have is that the storage for the system is the IBM DS3200 which will be configured with redundant SAS HBA’s to both ESX hosts, and to a third Windows server.

    I have always worked on the basis that VMware requires shared network storage and either NFS, iSCSI or FCP to leverage the benfits of DRS and Vmotion, but upon digging further, am beginning to doubt this.

    IBM have a document that suggests that the DS3200 is now certified by VMware for use in a Vmotion enabled VI3 cluster since ESX 3.5.1.

    Any thoughts on this?

    Paul

  7. DannyW’s avatar

    Hi! Paul,

    Were you able to get the DS3200 to work with VMotion and DRS? If so how did you set it up?

    Thanks!