Using RHEL/CentOS with KVM as the hypervisor and the shared storage between nodes would be a Lustre FS mount. Create all the VMs on the Lustre mount and use 10gb Ethernet or the fastest supported interface by the OS on each physical host for the connection into the Lustre network. Have the same connectivity on the Luster metadata and storage controllers and you would have a very robust solution with shared storage. All without having SAN hardware or software lock in.

Earlier this morning, Scott Lowe posed the following question:  What if hypervisors shared a file system? The concept here is that most hypervisors (notably VMware and [soon] Hyper-V) have a clustered file system that is…

I simply do not think the hypervisor should own such a thing. Why would it? Think about networks – as an example – a system hooks up into the IP network and uses the services provided. It does not own them or manage any aspect of it. It simply leverages the service so it can concentrate on on what it does.

Why should this be any differnt with file systems? It seems we stuck in a crossroads looking longingly back into the sunset of proprietary OS’s rather than looking ahead to the bright day break of applications riding unhindered on their storage and network.

I am a storage guy. I worked for DEC, EMC and now NetApp promoting storage solutions. Philosphically, I think we have to look to storage to provide enterprise file services (like the network provides IP services for example). I like to manage the data where it resides.

In fact, if you run with what is out there today, the problems are half way solved – now you just need cooperation on VM file formats… but I won’t be holding my breath for that.

If they do cooperate, then they can take advantage of tried and true filesystems and free up everyone from hardware choices that do nothing but provide expensive vendor lock in. They’d very likely drop the cost of developmetn way down – and hence raise revenue.

Just a thought.

This is great and relevant discussion. Thanks for starting it.

I’d suggest that the shortest path to success with this would be a file system that runs on Linux – maybe one of the currently available ones (GFS??) could be used and further developed to add whatever clustering functionality is needed specifically for hypervisors and their guests.

Finally the abstraction layer needed to support whichever hypervisor there is on top of it. This is probably where the hardest bits would be – requiring qualification for all the various release levels of all the parts that touch FS operations. This is where patches tend to get ugly.

Now, if that were a cluster FS, thereby eliminating the risks of a single NFS data source it would be all the better. Problem is, there are quite a few of these animals out there already. GFS and OCFS come to mind immediately, and I think Lustre fits the model as well. They are typically complicated to set up, finicky, and used only in highly specialized environments.

The biggest chance of something like this happening, IMHO, is pNFS. I don’t know a lot about it, but as I understand it it basically adds cluster scalability to NFS. VMWare and Xen already talk NFS, so it’s a natural next step. There are third party NFS clients for Windows as well, so a hyper-v port wouldn’t be out of the question.

WRT the security issues… Yes, there will be some… but they are just “different”, not necessarily worse. NFS access controls do work, they are just different than lun masking and the like. Coming from a Unix background instead of a storage background, I find I am much more confident in my ability to limit access using NFS than iSCSI =p

I definitely think the benefits outweigh the disadvantages!

Good points about the guests reading the same file system as the hosts; that would indeed present some unique security issues.

However my concern with the guests understanding the same cluster file system that the hypervisor could open a security risk. The way it is now no guest can read the file system to see the other vmdks. If they understand the file system the hypervisors would have to greatly change their permissions on those files to keep one guest out of another’s data.

I think there’s no chance of this happening. Firstly it would require vendors to open their formats for competitor evaluation and/or agree a standard format. We’ve not seen that happen in the storage array area where nearly every vendor implements their technology in a different way. The same thing occurs with the fabric where each vendor matches the basic standards but diverges with their own feature set.

We shouldn’t be surprised by this; vendors need to offer some value-add, otherwise they reduce to offering commodity products, reduce/stop innovation and become irrelevant.

Now, although I don’t think it will happen, I do think we need it. There’s far too much multi-standard stuff in IT which wastes resource/time and effort. The balance is getting that level of standardisation correct, without stifling innovation and creativity and so advancing the technology.