All comments are passed through a spam filter and then moderated before they appear. This is done to help prevent comment spam. Sorry for any confusion!

Tony, slowe, thanks for your comments.

I’ve just read http://www.netapp.com/go/techontap/matl/downloads/NAS-presentation.pdf .
They state to take care about NFS and single link usage on slide 13.

It is said : “NFS traffic uses singleVMKernel IP address per vSwitch / set of team NICsThis means that a single uplinkwill be selected for ALL traffic with:
•Load balancing based only on source IP/MAC/Port
•Load balancing with source IP and a SINGLE DESTINATION IP
•Single datastore”

Does it means that NFS traffic will only uses one uplink whatever the configuration selected on the NIC teaming on the ESX server Storage Vmkernel; if only one IP adress is used on the storage side, even with a VIF ?

To avoid this : does it means that I should
* on the ESX side, select a Load Balancing policy and the same on the Cisco switches

* on the NetApp side, declare the same number of IP adresses as the number of active-active trunking NICs. So 2 IP adresses for 2 x 2 NICs in a two level VIF like in the document ’s example.

* Create a DNS entry for each IP adresses.(like nfs1.toaster.company.com, nfs2.toaster.company.com)

* Then when connecting the datastores on an ESX, I should use the nfs1.toaster.company.com name and nfs2.toaster.company.com alternatively ?

* Spread VM on all volumes

Thanks

ML

I’d love for others with more experience to respond as well, but I did want to point out that NFS only performs better when you have multiple (at least 4-6) VMs running at the same time. If you ran your test with a single VM, FC or iSCSI will always outperform NFS.

Could you give us the export options of the NFS share you used for VM ? Do you use Jumboframes ?

I read on netapp doc’s that in Vmworld 2006, it was said to map VM swap file in a VMFS in iSCSI rather than NFS.
ML

Yes Netapp is proud of their NFS, however it’s worth it due to the snapshot/snapmirror/DFM/etc/et/etc integration and it works 100%. We also use NFS for Oracle and Sybase. Yes, Oracle is a HUGE fan of Netapp NFS. They are even integrating NFS directly into 11g and soon pNFS!

You can use NDMP to backup directly from the Netapp… VCB? don’t need it…. Netapp basically has it built in. We decided instead to snapmirror to another Netapp for backups and keep 21 days of snapshots. As an added bonus, in the event of a major failure (virus, data corruption, fire, etc) we could failover our entire VM INFR to the secondary Netapp in a matter of minutes. A complete restore of our SAN solution could have taken months to restore from tape.

ASIS is deduplication of redundant blocks. Since most VMs are just clones, ASIS can save you 50-80% on storage cost. ASIS is still somewhat new to Netapp, however it appears rock solid.

We currently don’t run IO intensive database VMs, and after monitoring the IO load of our VMs on our SAN, the decision was made to go with NFS.

With the new Flash based disk systems like FusionIO announed today, ( http://viroptics.blogspot.com/ )
it won’t be long before we virtualize even the largest databases and IO intensive servers across a protocols like NFS over 10Gb links.

I’m calling this DANFS or Direct Attached NFS. Maybe I’ll need to change it to Direct Attached Network Flash Storage

Anyway, NFS is a Ethernet based protocol and allows for many options like replication and easy failover of systems. Since VMs are just files, I don’t understand why some have a hard time with a protocol that working on files natively for 20+ years.

If you have gone down the SAN/iSCSI path and have a hard time sleeping nights, NFS may be a great way to get some shuteye…

Dan

I’m not Dan, but I can answer a few questions:

- CIFS is not an option–NFS is only an option because it’s supported by VMware ESX Server. CIFS, in my opinion, is far too chatty.

- A-SIS is “Advanced Single Instance Storage”, where the NetApp storage system de-duplicates the data (eliminating duplicate data) at the block level–below the file system level.

I’ll leave it to Dan to address his experience with NFS. Thanks for reading!

I’m very interested in your NFS experiences. I’m assuming by your mention of snapmirror, you are using netapp - so are we. But we’ve got most of our VM’s on netapp via FC.

Here are my questions:

With NFS, do all of the VMDK files sit on the netapp filesystem in the same way that any other file would?

Are there concerns relating to 1Gbps NFS as opposed to 2 or 4Gbps Fibre Channel?

What netapp models do you use? We’re primarily using 3020’s.

We don’t use NFS anywhere else in our environment, so we’d have to purchase the fairly expensive NFS license. Were you already using NFS for other needs?

Your thoughts on using CIFS as opposed to NFS?

What is ASIS?

thanks!

Eric

We are now running over 950 VMs via NFS on 35 ESX hosts. So far we see only positive benifits of using NFS over iSCSI or FC.

We are just about ready to turn on ASIS on the primary cluster which will more than double our storage capacity.

* Snapmirror all VMs to another filer for backups and DR

* 21 daily snapshots of ALL VMs for backup and quick recovery

* No TAPE backup (unless archive is required)

ESX3i should be another plus for NAS over FC due to the cost and added compexity of FC

We will start deploying ESX over 10G NFS this year.

FC? iSCSI? Why?