EMC Celerra Optimizations for VMware on NFS
Published on 31 Jan 2010 · Filed in Explanation · 591 words (estimated 3 minutes to read)Recently Jason Boche posted some storage performance numbers from his EMC Celerra NS-120. Out of those storage performance tests, Jason noted that the NFS performance of the NS-120 seemed a bit off, so he contacted EMC’s vSpecialist team (also known as “Chad’s Army”, but I like vSpecialists better) to see if there was something that could or should be done to improve NFS performance on the NS-120. After collaborating internally, one of our team members (a great guy named Kevin Zawodzinski) responded to Jason with some suggestions. I wanted to reproduce those suggestions here for everyone’s benefit.
Note that some of these recommendations are already found in the multi-vendor NFS post available on here on Chad’s site as well as here on Vaughn Stewart’s site.
In addition, most if not all of these recommendations are also found in the VMware on Celerra best practices document available from EMC’s web site here.
Without further ado, then…
-
As has been stated on multiple occasions and by multiple people, be sure that virtual machine disk/application partitions have been properly aligned. We recommend a 1MB boundary. Note that Windows Server 2008 aligns at a 1MB boundary automatically.
-
Use a block size of 8KB unless other recommended or required by the application vendor. Note that the default NTFS block size is 4KB. (Pages 128 through 138 of the Celerra best practices document contain more information on this bullet as well as the previous bullet.)
-
Turn on the uncached write mechanism for NFS file systems used as VMware datastores. This can have a significant performance improvement for VMDKs on NFS but isn’t the default setting. From the Control Station, you can use the
server mountcommand with the “uncached” option to turn on the uncached write mechanism. The whole command would look something likeserver_mount <data mover name> -option <options>,**uncached** <file system name> <mount point>(the asterisks around “uncached” are for emphasis; don’t actually include them in the command). Be sure to review pages 99 through 101 of the VMware on Celerra best practices document for more information on the uncached write mechanism and any considerations for its use. -
Change the VMware ESX settings
NFS.SendBufferSizeandNFS.ReadBufferSizeto a value that is a multiple of 32. The recommended value is 64. See page 73 of the best practices document for more details. -
If you’ve adjusted the
NFS.MaxVolumesparameter in order to have access to more than 8 NFS datastores, you should also adjustNet.TcpIpHeapSizeandNet.TcpIpHeapMaxparameters. The increase should be proportional; if you increase the maximum volumes to 32 (a common configuration), then you should increase the other parameters by a factor of 4 as well. Page 73 of the best practices document covers this. This VMware KB article and this VMware KB article also have more information. -
Although not directly related to performance, best practices call for setting
NFS.HeartbeatFrequency(orNFS.HeartbeatDeltain VMware vSphere) to 12,NFS.HeartbeatTimeoutto 5, andNFS.HeartbeatMaxFailuresto 10. -
Ensure that the LUNs backing the NFS file systems are allocated to the clar_r5_performance pool. This configuration will balance the load across the SPs, LUNs, etc., and help improve performance.
Depending upon the other workloads on the system, another NFS performance optimization is to ensure that the maximum amount of write cache on the SPs is configured. However, be aware this may impact other workloads on the array.
As Jason noted in his post, implementing these changes—especially the uncached write mechanism—offered performance benefits for NFS workloads.
Keep these configuration recommendations in mind when setting up your EMC Celerra for VMware on NFS.