Great articles (the other iSCSI one is excellent), I appreciate you taking the time to share what you have done. I began working on a project creating a SAN for our VMware lab with hopes of hosting some business non-critical systems on a varient of that configuration. However, onto the second phase of that project, moving from ESXi 3.5 to ESX 3.5 I cannot get the production VMware host to see anything as provided by the iSCSI Enterprise Target. Windows finds the target just fine.

The iSCSI SAN in production is on a different subnet, therefore I created an additional vswitch with service console and vmkernel. The firewall is configured with open ports for iSCSI.

VMware support doesn’t support IET, therefore I had a hard time getting any troubleshooting tips from them. I have been digging so deep, I am beginning to wonder if the problem is surface level…

Any ideas?

-Aaron

I haven’t performed any objective comparisons between IET and the NetApp F8xx series that I’m now using in the lab. Subjectively, however, the NetApp feels noticeably faster than the IET implementation I had previously. That is to be expected since the NetApp uses 14 physical disks to handle the traffic instead of just 4 disks with the IET setup.

As for stability, I had the IET daemon just crash on a number of occasions, taking down the VMware farm due to the lack of access to the VMDK files. Everytime this happened, I had to restart the IET daemon, reboot all the VMs (occasionally having to rebuild VMs), and then continue from there.

As for the NetApp “large price tag,” there’s more to NetApp than just iSCSI storage–consider Snapshots, FlexClones, SnapMirror, and all the other features that are made possible on NetApp storage. I’ve written about a few of these features:

http://blog.scottlowe.org/2007/05/15/netapp-flexclones-with-vmware-part-1/
http://blog.scottlowe.org/2007/05/17/netapp-flexclones-with-vmware-part-2/
http://blog.scottlowe.org/2007/05/11/how-to-provision-vms-using-netapp-flexclones/
http://blog.scottlowe.org/2006/12/30/recovering-data-inside-vms-using-netapp-snapshots/

I hope to be able to talk more in-depth soon about SnapMirror in VMware environments, as well as discussing the use of NetApp storage to provide NFS-based VMFS datastores.

I am interested in hearing how the NetApp appliance compares to the iSCSI Enterprise Target performance-wise. Also, what were some of the stability problems you experienced with IET?

I have had good luck with IET (outside of some bad LSI drivers initially) and want to know what NetApp provides performance-wise, or is it stability that is the factor for the large price tags?

At the time, I was simply bonding the NICs within ESX Server, and I didn’t have any problems with that. I didn’t try any bonding on the iSCSI target; instead, I used multiple NICs with multiple IP addresses and manually “load balanced” the inbound traffic. This was using an unmanaged Gigabit Ethernet switch, by the way.

Since then, I have moved to a managed Cisco Gigabit Ethernet backbone and setup link aggregation (802.3ad) and trunking to the ESX Servers, and I switched to a real Network Appliance storage system for iSCSI storage. Unfortunately, the NetApp has only a single Gigabit Ethernet NIC, so I can’t test any bonding on the storage side. You can read more abou the link aggregation and trunking configuration here:

http://blog.scottlowe.org/2006/12/04/esx-server-nic-teaming-and-vlan-trunking/

Good luck,
Scott

you mention that you use ESX NIC bonding for accessing you iSCSI-target. I’m trying to do the same, but can’t get it to work (a single connection works perfectly, but I would like to have a bit more performance and some fault tolerance).

Can you eleborate some more about the settings you made on the ESX-side to take advantage of NIC bonding.
I’m using Open-e iSCSI target (wich is based on linux and uses the Linux Ethernet Bonding driver)

Kind regards,

Richard

I’m continuing to use the iSCSI Enterprise Target in the test lab, and have performed some performance tuning. Multipathing isn’t really an option; this is due to the way ESX Server 3.0.1 implements the software iSCSI initiator. Using a hardware iSCSI initiator would change that situation, of course. Taking advantage of ESX Server’s NIC bonding, however, seems to work very well.

My only complaint about iET is that it isn’t quite as stable as I would like; it’s crashed a couple of times and taken down the VMware farm. (Interesting side note: Some VMs are more tolerant of the loss of disk connectivity than others. Windows, for example, doesn’t like it very much.)

Thanks for the link! Yes, I’ll be at VMworld this year (my first time!) and I’m looking forward to it. Hope to see you there.

Great post; I linked to it from the VMTN Blog. Good luck with the project, and be sure to check out the VMTN Forums if you run into difficulty.

Feel free to drop me a line if I can be of any assistance. Let me know if you’re going to VMworld — I’m coordinating blogger activities and resources there.

Regards,
John