I supposed I can assign the SAN NIC’s different IP’s and set up multipathing… I dont know if that will provide a performance increase though…

For your info, I’d like to share my hard to believe experience in configuring my iSCSI SAN with you here:

http://img38.imageshack.us/img38/1397/deployment.jpg

MD3000i is just a small entry level SAN device which can only use one single cable to access the iSCSI target, so no matter how complex the configuration is, the I/O performance will not be as great as the adding managed switch to perform VLAN trunking.

According to the following blog:
http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-mutual-iscsi-customers-using-vmware.html –> the last question #4 is the eye opener

so by using the deployment diagram that i supplied on top, i have to accept that it is not possible to achieve high performance greater than single cable connection due to the limitation of the ESX Sofware iSCSI initiator. Even by using the Intel Pro 1000 TOE enabled pNIC it’s all the same slow result.

hope that helps you in the future,

I feel bad after spending this much money without any greater performance of my Local Server RAID-5 SATA drive

I just wanted to say, I stumbled upon your blog via a google search for vmware. I’d just like to say thanks for creating it and it looks as though you have a real good, helpful community here too.

I’d also like to say I’ve learned a good few things myself from reading your posts, keep up the good work it’s rather inspiring.

Kind regards

Scott

You’re absolutely correct, of course; I pointed that out in my comment earlier (http://blog.scottlowe.org/2007/06/26/optimizing-iscsi-traffic-with-esx/#comment-32537), but probably didn’t state it clearly enough. Thanks for the clarification!

The other side of this is specific to the ESX iSCSI initiator. To get around the limitation with GigabitEtherChannel (or NetApp VIFs), the idea would be to create multiple iSCSI targets (with different IP addresses) so that the software iSCSI initiator would more effectively use multiple links. EtherChannel is great, as you’ve pointed out, for helping to keep multiple connections from fighting over bandwidth. I was trying to tackle more effective utilization from a single-server perspective.

Thanks for your comment!

In our scenario, I currently have 3 ESX servers (pretty beefy, 12 or 24 GB RAM, 4 or 8 core). For the iSCSI link, I have a 2 port gig etherchannel setup going to a Cisco 3750 switch. That switch then has a dual gig etherchannel going to a NetApp 3050 clustered head (one dual gig etherchannel to each head actually).

The Cisco 3750 has an RPS on it to give it dual power supplies (one AC, one DC).

So far it’s been very stable.

So….if you have 1 ESX server with a dual gigabit etherchannel going to a NetApp, you have fault tolerance (one gig link can go down with no problems) but no bandwidth increase.

If you have 2 ESX server with a dual gigabit etherchannel going to a NetApp, each server can use up to a full gigabit link (so you don’t really get a bandwidth increase per server but do keep the servers from fighting over bandwidth).

vSwitches? The only ones that would likely see improvement is the VMKernel port vSwitch. non-iSCSI traffic seems to do fine with the default setting. There should be one vSwitch that has the VMKernel port, with teamed pNICs in IP HASH teaming. Then use multiple (two is fine) EQL volumes. You should see good activity on both pNICs.

Don

I haven’t created a formal, quantitative testing plan; it’s more “off the cuff.”

I do have the vSwitches configured for IP hash teaming mode, but thus far haven’t seen any real difference in the various configurations I’ve tried. I intend to conduct more testing, more fully documenting each of the test scenarios, and then I’ll post more results here.

How are you testing the results? Did you change the teaming mode to IP HASH? Customers who have made the change notice an ‘improvement’ I usually see the result in the network stats. You see better utilization of both links.

It’s not documented anywhere that I know of. I found it by experimentation and observation. Passed it to customers who reported good results. Excluding one who had trunked switches and stand-by pNICs. Then it fell down.

Don