Thanks for the great article, even it’s aging a bit

I just wanted to ask if you think there’s still a need for a stack and etherchannel configuration, especially for smaller farms (less than 10 ESXi)?
Just because there’s a new load balancing setting in v5 called “route based on NIC load” which changes VM’s vNIC association if needed (when over 70% of the physical link’s used, I believe) and therefore guarantees better load distribution over the physical NICs than the good old settings.

Another question regarding stack configuration. Let’s say I use two stackable switches and stack them to stack-A, then I use another two of the same kind and stack them to stack-B because of redundancy needs. The two stacks are then interconnected with a 2x10Gbit etherchannel.
But now I have to connect my ESXi server’s interfaces (8) to each stack (4 and 4) because of redundancy reasons and in return I’m loosing 50% of the possible link aggregations I could configure if I’d cable them all to one stack. Isn’t there a possibility to configure an etherchannel spanning over the two stacks? Or am I getting something wrong here…

Thanks,
Roman

I hope this helps!

I’ve dealt a lot with NIC bonding on linux. I’ve conducted many tests. There are many different algorithms available. However, one of the major flaws is the inability to efficiently load balance in a routed environment. Balancing by TCP stream is impossible when traffic is passing through the gateway. I haven’t had the opportunity to test multiple source macs from the same host. But can you confirm the following:

- Etherchannel on, Route based on IP hash, 3 nics teamed

Connection 1:

VM (source 00:AF) est connection to 8.8.8.8 via gateway 192.168.0.1 using pnic1

Connection 2:

VM (source 00:AF) est connection to 9.9.9.9 via gateway 192.168.0.1 using pnic2

Connection 3:

VM (source 00:AF) est connection to 10.10.10.10 via gateway 192.168.0.1 using pnic3.

—
If the above is true, how exactly is the load distributed, is it a +1 for every new destination address? Or is there a mechanism that determines the capacity of the pnic, and moves on to the next

Let me reiterate—there are too many variables for me to simply say, based on the limited information you’ve provided, whether this is an optimal design or not. For example, you don’t specify how the hosts are going to be cabled to the switches. You also don’t specify what NIC teaming policy you are going to specify on the vSwitch, or if you’re using vSwitches, VMware distributed vSwitches, or the Cisco Nexus 1000V. You didn’t indicate how you were planning on handling iSCSI multipathing, or whether you’re using software iSCSI (most likely) or hardware iSCSI.

As you can see, there is quite a bit of information that is still unknown. Because of this, coming here and asking me to tell you if I see any issues is kind of pointless—I don’t have enough information to be able to accurately identify any issues.

I encourage you to work with your VMware partner for a full assessment of the proposed network and storage design.

FC
3560G + 3560G——————————-3750G + 3750G << Client Access
/\ /\ /\ /\ /\—multimode 10 GB VIF
ESXi ESXi ESXi ESXi NetApp

Please advise if you find any issues with this setup.

I would encourage you to engage the services of a skilled VMware specialist to help you vet out your design. There are simply too many variables in such a situation for me to be able to say, “Yes, that’s a good design” or “No, that’s a bad design.” If you’d like to provide some geographical context, I might be able to help recommend a good VMware partner in your neck of the woods.

Please advise

You should only use EtherChannel (and the matching load balancing policy, “Route based on ip hash”) at the vSwitch level, not at a port group level. You can set it at a port group level, but it won’t work properly. This means you should use different vSwitches for different EtherChannel groups.

Another question; do you know the best practice for setting this up? Meaning, a different vSwitch for different ether channel groups? Or, one vSwitch, different port groups, with the port group LB policy overwriting that of the vSwitch and manually moving only those adapters in the eth chan group under active adapters? I have looked a bit on this and can’t find anything conclusive on which is better.