VMware ESX, NIC Teaming, and VLAN Trunking with HP ProCurve

In an earlier article about VMware ESX, NIC teaming, and VLAN trunking, I described what the configuration should look like if one were using these features with Cisco switch hardware. It’s been a quite popular post, one I will probably need to update soon.

In this article, I’d like to discuss how to do the same thing, but using HP ProCurve switch hardware. The article is broken into three sections: using VLANs, using link aggregation (NIC teaming), and using both together.

Using VLAN Trunking

To my Cisco-oriented mind, VLANs with ProCurve switches are handled quite differently. Port-based VLANs, in which individual ports are assigned to one or more VLANs, allow a switch port to participate in that VLAN as either an untagged fashion or in a tagged fashion.

The difference here is really simpler than it may seem: the untagged VLAN can be considered the “native VLAN” from the Cisco world, meaning that the VLAN tags are not added to packets traversing that port. Putting a port in a VLAN in untagged mode is essentially equivalent to making that port an access port in the Cisco IOS world. Only one VLAN can be marked as untagged, which makes sense if you think about it.

Any port groups that should receive traffic from the untagged VLAN need to have VLAN ID 0 (no VLAN ID, in other words) assigned.

A tagged VLAN, on the other hand, adds the 802.1q VLAN tags to traffic moving through the port, like a VLAN trunk. If a user wants to use VST (virtual switch tagging) to host multiple VLANs on a single VMware ESX host, then the ProCurve ports need to have those VLANs marked as tagged. This will ensure that the VLAN tags are added to the packets and that VMware ESX can direct the traffic to the correct port group based on those VLAN tags.

In summary:

  • Assign VLAN ID 0 to all port groups that need to receive traffic from the untagged VLAN (remember that a port can only be marked as untagged for a single VLAN). This correlates to the discussion about VMware ESX and the native VLAN, in which I reminded users that port groups intended to receive traffic for the native VLAN should not have a VLAN ID specified.
  • Be sure that ports are marked as tagged for all other VLANs that VMware ESX should see. This will enable the use of VST and multiple port groups, each configured with an appropriate VLAN ID. (By the way, if users are unclear on VST vs. EST vs. VGT, see this article.)
  • VLANs that VMware ESX should not see at all should be marked as “No” in the VLAN configuration of the ProCurve switch for those ports.

Using Link Aggregation

There’s not a whole lot to this part. In the ProCurve configuration, users will mark the ports that should participate in link aggregation as part of a trunk (say, Trk1) and then set the trunk type. Here’s the only real gotcha: the trunk must be configured as type “Trunk” and not type “LACP”.

In this context, LACP refers to dynamic LACP, which allows the switch and the server to dynamically negotiate the number of links in the bundle. VMware ESX doesn’t support dynamic LACP, only static LACP. To do static LACP, users will need to set the trunk type to Trunk.

Then, as has been discussed elsewhere in great depth, configure the VMware ESX vSwitch’s load balancing policy to “Route based on ip hash”. Once that’s done, everything should work as expected. This blog entry gives the CLI command to set the vSwitch load balancing policy, which would be necessary if configuring vSwitch0. For all other vSwitches, the changes can be made via VirtualCenter.

That’s really all there is to making link aggregation work between an HP ProCurve switch and VMware ESX.

Using VLANs and Link Aggregation Together

This section exists only to point out that when a trunk is created, the VLAN configuration for the members of that trunk disappears, and the trunk must be configured directly for VLAN support. In fact, users will note that the member ports don’t even appear in the list of ports to be configured for VLANs; only the trunks themselves appear.

Key point to remember: apply your VLAN configurations after your trunking configuration, or else you’ll just have to do it all over again.

With this information, users should now be pretty well prepared to configure HP ProCurve switches in a VMware ESX environment. Feel free to post any questions, clarifications, or corrections in the comments below, and thanks for reading!

Tags: , , , , ,

Hi Scott,

Do you enable flowcontrol on the ProCurve switches. I have had some issues with IP storage, ESX, and flowcontrol on HP Procurves. My research came up with references to some procurves using a poor/non-standard implementation of flow control. Do you have any experience with this?

Does this work with ESXi as well ?

Francois,

I haven’t tested it with ESXi, but I don’t really see any reason why it wouldn’t work on any vSwitch other than vSwitch0. To change vSwitch0, you generally need command-line access, which of course isn’t possible with ESXi. In addition, you can’t use the Remote CLI because of network connectivity.

Otherwise, it should work fine.

Good luck!

Fransois

I currently have it running with ESXi and it works just beautifully. you can change the Vlan of vSwitch0 using the configurations menu on the ESXi.

Happy hunting.

Can you do link aggreation across multiple switches? In my scenario I am looking at a HP server wth 8 nics, 4 application and 4 storage. Using two HP ProCurve 3500YL’s, can I create one team for the application ports and spread them out over two switches? I know you can with Cisco when they are stacked, but I don’t know about doing this with HP???

Mark,

Though I haven’t tried it yet i think you can “Mesh” the two switches. once that is done I believe you can perform functions across switches.

Mark

You need to buy a Premium License for both 3500yl’s then you can use VRRP.

We have a Dell R900 with 4 onboard Broadcom Nics. We need to know hwo to disable ‘flow control’ on those nics. I am not a linux/unix guy, so please provide command that would allow this. Thank you in advance.

I just got ESXi set up, bought a Procurve 1800-8G as a result of this post, but the setup directions are still unclear to me.

I have an ESXi setup with 2 Intel Pro 1000 GT NICs. They’re both set up as VSwitch0, and they appear to be working well.

I’m still not entirely clear on what I must do on the HP Procurve 1800-8G side in order to get it set up correctly. I’ve tried turning on Trunk1 and associating it with ports 7&8 (where the 2 NICs for the ESXi is plugged into) and I then lose connectivity with the rest of my network.

The PC1800-8G setup is back to straight defaults except having jumbo frames turned on. What must be done first? Is it possible to simply post a step-by-step guide?

@John Smer

Just a guess, but check your VLAN configuration on “TrkX” after it is configured.

This is a great post, but there is a major GOTCHA which I couldn’t find anywhere else. You MUST set the trunk on EVERY VLAN on which it will receive traffic! So if you want the trunk to receive traffic for VLAN 5,6, & 7 then first setup the trunk, then add it to each VLAN with the ports tagged.
The command would be:
configure
vlan 5 tagged Trk1
vlan 6 tagged Trk1
vlan 7 tagged Trk1 (Use your own Vlan-id’s and trunk names of course)

This article is also extremely helpful
http://docs.hp.com/en/J4240-90039/apds01.html

I hope I just saved someone hours of work!