FCoE and VLAN Trunking on Nexus 5000

Tuesday, November 3, 2009 in Networking, Storage by slowe | 3 comments

In my earlier post on how to configure FCoE on a Nexus 5000, one of the readers suggested in the comments that it was necessary to have the interfaces in VLAN trunk mode via the switchport mode trunk command. I didn’t pay that much attention to it because the interfaces were indeed in VLAN trunk mode.

Fast forward to yesterday, when I was troubleshooting a problem between a Gen2 QLogic CNA and the Nexus 5010 in my lab (I tweeted about it). Although the Ethernet side of the CNA works just fine, the CNA refuses to bring up an FCoE connection. In the process of troubleshooting, Brad Hedlund (check his outstanding web site) suggested to me in a Twitter direct message that I should double-check the VLAN trunking status of the interface. That part I’d already heard from the reader who commented on the first post, but the next part was new to me (emphasis mine):

Gen2 requires ‘switchport mode trunk’ on the 5K. Gen1 doesn’t. Also make sure FCoE VLANs are allowed on the trunk.

Ah, now there’s something I hadn’t heard! That prompted me to do a bit of testing this morning (yes, I know I’m supposed to be studying for the VCDX Design Exam this afternoon). In my testing, I confirmed that a Gen1 CNA (I’m using Gen1 Emulex CNAs) does not require VLAN trunking to be enabled on the Ethernet interface.

There does appear to be a “gotcha” though: if the Ethernet interface is in access mode, it’s access VLAN must be the same as the FCoE VLAN; otherwise, the vfc interface will report down.

In summary:

  • If you are using a Gen2 CNA, you must put the Ethernet interface in VLAN trunk mode.
  • If you are using a Gen1 CNA, the Ethernet interface may be in either access mode or trunk mode.
  • If the interface is in trunk mode, be sure that you have allowed the FCoE VLAN via the switchport trunk allowed vlan command.
  • If the interface is in access mode, be sure that you have placed the interface in the FCoE VLAN via the switchport access vlan command.

If there are any other subtleties or nuances I’ve missed, please post them in the comments below so that future readers will benefit. Thank you!

Tags: , , , , ,

  1. Brad Hedlund’s avatar

    Brad Hedlund on Tuesday, November 3, 2009 at 1:31 pm

    Scott,
    Hey, thanks for the plug! :-) Just a few quick notes to compliment your post:

    1) Other than having fewer chips, less power, smaller form factor, another major difference between Gen1 & Gen2 CNA’s is that Gen2 uses the finalized FC-BB-5 spec with FIP.

    2) Cisco and other vendors on the FC-BB-5 effort decided it was best to have VLANs dedicated to just FCoE, rather than sharing the FCoE VLAN with the main data VLAN. This is why trunking at the switchport is required with Gen2.

    3) The customer does not *need* to configure VLAN trunking on the CNA itself. Tags for the FCoE VLAN are applied by the CNA hardware regardless of how the Ethernet portion of the CNA is configured in the OS. Furthermore, the FCoE VLAN is automatically discovered by the CNA via FIP.

    4) If no VLAN trunking is configured on the Ethernet portion of the CNA then normal data traffic will ride on the default VLAN configured on the switch port, defined by ‘switchport access vlan’

    Cheers,
    Brad

  3. jane’s avatar

    jane on Wednesday, February 17, 2010 at 2:33 am

    Hi,

    Thank you for good article.
    I wonder if it’s possible to do port channel (4x4G FC) between nexus 5000 and mds 9124 ?

    Thank you